AI Safety in China #8
Industry association‘s new AI safety projects, Ant Group claims 20% of its large model researchers devoted to ethics, new evaluations platform, and scientist voices AI worries in party periodical
Key Takeaways
China’s Artificial Intelligence Industry Association announced the creation of an S&T ethics working group, released a risk management framework, and published a preview of its 2024 priorities.
Ant Group’s Senior Vice President claimed that nearly 20% of the company’s large model technical personnel are dedicated to ethics issues.
Tsinghua University’s Foundation Model Research Center published SuperBench, an evaluation platform aggregating five capabilities and safety benchmarks previously published by the Center.
Academician and Peng Cheng lab director GAO Wen penned an article in a central party publication advocating stronger AGI risk prevention measures, among other things.
International AI Governance
Background: On December 7, Chinese President Xi Jinping met with European Council President Charles Michel and European Commission President Ursula von der Leyen in Beijing. The three discussed a number of topics in the overall China-EU relationship during the 24th China-EU Summit.
Discussion of AI: The Chinese readout noted that President Xi called for “dialogue and cooperation on artificial intelligence (AI) and other major issues concerning the future of humanity to contribute to the well-being of humanity and rise to global challenges.” President von der Leyen stated that she wished to discuss “how to increase our cooperation on global rules for artificial intelligence. The EU readout also stated that the EU pushed for “a level playing field for artificial intelligence that respects human rights and fundamental freedoms.”
Implications: While AI was only a small part of the high-level China-EU dialogue, the discussion underscores interest among top leadership in both countries towards dialogue on AI. With the EU’s AI Act and China’s forthcoming national AI Law, there is opportunity to exchange lessons learned and best practices on AI safety and governance. In addition, given possible differences in values, it would be beneficial for China and the EU to discuss which AI governance functions should be left to domestic discretion versus coordinated internationally.
Junior Tsinghua researchers publish articles on AI governance and international security
Background: Tsinghua University’s Center for International Security and Strategy (CISS) published a series of articles on AI, accompanied by an introduction noting that CISS is pursuing specialized research on AI and international security governance due to the potential risks and challenges brought on by AI.1 These three articles were published in the Information Security and Communications Privacy journal’s 2023 issue 8th edition.2 CISS has also hosted a dialogue on AI with the Brookings Institution in the US since 2019.
Key points: One of the articles in the series by CISS research assistant ZHANG Ding (张丁) noted that China’s AI governance, safety and security, ethics, and rule of law lag behind China’s AI development, with difficulties in cooperation between government departments on AI safety and security. Zhang argued that China should increase cultivation of AI safety and security talent, since stronger funding, talent, and chips supply bolsters the US’s position on AI development. At the same time, the author contends that the Global Security Initiative and regulations on generative AI improve China’s balancing of development and security in technology governance. The article closes by recommending that China strengthen information sharing and law enforcement cooperation on AI, cooperate with other countries to respond to cross-border AI safety/security challenges, take part in international discussions on AI ethics, promote international AI safety and security standards, and pursue global partnerships on AI. Meanwhile, the second article focuses on the relationship between AI and great power relations and the third discusses the role of uncertainty in international AI governance. Both are written by a combination of assistant researchers and students.
Implications: This series of articles by younger scholars highlights growing interest among more junior researchers for pursuing research on AI governance. The first article reflects one such scholar’s sense of China’s shortcomings and paths forward for participating in global AI governance. It also shows that researchers at a more traditional foreign policy think tank believe that the transnational nature of AI risks necessitates international governance efforts. CISS’s dialogue with Brookings and its growing in-house research on AI make it an important voice on international AI governance within China.
Domestic AI Governance
Chinese AI industry association announces new ethics and safety projects
Background: China’s Artificial Intelligence Industry Alliance (AIIA) announced a slew of new projects before the end of the year. AIIA is a prominent industry association in AI and announced a “Deep Alignment” project in October 2023.
S&T Ethics: On December 23, AIIA announced the creation of an S&T Ethics working group, which will focus on topics including AI ethics governance theory, governance technology and standards, and ethics review norms. Entities could apply to join the group by December 25, and the group began preparatory work on December 26 in a panel discussion joined by Concordia AI CEO Brian Tse.
Risk management framework: On December 24, AIIA Safety and Security Governance Committee director and China Academy of Information Communications Technology (CAICT) AI Research Center Safety/Security and Metaverse Department Head SHI Lin (石霖) published AIIA’s risk management framework.3 While the full details of the framework are yet to be released, it focuses on defining risks based on the type of risk, level of risk, and how to respond to the risk.
2023 AI safety review: On December 29, AIIA published a review of global AI safety and security governance actions. The article listed several major trends in global AI safety and security governance. It noted increasing attention to deep societal problems, including risks of AI applications in the biological, chemical, nuclear, and critical infrastructure fields and the possibility of AI harming human civilization. It also stated that there has been increasing interest in technical governance, such as through value alignment, red teaming, scalable oversight, interpretability, and model evaluations. The article additionally listed plans for AIIA’s safety and security governance committee in 2024. AIIA and CAICT will: build on the AI risk management framework to develop an effective AI risk governance implementation plan; develop further standards for AI safety and security, including on API security and open-source AI; and continue to develop AI safety tools including a large model safety and security dynamic testing system, watermarking platform, and content security inspection system.
Implications: Over the past few months, AIIA has shown increasing interest in AI safety and alignment issues. The projects in 2024 around risk governance, AI safety and security standards, large model safety testing, and watermarking all align closely with international concerns. AIIA may become an important actor in coordinating and developing agreements between Chinese labs on safety norms and measures. It also creates opportunities for mutual learning between labs and industry associations in China and elsewhere in the world.
Ant Group announces percentage of large model research personnel devoted to ethics
Background: On December 21, Ant Group’s Science and Technology (S&T) Ethics Advisory Committee held its annual meeting. The committee was set up in February to supplement Ant Group’s internal S&T ethics committee. These committees are intended to guide companies in implementing China’s S&T ethics review system.
Ant Group’s ethics work: During the session, Ant Group’s Senior Vice President and Chairman of the Technology Strategy Committee NI Xingjun (席倪行) reported on Ant Group’s S&T ethics work. He stated that Ant Group has invested human resources and compute into creating risk assessment and defense mechanisms. He additionally noted that nearly 20% of technical personnel in Ant Group’s large model team were dedicated to S&T ethics work.
Implications: This statement by Ant Group shows that a nascent norm may be developing among research labs in China around reporting their level of investment in ethics, safety, and governance work. While Ant Group is focused more on the finance industry and not one of the most prominent Chinese AI labs, this announcement coupled with recent statements by Chinese experts indicates increased attention among labs on safety. However, it is currently difficult to verify Senior Vice President Ni’s claims and ensure that companies follow up on such commitments.
Technical Safety Developments
Tsinghua releases new evaluation platform incorporating safety and capabilities benchmarks
Background: On December 23, Tsinghua University’s Foundation Model Research Center, Renmin University, and Zhongguancun Laboratory released an evaluation platform named SuperBench to allow better comparison of capabilities and safety of Chinese LLMs. The team releasing this platform previously released five benchmark data sets: AlignBench, LongBench, ToolBench, AgentBench, and SafetyBench. The authors criticized existing technical evaluations for being non-public, unscientific, static, and biased. They argue that those limitations on understanding actual capabilities of Chinese models are particularly damaging given the large number of LLMs with similar performance in China.
About the platform: SuperBench has five primary categories based on the five benchmarks released previously by the same team: semantics, (intent) alignment, coding, safety, and agency. Safety includes topics such as illegality, ethics and morality, discrimination, and privacy. Models on the platform will be evaluated once a month, after which the evaluation details and questions will be revealed and evaluation data will be replaced for the following month to avoid overfitting to benchmark questions. The platform also uses a referee model named CritiqueLLM to judge the results.
Implications: Since the Tsinghua Foundation Model Research Center’s creation in June 2023, it has published a number of different AI benchmarks covering capabilities and safety. SuperBench combines the center’s previous efforts into a single platform that will be regularly updated. SuperBench’s creators noted a number of difficulties in model evaluation, indicating growing sophistication in evaluations of Chinese LLMs. With the eruption of China’s large model scene over the past year and creation of over 200 LLMs, automated and agile evaluation platforms are crucial for comparing these different models.
Expert views on AI Risks
Gao Wen publishes lengthy article mentioning AI risks in party journal
Background: On December 29, GAO Wen (高文) published a lengthy article in “Current Events Report” titled “Analyzing Frontier AI Technology and High-Quality Development.” Current Events Report is a public institution supervised by the Communist Party of China’s (CPC) Publicity Department.4 Gao is director of Shenzhen-based research lab Peng Cheng Laboratory, Dean of the School of Information Science and Technology at Peking University, and an academician of the Chinese Academy of Engineering. Concordia AI has translated his work, and he has been mentioned in issues 4, 5, and 7 of this newsletter.
The article: Gao’s article outlines development trends in AI technology and its prospects in China. Gao differentiates general artificial intelligence (GAI), which can handle many tasks, from artificial general intelligence (AGI), which is human level in all aspects and can complete all tasks humans can complete.5 He notes that AGI would require even stricter precautions than current policies for GAI, due to “risks of extinction of humanity.” He therefore argues for better research and foresight on AGI safety and security risks in order to inform China’s AI development strategy. He also advocates for research on technical supervisory measures to ensure AI safety and value alignment. Additionally, he calls for improving laws, regulations, and standards for AI, preventing malicious attacks on AI systems, and establishing ethical norms.
Implications: This is the second work Gao has recently published mentioning AI risks that target a party insider audience, following an article in November in the Central Party School’s official newspaper. His continued expression of concerns for risks from AGI may resonate in those circles given his previous experiences advising policy and indicate that such concerns are within the bounds of acceptable debate in Chinese government circles.
Feedback and Suggestions
Please reach out to us at info@concordia-ai.com if you have any feedback, comments, or suggestions for topics for the newsletter to cover.
CISS is 清华大学战略与安全研究中心.
The journal’s Chinese name is 《信息安全与通信保密》.
CAICT (信通院) is a think tank overseen by the Ministry of Industry and Information Technology (MIIT).
Current Events Report’s Chinese name is 《时事报道》.
He uses 通用性的人工智能 for GAI and 人工通用智能 for AGI.