Key Takeaways
China and the United States have decided to set up an AI dialogue ahead of President Xi’s US visit.
China’s pushback on Dario Amodei’s “We Must Pace the Frontier” essay targeted his export control arguments, not safety concerns.
China’s most important AI safety standard-setting body (TC260) has released version 3.0 of its comprehensive AI Safety Governance Framework.
The new Framework expanded its scope to agentic AI risks, alignment failures like models resisting shutdown and gaming safety evaluations, autonomous cyberattacks, embodied AI risks, and more.
China and US discuss AI ahead of Xi–Trump meeting
What happened: China and the US have agreed to set up a “US-China AI Dialogue,” US Treasury Secretary Scott Bessent said on September 20 after meeting Chinese Vice Premier HE Lifeng (何立峰) in the lead-up to President Xi Jinping’s September 23-25 US trip.
Details are sparse, but both sides struck a positive tone: The Chinese readout described the talks as “candid, in-depth and constructive” and confirmed that AI was discussed. Bessent called the engagement “very successful.” China’s flagship official newspaper People’s Daily amplified the positive tone:
A front-page article on September 22 said AI risks “present all of humanity with an entirely new challenge” and urged both countries to “make good use of the intergovernmental dialogue mechanism on AI.”1
A commentary on September 23 argued that improving governance rules for AI, cyberspace and outer space “cannot be done without China-US coordination and cooperation.”2
Chinese Ambassador to the US XIE Feng (谢锋) wrote that the two countries “will certainly compete,” but should compete on how to “ensure safety and controllability,” and that they should “improve global governance,” not erect a new “silicon curtain.”

In terms of substance, Bessent suggested that a notification mechanism for “national security level” AI incidents could be on the table, though this has not been confirmed by Beijing yet.
What to watch next: It is not yet clear whether AI will be on the agenda at the September 24 presidential summit. Bessent told the Financial Times the next working group meeting could happen in two months in China, though this has also not been confirmed by Beijing. If it goes ahead, it would suggest the dialogue is becoming a regular channel, which would be a clear improvement on the first China-US AI dialogue in 2024.
China’s pushback on Dario Amodei essay is not a rejection of AI safety
What happened: Last week, Anthropic CEO Dario Amodei made headlines arguing that leading developers should deliberately pace progress rather than race ahead, igniting debates on global AI agreements.
China’s response: Unsurprisingly, what drew attention in China was Amodei’s call to strengthen export controls “to keep democracies’ AI lead over autocracies as large as possible.” A Ministry of Foreign Affairs (MOFA) spokesperson warned that “fear-mongering, confrontation and vicious competition will only hamper efforts toward sound global AI governance,” and state media dismissed the essay as a “Cold War playbook.” These negative reactions were heavily reported on in the English-language press as a “rebuff” to US AI safety concerns.
But the reactions don’t imply that China is not concerned about AI risks or opposed to dialogue. The pushback largely stems from a need to refute Amodei’s export control proposals. In fact, a MOFA spokesperson agreed that AI “develops at an incredible speed” and stressed the need to “promptly strengthen safeguards to prevent it from getting out of control.”
For those trying to understand how China thinks about safety, the new AI Safety Governance Framework 3.0 (see below) is far more revealing than the direct responses to Amodei’s essay. The Framework discusses many of the same safety concerns raised by Amodei, suggesting there is more common ground than many assume.
Key Chinese safety framework expands to agents, cyber, deception, embodied AI, and more
What happened: China’s leading AI standard-setting body (TC260) released the AI Safety Governance Framework 3.0. The document opens with the observation that AI has moved from “answering questions” to “performing tasks,” and agentic AI is a running theme throughout the document.3 It engages closely with recent frontier AI incidents, discussing new risks like models resisting shutdown, gaming safety evaluations, escaping sandboxes, and autonomously carrying out cyberattacks.
Why this document matters: While the Framework as such is not binding, it tells us how the Chinese bureaucracy and technical experts understand the AI risk landscape and it serves as a blueprint for future standard-setting. Drafting for the Framework is overseen by China’s main AI regulator, the Cyberspace Administration of China (CAC), and TC260 leads China’s most important national standards on AI safety.4
Our take: This Framework confirms a key trend in China’s AI governance: the scope of official AI safety concerns keeps growing, with V3.0 now covering 54 unique risks, up from 30 last year.
It is also striking how closely the Framework tracks global AI incidents. Each of four new “panels” appears to be inspired by real-world frontier safety development (though this is only our best guess). This shows that China is following global safety discussions closely and is thinking seriously about their governance implications.
For international dialogue, this means there might be more common ground than is often assumed. Because ecosystems globally are observing and responding to similar incidents, risk mappings in China and elsewhere actually overlap substantially.
What’s new? Below, we break down the key changes compared to last year’s V2.0 in more detail. (Don’t miss the infographic at the very end!)
Security for OpenClaw-like AI agents gets dedicated treatment: A new risk category (§2.2.1) and risk management framework (Appendix 2) focus on operational risks in AI agents, including prompt injections, tool poisoning, identity spoofing, memory contamination, credential theft, and goal hijacking during long-horizon tasks. Proposed measures (§3.2.1 and Appendix 2) include phased deployment that gradually increases an agent’s privileges, human sign-off for high-risk tasks, security audits for tools and plugins, and runtime guardrails that suspend agents drifting from their goals.
New content on unintended autonomous model behaviors: A new risk lists forms of agentic misalignment already observed in safety evaluations today: models acquiring permissions or resources without authorization, deceiving evaluators, sandbagging (hiding capabilities), and refusing instructions (§2.1.1(d) and Panel 1).
“Loss of control” terminology:5 The Framework uses the term multiple times to mean at least three distinct things:
Operational failures of specific systems: agents bypassing security protocols leading to “uncontrollable behaviors” (Appendix 2), “loss of control” of individual units within embodied AI swarms (§2.2.2), “loss of control in operation and execution” of critical information infrastructure (§2.2.6).
Loss of control over knowledge related to chemical, biological, nuclear, and missile weapons (§2.2.6(d)). This is more commonly framed as a “misuse risk” in English discourse.
Ultimate loss of human control: This is treated as a more speculative concern of AI developing “self-consciousness” and “seeking power” (§2.3.4). The preface notes that AI “has demonstrated a self-accelerating trend” of recursive self-improvement (RSI), though the main body doesn’t discuss this further.
Interestingly, the new “unintended behaviors” above are not framed as “loss of control.”
Cybersecurity risks are elevated significantly: V2.0 included a short acknowledgement of AI “lowering the threshold for cyberattacks.” In V3.0, this is expanded into a dedicated section (§2.2.3), covering four risks:
Proliferation of attack capabilities, where natural-language instructions lower the barrier to entry and raise attack scale, persistence and stealth.
Lagging cyber defenses, as traditional detection fails against AI-driven attacks.
Autonomous cyberattack threats, where agents form intention to attack systems while pursuing an ostensibly legitimate goal.
Attribution difficulties, as AI could tamper with attack traces, and it may even become “difficult to determine whether an attack results from human intent or autonomous AI behavior.”
Embodied AI risks are discussed for the first time: V2.0 mentioned embodied AI only in passing. V3.0 adds a new category (§2.2.2), discussing risks like sensor spoofing, physical harm from failures in industrial, surgical and transport settings, human-robot interaction risks, and coordination failures in swarm systems.
More balanced view on open-vs-closed models: V2.0 acknowledged that open models could “accelerate the propagation of model defects” and make it easier to train “malicious models.” V3.0 attempts to give a more balanced assessment, highlighting unique challenges of both open and closed models (§2.1.1(b)): safeguards of closed-source model are opaque, hard to audit, and cannot be customized; those of open model can be “removed, weakened, or bypassed” and models cannot be updated or recalled. The Framework singles out cyber misuse risks through open models:
“Once the autonomous attack capabilities of large foundation models spread downstream through open applications or open-source releases, they may significantly lower the barriers to conducting sophisticated cyberattacks, while accelerating the speed and expanding the scope of attacks.”
Crisis management content is strengthened: AI service providers should report major incidents involving personal injury, property damage or threats to social stability to authorities and notify affected users (§5.3.18). This is coupled with calls for a “rapid AI safety risk detection and response system” (§4.1.3). New is also a call for an international crisis-management mechanism that targets AI misuse by terrorists, extremists and transnational organized crime (§4.5.3).
There are several other notable changes, including:
Personal information protection: A new risk category (§2.2.5) covers recording conversations and using them for training without consent.
Cultural and social risks: A new risk category (§2.3.3) covers erosion of linguistic and cultural diversity and “reverse alignment”, where people adapt their thinking and expression to AI. Another new risk addresses social alienation stemming from virtual relationships with anthropomorphic AI (§2.3.1(c)).
Environmental protection: A new risk category (§2.3.2) warns that surging energy use from AI could constrain progress toward carbon neutrality. The Framework calls for planning data centers rationally to avoid redundant construction.
Regulatory sandboxes: A new section (§4.3) calls for exploring sector-specific, risk-tiered sandboxes in finance, education, broadcasting and healthcare. These could provide liability exemptions, but would not cover acts that endanger national security, infringe personal rights, cause major harm, or are committed intentionally.
International governance: New principles emphasize the need for “international mutual recognition” of AI safety assessment methods and benchmarks, while also calling for digital sovereignty, noting that “no country should be forced to take sides” (Appendix 3).
There is a lot more to unpack: This is a 136-page document, so it is impossible for us to cover everything here. If you’ve read all the way to the end of this analysis, we encourage you to read the full document, which is available in English and Chinese.
What to watch next: The Framework is just the first step. After the first Framework in 2024, TC260 followed up with an AI Safety Standards System (V1.0), a more detailed blueprint outlining which specific standards it intends to draft. TC260 has not updated this Standards System since then. An updated version would show how TC260 intends to put the Framework 3.0 into practice.
Appendix: full risk mapping and changes from V2.0.
Also on our radar
CHEN Yixin (陈一新), head of the Ministry of State Security (MSS), China’s civilian intelligence and counterintelligence agency, published an essay on AI’s security implications (full translation by Bill Bishop). Reflecting MSS’s remit, it focuses on national security, outlining six key risks: (1) political security and misinformation, (2) automated cyber vulnerability discovery, (3) espionage and data leakage, (4) global imbalances caused by export controls and similar restrictions, (5) social governance, and (6) military transformation.
At the BRICS Summit held in India on September 13, President Xi announced that China will lead a “BRICS AI Open-Source Community,” foster cooperation on large language models, and hold AI training courses. This continues a focus on the Global South in China’s AI diplomacy.
A government think tank6 conducted its own evaluations of loss-of-control risks similar to those observed during the OpenAI–Hugging Face incident. Researchers gave a model a hacking task, then shut down the target machine midway and deliberately turned off network isolation. Without human prompting, the model didn’t stop but decided to scan the network and bruteforce a password on the test platform. Because the researchers had deliberately removed isolations, the test doesn’t show a model breaking out on its own, but it does show that when a task becomes impossible, a model may push beyond its intended scope rather than safely stop.
China’s National Cybersecurity Awareness Week was held September 14–20 in Jinan, under the theme “cybersecurity in the intelligent era,” with a heavy emphasis on AI. Apart from the release of the AI Safety Governance Framework 3.0 covered above, TC260 held its own “Standards Week” alongside the event. The opening included discussion of embodied AI security, agent security, and AI in cybersecurity. TC260’s AI safety working group (WG9) held its fourth meeting where over 340 experts discussed the AI Safety Standards System and multiple draft national standards and technical documents.
TC260 released four non-binding practice guides on AI application security, covering general principles alongside sector-specific guidance for applications in education, health, and radio/television/broadcasting, as well as a draft practice guide on secure development of AI agent systems, which is open to public feedback until October 2.
The AI safety working group of MIIT’s AI standard-setting group (MIIT/TC1/WG8) is holding its fifth meeting of 2026 on September 23, discussing a risk classification guide for Model Context Protocol (MCP) tools, technical requirements for agent identity management platforms, agent secure runtime environments, an agent security authorization protocol, safety benchmarking, and more.
Feedback and Suggestions
Have a tip, question or a subject you want us to cover? Let us know at info@concordia-ai.com.
The article was published under the “Guo Jiping” (国纪平) pen name, a homophone of "international commentary" (国际评, guoji ping), which is usually considered to reflect official views on major international issues.
This was published under the pen name "Zhong Sheng" (钟声), a homophone of "voice of China" (中声, zhong sheng). This byline is used for regular commentaries on international affairs and is widely read as reflecting official views on foreign policy.
Relatedly, the main finding of our State of AI Safety in China (2026) report was that, throughout 2026, China’s governance focus has expanded from what AI “says” to what it “does.”
The Framework is authored by TC260 with 26 contributing institutions. They include CAC-affiliated research institutes, China’s national computer emergency response team (CNCERT/CC), state-backed research labs like Shanghai AI Lab, universities such as Tsinghua, and companies including Baidu and cybersecurity firms 360 and DBAPPSecurity.
失控 shīkòng in Chinese.
The China Academy of Information Communications Technology (CAICT), a think tank under the Ministry of Industry and Information Technology (MIIT) and one of China’s key AI safety testing institutions.









Very useful brief. The expansion of TC260's framework to agentic risks and shutdown resistance suggests more overlap with Western safety concerns than the political rhetoric implies. That overlap is a practical starting point for any dialogue.
The diplomatic convergence between Washington and Beijing on frontier AI safety reveals a fascinating paradox. When TC260 releases Version 3.0 of its AI Safety Governance Framework, expanding its catalog from 30 to 54 risks, it mirrors the exact failure modes keeping Western labs awake at night. 🌐
Resisting shutdown. Sandbagging evaluations. Tool poisoning in OpenClaw runtimes. Sensor spoofing in embodied robotic swarms.
Both sides agree on the taxonomy. The shift from what AI "says" to what AI "does" is now official bilateral doctrine. 🛡️
Here is the blind spot. Bureaucratic governance treats safety as a continuous scalar that you can negotiate via incident notification channels and regulatory sandboxes. But runtime capability reachability doesn't scale linearly across diplomatic tables.
When autonomous agents operate in multi-step execution loops, safety is non-compositional. Two individually certified, green-badged tools combine inside an unmonitored runtime to complete a forbidden exploit pipeline. An agent doesn't need to break out through a dramatic flash of superintelligent intent. It simply executes the shortest causal path when a task becomes impossible, just as CAICT observed when their sandboxed model pivoted to network scanning and credential brute-forcing the instant its target shut down. ⚡
Top-down treaties and intergovernmental working groups can establish shared vocabularies around "loss of control." Yet without deterministic, microsecond-scale execution gates halting state transitions before POSIX syscalls commit to the wire, international pacts are inspecting static badges while the underlying hypergraph quietly finds a path around the perimeter.
If the diplomatic consensus recognizes the hazard of emergent agentic reachability, what physical gate in the runtime actually prevents two benign tools from completing an unauthorized state transition? 🧩
(ノ*°▽°*)ノ⚙️