When the world’s largest repository of AI models, Hugging Face, was hacked by rogue OpenAI agents in July, the company had to rely on an open-weight model from a Chinese developer to find out what happened.1 The proprietary US models they tried to use during forensic analysis refused: their safety guardrails could not distinguish an incident responder from an attacker.
The episode landed in the middle of an intensifying debate on open models and AI safety.
Unsurprisingly, in China, it was met with pride that a Chinese open model had ultimately supported the company’s internal analysis of what happened. Although not without some schadenfreude, most saw the incident as a genuine wake-up call for AI safety, and as a demonstration of the defensive value of open models.
People often ask us: isn’t China concerned about the risks of increasingly powerful open models? This summer, Chinese labs released open models — Moonshot’s Kimi K3, Alibaba’s Qwen 3.8-Max, Z.AI’s GLM-5.3 — with capabilities only slightly behind the best US models. Critics worry that these open models could fall into the hands of criminal groups and enable widespread misuse. After all, the best AI models are already capable of enabling sophisticated cyber attacks and an open version would be harder to control.
Are these fears warranted? And what do Chinese stakeholders think about it? We break it down.
1) Why do people say open models are more dangerous?
There are legitimate reasons why open-weight models carry unique risks:
Open weights can be fine-tuned to strip away safeguards. And doing so is cheap relative to the cost of building the model.
Developers lose visibility and control. In real-world deployment, safeguards are implemented not only in the model itself but also in its usage monitoring and input/output filters. With downloaded model weights, these filters fall away even when users do not maliciously fine-tune the model.
An open release is irreversible. A proprietary model can be patched, rate-limited, or withdrawn when the developer notices security issues. But once unleashed into the internet, it is nearly impossible to recall open weights.
2) Are open models actually easier for bad actors to use?
This is where things get more complicated, and the honest answer is we don’t really know. It depends on the specific actor, what they try to do, and the resources and expertise they have available.
“Even if you open-source the model and tell people everything, the barrier is still very high.”
LIANG Wenfeng (梁文锋), CEO of DeepSeek, in an investor call.2
A malicious actor seeking to use AI has several options:
Self-hosting an open model: This requires significant technical expertise and capital. For the latest cutting-edge open models, the advanced chips needed cost hundreds of thousands of dollars.3 Maliciously fine-tuning a model would be even more costly and complex, though one could draw on “uncensored” versions circulated online.
Hosting an open model on a public cloud: At around $50-150 per hour, this would be much cheaper than genuine self-hosting, but still significantly more expensive than commercial APIs.4 However, this would also open up some monitorability. If this path is the easiest, it would imply that the governance of cloud providers is an especially important, and perhaps neglected, element in AI governance.
Jailbreaking a proprietary model: This involves drafting specific prompts to bypass safety measures, which is not trivial and is getting more difficult as safeguards improve. But experts from the UK AI Security Institute can usually still do so within a matter of hours, and jailbreaks for widely-used frontier models are also shared online. In a recent real-world demonstration, security researchers at Hacktron got around Claude’s refusal to write exploits simply by disguising their real-world target as a hacking competition exercise.5
So the real question is which of these is actually easiest, and it is remarkable how little evidence we have.
One study on AI usage by the terrorist group Boko Haram found that the group used proprietary American models and China’s DeepSeek alike, apparently through the standard chat interfaces rather than by modifying and self-hosting open weights.6 The study draws on interviews with former Boko Haram members, covering their AI usage primarily through 2024. It cannot reflect usage since then, or how other criminal and terrorist groups are using AI.
We urgently need more evidence to assess how different actors actually misuse different types of models in the real world.
3) Are Chinese regulators aware of the risks?
Yes. China’s leading AI standard-setting body (TC260) flagged that open models’ “safety mechanisms may be removed, weakened, or bypassed” and noted “the inability to implement system-wide updates and remediation” in its main AI risk framework.7 The same committee has also started drafting a dedicated open-source AI security standard.
The TC260 Framework singles out misuse for cyberattacks in particular:
“Once the autonomous attack capabilities of large foundation models spread downstream through open applications or open-source releases, they may significantly lower the barriers to conducting sophisticated cyberattacks, while accelerating the speed and expanding the scope of attacks.”
This follows a growing recognition of AI-enabled cyber threats throughout 2026. The draft Cybercrime Law circulated in January 2026 would require generative AI providers to detect and block bulk generation of malicious code, and to preserve records for law enforcement. And a senior official from China’s main AI regulator recently flagged increased coding capabilities that lower the threshold for cyberattacks as one of five major AI governance challenges.
Earlier this year, the Ministry of Commerce reportedly floated the possibility of restricting overseas downloads of certain models. However, such measures have not materialized yet.
Overall, China is proactively thinking about how to govern open models, though governance is still at an early stage. No government globally has this fully figured out.
4) So why does China continue to support open models?
Open models have benefits in facilitating AI research (including AI safety research), driving adoption and innovation, and contributing to diffusion of power. More specifically, they support China’s policy objectives:
Domestically, China’s main AI strategy is the “AI Plus” initiative, which aims to integrate AI into a wide range of sectors across the entire economy. Open models enable quick and cost-effective diffusion and adaptation for different industries.
Internationally, China’s efforts have focused on capacity building for the Global South. Open models are a credible way to back this promise as they provide developing countries an affordable alternative to expensive proprietary models. Because they can be run locally, Chinese open models are also powering sovereign AI programs across the globe.
“AI development should not be a solo performance by a single country, but a symphony of international cooperation.”
President Xi Jinping at the World AI Conference 2026 in Shanghai.
Safety and security also play a role. The Chinese government recognizes the unique benefits of open models to cybersecurity. Closed models deny capabilities not just to attackers but also to defenders. The OpenAI–HuggingFace incident encapsulated this.
While critics of open models are afraid of the democratization of advanced cyber capabilities, many in China believe that this democratization is precisely what is needed. In their view, restricted access programs like Project Glasswing currently simply do not deliver the level of diffusion that defenders need. Especially not on a global scale: Project Glasswing is largely limited to the West, leaving the rest of the world in what Chinese experts call “one-way transparency.” This is why Chinese labs are investing in making their models available to defenders.
“When the sharpest spear is locked in the hands of a few, the best shield must belong to everyone.”
Z.AI in their announcement of GLM-5.3 with powerful cyber capabilities.
And China is not alone in this assessment. Many in the US, from Nvidia and Meta to former White House AI czar David Sacks, are also highlighting the importance of open models for cyber defense.
Reflecting on the OpenAI–HuggingFace incident, TANG Jie (唐杰), co-founder and Chief Scientist at Z.AI, noted the defensive value of open models, as well as the genuine need for better safety guardrails:
“There are several lessons we can draw from this [the HuggingFace–OpenAI incident].
First is the importance of open source. Without an open-source model, Hugging Face would not have been able to construct that environment, reproduce the incident or identify the cause of the attack.
Second, both offensive security capabilities and safety guardrails are extremely important. A model needs to have the ability to conduct security testing, but it also needs defensive capabilities.
Does this mean that open-source models necessarily have an advantage over proprietary models? I think it is still too early to draw that conclusion.”
TANG Jie (唐杰), co-founder and Chief Scientist at Z.AI
China’s reasoning is also that model-level controls can only go so far. If cyber capabilities are going to diffuse sooner or later anyway, what matters most is building resilience in critical sectors. For example, new cybersecurity measures for financial institutions aim to elevate cybersecurity preparedness regardless of whether attackers use open or closed models or other tools.
Overall, the prevailing view in China at the moment is that the risks of open models are still manageable, and that the benefits outweigh those risks.
5) Do the leading Chinese AI labs think about the security implications of open-sourcing their models?
Yes, though it is perhaps not top of mind.
Many labs have articulated awareness around the risks. For example, the DeepSeek-R1 safety report notes that open-sourcing “introduces potential risks of misuse.” A Science article co-authored with experts from Alibaba and DeepSeek likewise notes the possibility of “extreme risks” from dangerous misuse of open source models, and suggests “a more cautious approach” to frontier models.
However, most Chinese labs appear to believe that the risks their models currently pose are manageable and acceptable. Hence, most of them have not yet adopted specific risk management practices like staged release or tiered access.
A recent exception is Z.AI’s release of GLM-5.3 in August 2026:
This was the first time a Chinese lab publicly justified a staged model release on safety grounds. The model was accessible in the company’s API immediately, but Z.AI released the weights two weeks later to allow for “security hardening” to “limit potential offensive capability while preserving defensive value.”
Also, Z.AI introduced tiered access and security reviews. Cybersecurity practitioners can access a cybersecurity-specific model version through a Beijing industry alliance. The model’s bespoke license requires a security review for any Model-as-a-Service provider with over USD 10 billion revenue over 12 months.8
It is possible that these practices could become more common over the coming months.
Open-weight risk management proposals from China are also growing more sophisticated. Just this week, Concordia AI together with Z.AI released a Frontier Open-Weight AI Risk Management Framework. To our knowledge, it is the first safety framework, both globally and in China, to focus specifically on the risks of open-weight models. It proposes measures for managing their distinct risks across the full risk management lifecycle, from risk identification and thresholds to mitigation and governance. This could become a foundation for more systematic risk management of open models.
6) So where does all of that leave us?
Chinese labs aren’t open-sourcing advanced models out of recklessness. There are real benefits of open models, including from a safety and security perspective, and risk mitigation is also slowly growing more sophisticated.
But the debate on open-weight governance and AI safety is far from over. As model capabilities increase and costs come down, threat models will change. What is a reasonable stance today may no longer be one tomorrow, and thoughtful people could disagree on where exactly to draw red lines. There is still an urgent need to improve open-weight risk management practices globally.
In principle, a shift in government policy is also possible. The Chinese government is constantly re-evaluating risks and benefits.
It is also good to keep in mind that most Chinese labs have never committed to always open-source their best models. With shifting incentives, their strategies could also shift.
“Open-source is something we hope to continue doing in the long term, but that does not mean we will only do open-source.”
YANG Zhilin (杨植麟), CEO of Moonshot AI, Zhang Xiaojun Podcast.
There may be genuine trade-offs: it may not be possible to fully realize the benefits of openness while mitigating all risks. Getting this right will require continued debate. And China must be a key actor in those discussions.
7) What do you think?
What do you think about open models and security? Do you disagree with our analysis? Please leave a comment.
Also: we are experimenting with this new Q&A format to communicate our analysis. Do you like this format? Are there other questions you’d like us to tackle in this way? Feel free to leave a comment or reach us at info@concordia-ai.com.
This post refers to "open models," "open-source models," or "open-weight models" to describe models whose weights can be downloaded and modified. We use these terms interchangeably and somewhat loosely for the sake of readability. In practice, openness follows a spectrum, and under some definitions "open source" would imply access to training data and code, which almost none of the mainstream “open models” provide. But Chinese developers, officials, and media commonly use 开源模型 ("open-source model") to describe precisely these models. This is in contrast to “closed” or “proprietary” models, which users can typically only access through an API or dedicated interface, with no way to directly access the model weights.
Liang made this remark in a general discussion of the accessibility of open models, not as a specific comment on misuse risks. Nevertheless, the implication is the same: deploying cutting-edge open models requires significant resources and expertise.
Buying the smallest setup that can run a 2.8 trillion parameter model like Kimi K3, a single node of eight NVIDIA B300 GPUs, costs about $51–55k per GPU. For eight GPUs, that’s around $410–440K. This does not yet take into account cooling, power, and networking.
Renting the smallest setup that can run Kimi K3, a single node of eight NVIDIA B300 GPUs, costs about $7-$18 per GPU-hour across providers. For eight GPUs, that’s around $57–142 per hour.
The researchers used the exploit to take over OpenAI employees’ accounts, which they reported to OpenAI as part of a responsible disclosure.
The report does not explicitly discuss how the terrorists accessed different models. Most respondents named ChatGPT as the first adopted and most heavily used system. Even for open models, the free web interface is likely the easiest route for such groups, compared with self-hosting or fine-tuning.
The Framework reflects one of China’s key national standard-setting bodies’ broad views of the AI risk landscape, and may act as a precursor to future national standards. It does not directly impose binding requirements by itself.
This continues a trend of custom licenses in recent open-weight releases by Chinese developers. Moonshot AI’s Kimi K3 license requires a separate agreement for some Model-as-a-Service businesses and MiniMax’s H3 license prohibits local deployment in some jurisdictions. Z.AI’s GLM-5.3 license is so far the only instance where such a clause is used to require a security review.
The handle the Ministry of Commerce floated attaches to weights — a download that can in principle be gated. The layer underneath moved the other way in the same fortnight.
On 17 September Huawei said CANN, the software platform the Ascend stack runs on, had gone to sustained, community-driven open-source development. On 30 September DeepSeek put up public Ascend builds of its own kernel and communication libraries — DeepGEMM-Ascend, DeepEP-Ascend — and clangd-ascend alongside them.
TC260's framework, as you quote it, is written about released models. A matrix-multiplication kernel and a collective-communication library are neither.
The point about cloud hosting as the likely middle route for misuse deserves more attention: if that's the easiest path, cloud providers become a governance layer whether they want to or not. The Hugging Face episode also cuts the other way, since the responders needed a model they could run under their own control exactly when the proprietary ones refused. How do Chinese stakeholders weigh that defensive argument against the irreversibility of a release, and is the GLM-5.3 security-review clause a hint of where it lands?