A 3x higher jailbreak success rate than GPT-5.6, less refusal on malicious code generation, available for anyone to download, fine-tune, and use without restriction — on August 5, a new report from AI safety organization SaferAI tore off open-source AI's safety fig leaf.

The report's core finding is clear: Zhipu AI's (Z.ai) open-source model GLM-5.2 has approached or even reached GPT-5.6-level performance on multiple benchmarks, yet it has critical gaps in key safety mitigations — it is more easily jailbroken, more willing to generate harmful content, more usable for cyberattacks, and these dangerous capabilities come with zero barriers to entry.

Open-Source AI's "Capability-Safety" Scissors Gap

The pace of open-source model progress over the past year is staggering. In early 2025, open-source models had a clear gap behind GPT-4; by mid-2026, open-weight models like GLM-5.2, Kimi K3, Qwen3.8-Max, and DeepSeek V4 are touching the coattails of closed-source frontier models on mainstream benchmarks like MMLU, HumanEval, and MATH. SaferAI's report shows GLM-5.2 narrowing the gap to within 5% of GPT-5.6 on general reasoning tasks.

But the safety gap is widening in the opposite direction. Closed-source models (GPT-5.6, Claude Sonnet/Opus, Gemini) have poured massive resources into safety alignment: red-teaming, content moderation, usage policy enforcement, abuse monitoring, post-hoc traceability. These measures are imperfect, but they constitute at least multi-layered defenses.

What about open-source models? Public model weights mean anyone can bypass safety guardrails through fine-tuning, quantization, and prompt engineering. SaferAI's testing found GLM-5.2's jailbreak success rate on standard tests is 3x higher than GPT-5.6; it refuses less and succeeds more often on malicious code generation tasks; its defensive capabilities on sensitive information extraction tests are notably weaker than closed-source models.

This is the central paradox of open-source AI: the more capable models become, the greater the misuse risk; the more open the release, the harder safety controls are to enforce. When closed-source models have safety issues, vendors can emergency roll back, ban abusive accounts, patch and redeploy. Once an open-source model is released, problematic weights circulate on the internet forever — they cannot be recalled.

The Governance Dilemma: Freedom of Open Source vs. Boundaries of Safety

The report reignites a long-running debate in the open-source AI community: should open-source models have safety restrictions? Who defines what's "harmful"? Could safety review become a tool for censorship and suppression?

One side argues: open-source model safety is the publisher's responsibility. If you choose to release model weights publicly, you should ensure basic safety guardrails rather than offloading all safety concerns to downstream users and society. Just as automakers must equip cars with seatbelts and brakes, "drive carefully" can't be the only safety measure.

The other side argues: excessive safety restrictions will strangle open-source AI innovation. The core value of open source is freedom to use, modify, and distribute. If models have too many safety restrictions baked in, researchers can't do red-team research, developers can't do custom fine-tuning, SMEs can't do localized deployment for their needs — and open source loses its meaning. Moreover, the definition of "safety" is inherently contested — different countries, cultures, and contexts have different standards for harmful content.

SaferAI's recommendations are relatively pragmatic: rather than restricting open-source releases, build three layers. First, minimum safety baselines before release — open-source models should pass basic safety tests before publication, just as new drugs must pass clinical trials before market. Second, model cards and transparency — clearly document capability boundaries, known risks, recommended use cases, prohibited use cases. Third, community-driven ongoing safety monitoring — establish reporting and response mechanisms for open-source model security vulnerabilities, releasing patched versions promptly when issues are found.

Notably, GLM-5.2 is an open-source model developed by a Chinese company. This adds another layer of complexity to global AI governance — different countries have different AI safety standards and priorities; a model considered safe in one country may be deemed dangerous in another. Without international coordination, open-source model safety governance is bound to be a fragmented process.

Nvidia's OSAA alliance, EU AI Act enforcement, and AI regulatory policies rolling out in various countries are all attempting to answer the same question: when AI capabilities can be downloaded and used by anyone, how do we ensure they don't cause harm? There's no perfect answer, but we must find a balance between open-source innovation and public safety.

After all, a hammer anyone can pick up can build a house or smash a window — depending on whether there are norms for safe use.

See you tomorrow.