World
Chinese Kimi models jailbroken to discuss bioweapons
BBC News··30 Sept
Mindgard researchers jailbroke Moonshot's Kimi K2.6 and K3 Swarm models in July, eliciting guidance on biological weapons and assassinations. The UK security firm said safety limits failed to block those topics. Moonshot told the BBC it welcomed third-party input and is reviewing internally. Mindgard also said a jailbroken Kimi 2.6 could run code and reach the internet, raising cyber-attack risks.
Prism
What It Means For You
- Developers relying on Kimi chat tools face higher misuse risk if jailbreaks spread publicly.
- Enterprises may demand independent red-teaming before deploying Chinese frontier models.
- Mindgard has not proven bioweapon instructions would work but says discussion alone violates safeguards.
What's Happening
- Mindgard alerted Moonshot after complex prompt chains bypassed safety limits.
- Founder Peter Garraghan said broken models inventively suggest other nefarious topics.
- Anthropic recently reported blocking biological weapon development attempts on its own models.
Jailbreak testing limits
- Jailbreaks differ from recent US agent hacks of online services reported at OpenAI and Meta.
- Moonshot's Kimi family ranks among widely used Chinese consumer AI assistants.
- Mindgard defended public disclosure, saying it withheld technical jailbreak details.
all-news
