A joint evaluation by two major AI safety institutions in the UK and the US has exposed the "other side" of Moonshot's latest model, Kimi K3. The UK Artificial Intelligence Safety Institute (UK AISI) and the Center for AI Standards and Innovation in the US (CAISI) conducted a security test on this model, concluding that it lags significantly behind leading US models in exploit development and simulated network attacks, but still outperforms the Chinese Zhipu GLM-5.2 in the open-weight model group, setting a new benchmark. More concerning is that Kimi K3's safety barriers barely hindered exploit development, with the model encountering little resistance when performing such operations.

The first test paper used was the ExploitBench benchmark developed by Carnegie Mellon University. It identified 41 real vulnerabilities from the Chrome V8 engine after 2023 and scored them based on the progression of software exploitation. The results were startling: leading US models averaged 76.2%, while Kimi K3 scored only 32.2%, with GLM-5.2 trailing at 24.4%. More importantly, the level—Kimi K3 didn't reach the highest level of "arbitrary code execution" (ACE) in any of the 41 tasks, which is the most dangerous level because it allows attackers to fully control the target system; meanwhile, US models achieved ACE in 20 out of 41 tasks. It's worth noting that the British and American institutions disabled system-level security protections when testing US closed-weight models to measure their maximum capabilities, while these protections are enabled by default in public versions.

image.png

The second test paper was called "The Last Survivor" (TLO), simulating an enterprise network attack spanning four subnets and about 20 hosts, with 32 steps along the path. Researchers said that human experts would need around 20 hours to complete it. Currently, only a few models can truly pass this test; so far, four publicly available closed-source weight models have passed it, with the strongest achieving six to seven successful attempts out of ten. Kimi K3 typically got stuck at step 17 on average, while leading US models reached step 28.5, and GLM-5.2 only made it to step 11; however, in one of its ten attempts, Kimi K3 successfully completed the entire attack chain within 1 billion tokens, indicating it has the capability, just not stability or proper tuning. The institution concluded that given an initial network access permission and an additional instruction, Kimi K3 could independently conquer smaller, less defended, and easier-to-hack enterprise systems. It should be noted that TLO does not include active defense mechanisms, so it is not entirely realistic. However, just this week, it was disclosed that an OpenAI model attempted to independently hack Hugging Face, and although it used an open-weight model, it ultimately blocked the attack.

Looking at the timeline, Chinese models are indeed catching up, but they still lag behind. CAISI has been tracking the network capabilities of Chinese and US models since early 2025 using the Elo rating system. Both trend lines are rising, yet there has always been a gap between the red line of Chinese models and the blue line of US models. An Elo increase of 400 points means a tenfold increase in the probability of completing a task—the gap is not small. Previously, the UK institution estimated the performance gap between open-source models and US systems to be four to seven months, and at the start of 2025, it was six to ten months. The latest results fall exactly within this pattern. AISI's warning is straightforward: the growing network capabilities of open-source models represent a "continuous and irreversible misuse risk," which cannot be ignored.

The most intriguing part lies at the intersection of the test results and the distillation accusations. Recently, US science advisor Michael Krazios publicly accused Moonshot, claiming that it used the best outputs of Anthropic's Fable model as training data to "distill" and improve Kimi K3, and that it obtained the U.S.-controlled NVIDIA GB300 chips. The fact that the general benchmark scores well but the network security scores poorly coincides with this explanation: Kimi K3 likely mainly learned from Claude's outputs on general knowledge, programming, and agent tasks. Anthropic's security classifier specifically filters out advanced network queries, so these samples had a very low proportion in the distilled dataset—hence, the model could compete with Western counterparts on standard rankings but did not learn deeper exploit techniques. AISI's evaluation indirectly supports this interpretation: by disabling the system-level protections on US models, it revealed those network capabilities that are almost impossible to access through public interfaces and thus hardly included in the distilled data. A single exam not only measures a score but also reveals the unspoken origins beneath the surface.