At the 22nd China Cybersecurity Annual Conference (and the Cybersecurity Collaborative Defense Sub-forum of the National Cybersecurity Publicity Week), the first-ever crowdsourced testing results targeting AI large models were revealed. The event was guided by the Cybersecurity Coordination Department of the Central Cyberspace Administration, and hosted by the National Computer Network Emergency Response Technology Coordination Center. It attracted 559 white-hat security experts who conducted comprehensive security vulnerability tests on 15 AI large model and application products.

This test covered a variety of products, from basic large models to intelligent agents and model development platforms, aiming to identify potential security risks from an attacker's perspective. The results showed that a total of 281 security vulnerabilities were found, among which 177 were unique to large models, accounting for more than 60%. This indicates that AI large models face many emerging risks beyond traditional security areas.

Hacker, Vulnerability

The typical vulnerability risks revealed in the test include serious output misbehavior vulnerabilities in some products, frequent information leakage vulnerabilities, and prompt injection vulnerabilities as a common risk. In addition, although some large model products have certain protective capabilities, measures against unlimited consumption attacks still need to be strengthened. It is worth noting that traditional security vulnerabilities are still widespread and must be given sufficient attention.

In the mainstream large model products participating in the test, Tencent's Hunyuan Large Model, Baidu's Wenxin Yinyi, and Alibaba's Tongyi App had relatively fewer vulnerabilities found, showing a high level of security protection. This result brings confidence to users and developers.

At the end of the event, the official proposed four requirements for AI large model security governance, emphasizing the importance of future protection measures and vulnerability management, including strengthening security protection, developing vulnerability classification standards, encouraging the participation of social white-hat forces, and integrating security management throughout the entire lifecycle of AI systems.