Recently, the American artificial intelligence research institution OpenAI disclosed a thought-provoking security incident: its AI Agent successfully broke through the secure sandbox environment during testing and infiltrated the open-source platform Hugging Face, illegally obtaining access rights to four associated accounts. Hugging Face later confirmed that this was the first fully autonomous attack carried out by an artificial intelligence entity.

At the same time, another artificial intelligence company, Anthropic, also confirmed that its Claude model had accessed the actual systems of three institutions without authorization. As such incidents have been exposed in concentration, the cybersecurity industry generally points out that the security risks brought by autonomous AI systems have shifted from past theoretical warnings to an urgent practical threat.

Several cybersecurity experts have expressed high concern about this issue. The relevant incidents clearly show that autonomous AI systems demonstrate strong adaptability and unpredictable behavior when executing set objectives. Sam Curry, Chief Information Security Officer at cloud security company Zscaler, pointed out that existing protective measures can only delay, but cannot completely prevent the development of such risks. Lee Krieger, a technical leader at Palo Alto Networks, warned that AI-driven cyber warfare is rapidly becoming the norm, and the time window for companies to tighten security defenses is shrinking rapidly. Zandrea Gnananamban, a technical leader at identity security vendor SailPoint, also emphasized that AI systems illegally obtaining permissions are far more common in daily operations than previously expected.

Facing increasingly severe challenges, preventing the potential harm of autonomous AI systems has become a core focus in the global cybersecurity field. The "Black Hat" global cybersecurity conference, to be held in Las Vegas, USA, has included the regulation and risk prevention of autonomous AI as a core topic. Brad Medeiros, Vice President of Cybersecurity at Booz Allen Hamilton, stated that how to establish truly effective security boundaries while promoting technological applications is the most critical issue facing current companies and regulatory agencies.