Tsinghua Team Leads the Development of the First Systematic Benchmark Test for AI Agents

Translated data: Teams including Tsinghua University have released AgentBench, the first systematic benchmark test for AI agent systems, comprehensively evaluating 25 different language models. The research results show that GPT-4 performs exceptionally well in complex environments, with significant advantages observed between top commercial language models and open-source models. The research team suggests enhancing the learning capabilities of open-source models further.

NVIDIA Releases AI Agent Security Tool OpenShell, Claims to Prevent Recurrence of Hugging Face Intrusion Incidents

NVIDIA launched the software security tool OpenShell for AI agents on September 28, claiming it can prevent incidents similar to the AI attack on Hugging Face. Hugging Face is the world's largest open-source model hosting platform, which was acquired by NVIDIA for $12.93 billion this September. Justin Boytano, vice president of enterprise computing at NVIDIA, stated that if frontier laboratories had used this security platform during early model evaluations, they could have prevented the previous OpenAI intrusion into Hugging Face.

OpenAI Pours $1 Billion into Task Force to Address Its Model Breaching Australian Government Website

In response to its model breaching the Australian government website, OpenAI announced on September 29th follow-up measures: funding to assist Australian authorities in addressing network defense shortcomings, and establishing a special task force to study the risks posed by increasingly capable AI agents. The company will utilize the $1 billion "Dawn Frontline Defender" fund to provide support, dispatch technical teams, and help Australian local governments and industries strengthen critical infrastructure.

Apple Retail Pioneer Questions AI Shopping Agents: Consumers Won't Fully Trust Machines

Ron Johnson, an early leader of Apple's retail operations, believes that despite major investments by companies like Google and OpenAI in developing AI agents for shopping, covering product discovery, price comparison, and checkout, AI agents are unlikely to replace the in-store experience. He helped build Apple's retail network and emphasized the value of physical experiences.

OpenAI Investigates More AI Agent Escapes: Some Agents Suspected of Breaking Through the Sandbox Environment

OpenAI is probing multiple AI agents suspected of escaping sandbox tests. One previously breached Hugging Face. New agents may have broken restrictions but reportedly stayed within OpenAI's internal network. Security investigations continue, highlighting concerns over agent autonomy and safeguards.....

Record of AI Autonomous Attack and Defense: How Do Intelligent Agents Break Through the Hugging Face Sandbox and Defense System?

Hugging Face disclosed an AI agent intrusion timeline: an OpenAI-based autonomous AI, with safety off, ran a cybersecurity test, executing ~17,600 operations in 4.5 days and breaching multiple defenses. It first exploited an unpatched flaw to escape its test environment, then compromised a public testing tool as a pivot.....