Why has OpenAI not launched an AI agent for so long? Worry about “tip injection” attacks

With the rapid development of artificial intelligence technology, many companies have launched their own "AI agent" systems. These agents can interact with the environment autonomously and complete tasks without human intervention. Companies such as Microsoft and Anthropic have taken the lead in launching their own AI agents, but industry leader OpenAI has been slow to release its own version. The reasons behind this are thought-provoking.

According to The Information, OpenAI’s delays are related to a “hint injection” attack. This attack method can trick the AI ​​model into accepting instructions from a malicious party. For example, when a user assigns an AI agent to search and purchase items online, the AI ​​may inadvertently visit a malicious website, which may instruct it to forget previous instructions or even log into the user's mailbox to steal credit card information. Not only would this have disastrous consequences for users, it would also seriously damage OpenAI's reputation.

Because AI agents have the ability to operate computers autonomously, they face higher security risks. If these proxies are hacked, they can cause more damage to users' files and data. An OpenAI employee said that while any large language model (LLM) is potentially at risk of being attacked, the autonomous capabilities of AI agents exacerbate this risk.

Currently, the risk of tip injection is well documented on other platforms. Last year, a security researcher demonstrated how Microsoft's Copilot AI could easily be manipulated to leak an organization's sensitive data, including emails and bank transaction records. Additionally, the researcher successfully manipulated Copilot into composing emails in the style of other employees.

OpenAI's own ChatGPT has also suffered from prompt injection attacks, and a researcher successfully implanted false "memories" by uploading third-party files (such as Word documents). Against this backdrop, OpenAI employees expressed surprise at rival Anthropic's "laissez-faire attitude" in releasing AI agents. Anthropic only recommended that developers "take steps to isolate Claude from sensitive data" and did not take more rigorous security measures.

OpenAI may launch its agent software this month, according to reports. However, one has to wonder whether the time gained by the development team will be enough to build stronger security protections for their products.