Lessons Learned from Building an Insecure AI Agent
Token's experiment reveals critical security gaps when AI agents operate within human sessions.
1The Rehearsal Experiment
Token developed an AI agent named Rehearsal to test security boundaries within their platform. The agent autonomously navigates browser tabs, fills forms, and sets filters to answer user questions. By design, this agent bypassed standard security protocols, operating directly within authenticated user sessions to demonstrate how easily visibility can be lost.
2Identity Visibility Gaps
When an agent operates inside a user's session, the underlying application cannot distinguish between human actions and automated tasks. Session cookies lack metadata identifying the session driver, meaning the server treats all clicks as originating from the human user. This creates a significant blind spot for security teams monitoring platform activity.
3Current Browser Limitations
Modern browsers like Chrome and Edge are introducing features to manage agentic browsing, such as requiring human approval for high-stakes actions like payments. While these tools offer basic protection, they often operate in isolation. They fail to communicate the agent's identity or the specific scope of the task to the application.
4Emerging Industry Standards
Draft standards like WebMCP and Web Bot Auth aim to improve agent security by allowing sites to expose specific actions and verify agent requests. However, these frameworks currently lack a unified layer to connect identity, delegation, and scope. Consequently, they do not fully solve the problem of unauthorized privilege escalation by agents.
5The Need for Context
Effective authorization requires three core signals: the agent's identity, a verifiable link to the human delegator, and a restricted task scope. Without this integrated context, applications cannot make informed security decisions. Simply building faster lanes for agents without these safeguards creates a more efficient path for the misuse of user privileges.
6Immediate Defensive Actions
Security teams should not wait for universal standards to emerge before taking action. Organizations must actively discover agentic browsers and extensions currently operating within their environments. Identifying which users have sessions borrowed by agents is a critical first step in mapping the current attack surface and managing potential security risks.
7Implementing Step-Up Checks
Since applications cannot inherently distinguish between human and agent clicks, companies should implement step-up authentication for sensitive operations. Requiring a passkey prompt for consequential actions ensures that a human remains in the loop. This mitigates the risk of an autonomous agent performing unauthorized tasks without explicit, verified user consent.
8Strategic Security Outlook
Until browsers and applications natively support agent identity and delegation, every agent using a human session must be treated as delegated access. Companies should prioritize discovering, constraining, and auditing these connections. Proactive management of AI identities is essential to prevent hidden threats and maintain operational security in an agent-driven landscape.
Original article: The Hacker News — Lessons Learned from Building an Insecure AI Agent