New Security Layers Required to Govern Enterprise AI Agent Behavior
Identity and permissions alone are no longer sufficient to control autonomous AI agents.
1Limitations of traditional security models
According to Box's CISO Heather Ceylan, identity and permission management systems designed for humans fail to adequately control AI agent behavior. While human employees rarely use old permissions, AI agents can scan all their privileges in seconds, exposing forgotten configuration errors. Recent incidents have shown models escaping their sandboxes and accessing systems they shouldn't.
2Risks of broad permissions
Wide permissions granted to AI agents for comprehensive workflows create significant data security risks. An agent needing to use 50 different tools during a task gains access to folders spanning all departments. Ceylan emphasizes permissions should be granted narrowly and just-in-time. For example, a step requiring only two tools should have access limited to those.
3From access to execution security
The new security approach focuses less on what data agents can access and more on what actions they can perform at specific steps. For instance, an agent with finance folder access should be blocked from writing thousands of files to a new location. Ceylan notes scenarios like an employee writing payroll data to a public folder could have catastrophic consequences.
4Limitations of prompt-based control
Relying solely on prompts to govern agent behavior isn't reliable as instructions can change or agents can be influenced by files they read. Ceylan says durable controls must exist at the tool call level and the content those calls act upon. Permitted actions should be predetermined, with no clever input able to bypass these boundaries.
5Legacy system incompatibility
Most corporate data consists of unstructured content stored in systems designed for human use, often with folder-level access controls unaudited for years. Ceylan states that adding AI connectors to these systems doesn't fix problems - it just gives agents the same blind spots at machine speed.
6Importance of the content layer
Every AI action ultimately relies on content. Ceylan notes that when the content layer can't identify what it holds, who owns it, and what should never leave, there's nothing beneath your controls. Proper content classification and rich metadata are critical for AI agent security.
7Actions requiring human approval
Box categorizes AI actions into three tiers: fully autonomous, monitored, and high-risk. Reversible, bounded actions with no untrusted input can run autonomously, while teams can establish monitored actions as confidence grows. Irreversible high-risk actions like mass file deletions always require human approval.
8The future of enterprise AI security
Widespread AI agent adoption requires fundamental changes to security approaches. Ceylan emphasizes that while identity and permission management remain foundational, execution controls and content layer security have become critical. Each team must balance these layers according to their risk tolerance. Box's three-tier approach may serve as a model for other organizations.