Why JSON Schema Fails as an AI Governance Layer
Structural validation cannot replace rigorous semantic enforcement in agentic authorization workflows.
1Governance Beyond Model Labs
While industry leaders debate oversight for frontier models, the actual security of agentic systems remains unaddressed at the code level. Decisions to move money or rotate keys are made by local functions, not centralized policies. These functions often contain subtle bugs that bypass security, regardless of how models are governed.
2The Protocol Ambiguity Problem
An independent stress test on the PIC protocol revealed that ambiguous specifications lead to critical vulnerabilities. When a protocol allows multiple defensible interpretations of data, different implementations will resolve inputs differently. This discrepancy creates a gap where attackers can exploit the disagreement between components, even when cryptography remains perfectly intact.
3Schema Limitations in Validation
JSON Schema effectively validates structure, types, and required fields, but it cannot enforce complex semantic rules. It fails to ensure that an identifier points to exactly one provenance entry. Because the schema treats duplicate entries as distinct items, it permits payloads that lead to inconsistent logic across different verifier implementations.
4Historical Precedents in Security
The industry has encountered these issues for two decades through HTTP parameter pollution, XML signature wrapping, and JWT algorithm confusion. These vulnerabilities occur when two components read the same input differently. Agent protocols are currently repeating these mistakes, as they lack the rigorous testing applied to older token and signature formats.
5The Danger of Input Normalization
Developers often normalize inputs like trimming whitespace or lowercasing strings to improve user experience. While helpful in search boxes, this instinct is dangerous at authorization boundaries. If a verifier normalizes a tool name or digest before checking policy, it may inadvertently match an allow-list entry that the original signer never authorized.
6Signatures and Canonicalization Risks
Signatures cover raw bytes, not abstract numbers. If a producer signs an uppercase hex digest and a verifier lowercases it before checking, the signature verification will fail. Conversely, if the verifier normalizes values after checking, it risks applying policy to data that was never actually signed by the original authority.
7Strict Rejection Over Normalization
To avoid implementation drift, protocols should favor strict rejection over silent normalization. By refusing payloads that do not meet exact formatting requirements, developers avoid the ambiguity of different libraries handling data differently. This approach forces producers to conform to a single, verifiable standard, making the system behavior predictable and easier to audit.
8Confused Deputy in JSON
Trust labels within a payload are frequently misused, creating a classic confused deputy problem. If a verifier trusts a 'trusted' label provided by the agent requesting the action, it essentially allows the requester to authorize itself. Trust must be derived internally by the verifier based on cryptographic evidence and operator keys.
9Integrity Versus Authority
A matching hash only proves data integrity, not authorization. An attacker can supply a file with a correct hash, but that does not mean an authorized party approved the action. True authority requires the verifier to check the signing key against a trusted list and ensure the signature covers all relevant fields.
10Systemic Risks in SDKs
Modern agent frameworks like the Model Context Protocol involve multiple layers, including models, clients, and transport hosts. Each layer may read the same data slightly differently. If these components do not share a unified understanding of the protocol, the security of the entire approval path is compromised by these minor implementation discrepancies.
11Auditing Your Approval Path
Engineers should test their approval paths by asking three critical questions: how are duplicate fields resolved, how are malformed strings handled, and who defines trust? If the CLI and the HTTP bridge handle these cases differently, the system is vulnerable. Consistency across all components is essential for maintaining a secure boundary.
12Designing Secure Governance Layers
A robust governance layer must separate structural validation from semantic enforcement. While JSON Schema is useful for initial filtering, it cannot replace a dedicated logic layer that handles provenance, signature scope, and trust derivation. Precise specifications that eliminate ambiguity are the only way to ensure that agents act within defined boundaries.
Original article: Stack Overflow Blog — Why JSON Schema Fails as an AI Governance Layer