Secret Guard: A Local LLM Gateway for Secure Agent Workflows
Protect sensitive credentials by replacing them with realistic mocks before LLM processing.
1The Agent Security Dilemma
Modern AI agents require access to private keys, tokens, and passwords to perform tasks on real systems. However, sending this sensitive data to LLM vendor servers creates significant security risks. Data can be logged, used for model training, or exposed during a breach, leaving your credentials outside your direct control.
2Gateway Redaction Mechanism
Secret Guard acts as a local gateway that intercepts outbound requests before they reach the LLM provider. It replaces real secrets with equal-length, realistic mocks that maintain the structure of the original data. This ensures the LLM receives valid-looking input while never actually touching your sensitive, real-world authentication credentials.
3Transparent Response Restoration
When the LLM sends a response back, the gateway performs a reverse operation to restore the original secret values. This process is transparent to the local agent tools, allowing them to receive authenticated responses as if the LLM had processed the real data. Authentication flows remain uninterrupted and fully functional.
4Configurable Secret Strategies
Users can define specific behaviors for different credentials beyond simple redaction. The reject strategy returns a 503 error if a secret appears, while the placeholder strategy swaps it for a non-restorable token. Placeholders act as canaries, triggering alerts if a secret is ever inadvertently leaked in an LLM response.
5Self-Protection for Gateway Tokens
The tool automatically protects its own API tokens and registered provider keys without requiring manual configuration. If a prompt injection attempt tries to force an agent to reveal the gateway's internal token, the system ensures only a mock value is exposed to the LLM vendor, maintaining the integrity of the local gateway.
6Integration and Deployment Options
Integration requires pointing an agent tool's base URL to the local gateway, often involving a single line of configuration. It supports binary archives for Linux, NixOS modules for structured deployment, and Cargo builds for source-level installation. Optional OIDC authentication is available via specific build features to enhance multi-user security.
7WebUI and Session Auditing
The built-in WebUI provides a comprehensive timeline of all forwarded requests and responses. Users can inspect exactly what the LLM saw by viewing the mocked requests alongside the restored real values. This visibility is critical for verifying that redaction is functioning correctly and for debugging complex agent tool interactions.
8Protocol Support and Limitations
The gateway supports OpenAI and Anthropic protocols with cross-protocol translation, enabling streaming and complex tool calls. Gemini and Ollama are currently supported as experimental byte pass-throughs. While the tool is robust, users should verify real-provider field validation in the Records page when adopting new protocols for the first time.
9Security Through Local Control
By keeping the secret-guard.toml configuration file local and ensuring that real keys never leave the machine, the tool mitigates risks associated with compromised agent harnesses. This architecture provides a necessary layer of defense for developers who must balance the utility of AI agents with the strict requirements of data privacy.