Brifing

Building Overexposed: A Tool for Automated Application Exposure Assessment

How to accurately identify internet-facing services by combining cloud inventory with anonymous probing.

Shopify Engineering2 min

1Moving Beyond Static Configuration

Initial security scanning relied on reading infrastructure repositories, which only reflect intended states rather than actual deployments. This approach often missed live services. By shifting to Google Cloud Asset Inventory, the team now captures real-time data on Kubernetes Ingresses and Cloud Run services, ensuring the scanner evaluates what is truly running in production.

2Probing as an Anonymous User

The Overexposed scanner acts as an external stranger to verify actual internet accessibility. It sends anonymous requests to endpoints to determine if they truly require authentication. This method distinguishes between services that are technically public but protected by identity-aware proxies, and those that are genuinely exposed without any login requirements.

3Mitigating SSRF and Network Risks

To prevent Server-Side Request Forgery, the scanner validates hostnames and IP addresses against a blocklist of private and loopback ranges. It connects directly to the resolved IP to avoid DNS-based redirection attacks. These network-level safeguards ensure that the scanner does not inadvertently probe internal systems or sensitive cloud metadata endpoints during operation.

4Standardizing Verdicts and Logic

The team consolidated classification rules into a single table to handle diverse HTTP status codes consistently. A 404 response, for example, might indicate a missing path or a front-end rejection depending on the service type. This centralized logic allows reviewers to understand why a specific endpoint was flagged as exposed or protected.

5Managing Allowlist Exceptions

Allowlisted endpoints are not ignored; they are probed during every scheduled scan to ensure compliance remains valid. If an endpoint’s authentication requirement changes or the service disappears, the system flags it for review. This continuous validation prevents stale exceptions from creating long-term security gaps in the organization's public-facing infrastructure footprint.

6Reducing Inconclusive Scan Results

Early iterations of the scanner produced high rates of inconclusive results, reaching nearly 75% in July. By grouping these outcomes by reason and refining classification logic, the team improved clarity. Within weeks, they successfully reduced inconclusive Ingress results by approximately 80%, providing actionable intelligence that teams could immediately investigate and resolve.

7Leveraging AI and Mutation Testing

The team utilized AI coding agents to accelerate development while implementing mutation testing to verify the integrity of their security guards. By intentionally removing fixes to see if tests still pass, they identified gaps in test coverage. This rigorous validation process ensures that the scanner remains reliable and resistant to regression errors.

8Transparency in Security Operations

Effective exposure management requires honest results that explain the reasoning behind every verdict. By storing configuration context and probe evidence without sensitive data, the team enables efficient investigations. This transparent approach transforms security scanning from a guessing game into a reliable mechanism for maintaining access control across complex cloud environments.

Original article: Shopify Engineering — Building Overexposed: A Tool for Automated Application Exposure Assessment