Integrate Custom AI Models with GitLab Duo via Microsoft Foundry
Self-hosted AI coding solutions with full infrastructure control.
1The Need for Controlled AI Deployment
Organizations with strict data sovereignty, residency, or regulatory requirements face unique challenges when adopting AI coding tools. GitLab Duo Self-Hosted addresses this by allowing complete control over model infrastructure, including hosting location, network paths, and credentials. This solution is particularly valuable for industries like finance and healthcare where code confidentiality is paramount. The system supports multiple model families while maintaining data within specified boundaries.
2Microsoft Foundry as a Versatile Platform
Microsoft Foundry serves as more than just an OpenAI endpoint, offering a comprehensive catalog including GPT, Claude, Llama, and Mistral models. This diversity allows teams to select specialized models for different tasks - agentic work, code suggestions, or high-volume requests - all managed through a single Azure subscription. The platform's flexibility enables organizations to match models to specific coding needs while maintaining centralized management and billing.
3Key Benefits of GitLab Duo Self-Hosted
The solution offers four major advantages: choice of Azure region for data residency compliance, access to multiple model families without separate contracts, feature-level model assignment for workload optimization, and full Azure integration including Entra ID and role-based access control. These features combine to provide enterprises with both flexibility and control over their AI-assisted development environment.
4Technical Architecture Overview
The system comprises three core components: the self-managed GitLab instance where users interact with AI features, an AI Gateway that routes requests (running on port 5052), and model endpoints in Microsoft Foundry. A single gateway serves all configured models, with GitLab determining the appropriate endpoint based on the requesting feature. This architecture maintains data flow within controlled infrastructure.
5Data Sovereignty Assurance
In fully self-hosted configurations, all inference data - including code inputs, prompts, and responses - remains within the organization's network. The solution only transmits minimal billing metadata (instance ID, anonymized user ID, call count, timestamp) for online licenses, with offline licenses requiring no external connections. This design meets strict compliance requirements for sensitive industries.
6Azure Deployment Options
Within Azure, organizations can choose between global, data zone, or regional deployments to meet jurisdictional requirements. While data at rest stays in the selected geography, inference processing location depends on deployment type. This granular control allows precise compliance with data residency regulations across different markets and industries.
7Model Selection Considerations
Choosing a model requires checking both GitLab support and Foundry availability, as these are independent factors. GitLab evaluates models across four capability areas: code completion, generation, Agentic Chat, and Agent Platform. The platform currently shows strongest support for GPT models through Azure, with Claude, Llama, and Mistral offering varying capabilities.
8Supported Model Families
The solution works with four main model families: GPT (available directly through Azure with deepest GitLab integration), Claude (strong agentic capabilities but with Marketplace prerequisites), Llama (performance varies by model size), and Mistral (including code-specialized Codestral). Each family has different region availability and feature support, requiring careful evaluation for specific use cases.
9Version Management Strategy
With both GitLab's supported-model matrix and Foundry's catalog evolving rapidly, organizations should implement a version tracking system. The documentation recommends choosing current general-purpose GPT models for full functionality across all features as the simplest path, while acknowledging that specialized models may better serve specific needs despite additional management overhead.
10Hybrid Deployment Options
While the solution emphasizes self-hosting, hybrid configurations are possible. Features can be routed to GitLab-managed models when needed, though this changes the data flow pattern. Organizations should carefully document which features use which routing to maintain compliance awareness and avoid accidental data exposure.
11Implementation Best Practices
Successful deployment requires coordinating across infrastructure, security, and development teams. Key steps include: mapping features to appropriate models, configuring Azure networking and access controls, establishing monitoring for usage and costs, and documenting all model assignments and routing decisions for compliance purposes.
12Future-Proofing AI Development
The GitLab Duo Self-Hosted solution represents a significant step forward in enterprise AI adoption, balancing innovation with control. By enabling organizations to leverage cutting-edge coding assistance while maintaining data sovereignty, it removes critical barriers to AI implementation in regulated industries and sets a new standard for responsible AI development tools.
Original article: GitLab — Integrate Custom AI Models with GitLab Duo via Microsoft Foundry