# Building a Production-Grade RAG Pipeline for Semantic Code Search Technical strategies for scaling vector storage, indexing, and retrieval in codebases. - **Source:** [The JetBrains Blog](https://brifing.si/source/782eebd1-7be8-49cc-b27c-e552516691b2) - **Published:** 2026-11-05 - **Topic:** [technology](https://brifing.si/c/technology) - **Original article:** - **On Brifing:** https://brifing.si/s/b3a82822-28ee-48b5-8c1d-e9a8b4e930f1 ## Scaling Vector Storage Building a production-grade RAG system requires managing millions of vectors while maintaining low-latency search. Codebases are dynamic, requiring frequent updates across multiple branches. The system must efficiently handle repository filtering and access control, ensuring that search results remain relevant and secure for every user within a high-performance storage layer. ## Approximate Nearest-Neighbor Search To achieve millisecond response times, the system utilizes approximate nearest-neighbor structures like HNSW or IVF. These algorithms sacrifice perfect recall for significant speed gains by skipping exhaustive comparisons. While not guaranteed to return the absolute best match, they provide the necessary performance to handle massive datasets in real-world production environments. ## Hybrid Search Strategy The team employs a hybrid approach to balance speed and precision. An initial approximate search narrows the vast corpus down to a manageable set of candidates. Subsequently, an exact search is performed on this smaller subset. This combination allows the system to maintain scalability while ensuring high-quality results for complex queries. ## Metadata Filtering Challenges Semantic search often struggles with strict constraints like repository or folder-level permissions. Integrating metadata filters with ANN indices is difficult because filtering can disconnect graph structures. The team evaluates three strategies: pre-filtering, bundling metadata directly into the index, or post-filtering, with bundled metadata offering the most balanced performance for ACL requirements. ## Incremental Indexing Logic Re-indexing entire codebases upon every commit is computationally prohibitive. The system uses a Merkle tree structure to mirror directory layouts, allowing for top-down diffing. Only modified clusters are re-indexed, while unchanged subtrees are linked as-is. This incremental approach ensures that index updates remain efficient and avoid database CPU bottlenecks during operations. ## Managing CPU Bottlenecks Constructing HNSW indexes is a primary CPU-intensive task in the database pipeline. To maintain system stability, vector ingestion is decoupled from index building. Indexes are created asynchronously using strictly controlled parallelism. This non-blocking concurrent creation ensures that database instances remain responsive while snapshots only become searchable once the index is ready. ## Horizontal Database Scaling To overcome the limitations of single database instances, the architecture scales horizontally across multiple shards. Each shard is backed by a primary instance and geographically distributed read-only replicas to minimize latency. An application-side routing layer pins customers to specific shards, ensuring data isolation and efficient query handling across the global infrastructure. ## Local-to-Remote Cascade Indexing Developers often work on local machines before pushing code to remote branches. The system supports local indexing using USearch, a compact engine that mirrors the server-side model. This allows for seamless transitions where local indexes are synchronized with the file tree, eventually merging into the central index once code is pushed. ## Post-Retrieval Reranking Nearest-neighbor search produces a raw candidate list that requires refinement. Post-processing steps, such as removing duplicates and grouping file fragments, improve output quality. Reranking is then applied to the limited candidate set, allowing for a more computationally expensive, precise evaluation of relevance compared to the initial, high-speed vector search pass. ## Agentic Search Orchestration The system explores using dedicated sub-agents to orchestrate complex retrieval tasks. These agents make judgment calls on which tools to invoke and set dynamic thresholds based on the specific query. By offloading these decisions to specialized sub-agents, the main agent maintains a focused context, reducing token consumption and improving overall task efficiency. ## Optimizing Agent Context The final result formatting is tailored to save agent time while preventing context pollution. Results exceeding a relevance threshold include full code snippets and metadata, while lower-scoring results provide only basic identifiers. This tiered approach ensures the agent receives sufficient data to assess relevance without being overwhelmed by excessive, irrelevant information. ## Lessons in Production RAG Building Air Context demonstrated that vectorization is only the beginning of a RAG pipeline. Success depends on sophisticated storage, incremental indexing, and intelligent post-processing. By combining approximate search with exact reranking and agentic orchestration, developers can create robust systems that handle the scale and complexity of modern, production-grade software development environments. ## Sources covering this story - https://blog.jetbrains.com/ai/2026/11/building-a-rag-pipeline-for-semantic-code-search-a-developer-diary-and-field-notes-part-2/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._