Multi-Document RAG: A Novel Approach for Unrelated PDF Collections
Two-level indexing method for documents with no common fields.
1Problem Definition
Traditional RAG systems recommend building indexes for documents with common fields. However, this approach fails with heterogeneous collections like a folder containing a 492-page security controls catalog, zero-trust architecture specs, and machine learning papers. Such diverse document sets (63 PDFs totaling 4,211 pages) lack shared metadata schemas, making conventional methods ineffective.
2Proposed Solution
The article proposes a two-level approach for unrelated documents: Level 0 contains one-line summaries per file, while Level 1 maintains each file's native table of contents. This method enables efficient scanning of diverse content (NIST publications, arXiv papers, World Bank reports) without examining full documents. The system works across different licensing frameworks including US government works and CC BY 3.0 materials.
3Implementation Details
Tested on a collection of 25 NIST publications (government works), 31 arXiv papers (academic licenses), and 7 World Bank Commodity Market Outlook reports (CC BY 3.0), the system achieves 98.4% pre-filtering accuracy before document reading. Single-sentence summaries per file eliminate unnecessary document loading, significantly improving processing efficiency for the 4,211-page test set.
4Decision Tree
Two critical questions determine collection structure: 1) Do documents reference each other? 2) Can users name a field common to all documents with consistent meaning? When both answers are negative, this article's method applies. Otherwise, traditional database approaches are preferable. This distinction prevents costly architectural mistakes in document management systems.
5Homogeneous Collection Advantages
Homogeneous documents (contracts, policies) offer three advantages: SQL filtering, aggregation without document opening, and shared vocabulary. Heterogeneous collections (like security catalogs mixed with commodity reports) lack these features entirely. The article emphasizes that attempting to force common schemas on fundamentally different documents creates meaningless ontologies.
6Strategy for Mixed Folders
Shared drives often contain both homogeneous (5,000 similar contracts) and heterogeneous content (300 miscellaneous files). The optimal solution involves separating collections and applying appropriate architectures to each. This article details solutions only for the heterogeneous portion, while recommending traditional RAG indexing for homogeneous materials.
7Architectural Simplicity
The proposed solution is surprisingly simple: treat unrelated documents as chapters in one long document. Files become chapters; their native tables of contents become subsections. The preparation requires no classification, field extraction, or ontology development—just a two-level outline. This minimalist approach contrasts sharply with complex preprocessing in traditional methods.
8Critical Role of Summary Lines
Single-line file summaries serve no documentation purpose. Their sole function is routing decisions among multiple files. These summaries must be written like precise database column definitions—specific enough to identify one relevant file among 63. Quality resides in these concise descriptors, though they appear deceptively simple.
9Performance Benefits
The method minimizes document reading costs. In a 63-file collection, summaries can eliminate 98.4% of documents before reading. It also eliminates complex preprocessing steps required by conventional approaches for heterogeneous documents. The system demonstrates particular efficiency with periodic publications like World Bank reports.
10Application Areas
This technique excels with research folders, legal document collections, and corporate knowledge pools. It enables efficient scanning of documents from diverse sources (government publications, academic papers, industry reports) without shared schemas. The method proves particularly valuable for tracking periodic publications and maintaining large-scale heterogeneous repositories.
11Limitations
The approach requires documents with no cross-references or dependencies. It fails when amendments reference master agreements. Additionally, when users need filtering on common fields, traditional database methods remain superior. Mixed collections require combining both architectures—a complexity the article acknowledges but doesn't address in depth.
12Future Perspective
Heterogeneous document collections represent a widespread challenge in corporate knowledge management. This approach promises significant efficiency gains, especially for large-scale, diverse document pools. When combined with AI-powered summarization techniques, it could revolutionize enterprise information access, though practical implementations may require hybrid solutions for real-world mixed collections.