# Loop Engineering for RAG Systems: Processing Top-K Sequentially

Smart query optimization reducing LLM token costs by up to 80%.

- **Source:** [Towards Data Science](https://towardsdatascience.com)
- **Published:** 2026-07-22
- **Topic:** [Data & AI](https://brifing.si/c/data-ai)
- **Original article:** <https://towardsdatascience.com/loop-engineering-for-rag-generation-when-top-1-is-enough-when-you-need-top-k/>
- **On Brifing:** https://brifing.si/s/187f18b4-b2d3-4ac5-9f9e-e44e04ede471

## The Inefficiency Problem in RAG Systems

Traditional RAG systems send all top-K results to LLMs at once, causing unnecessary token consumption. For example, a policy date question processes 5 chunks even when the first one contains the answer. This approach creates significant costs especially in large document corpora.

## The Sequential Feeding Revolution

The proposed method starts with top-1 result and waits for LLM's 'sufficient' signal. If answer is found, the loop stops; otherwise it proceeds to next chunk. This technique reduces token costs by 80% for simple queries and is detailed in Article 8 of Enterprise Document Intelligence series.

## Cases Where Batch Mode Excels

Listing ('show all exceptions') and comparison ('did premium increase vs last year?') queries require batch mode. Sequential feeding fails here as LLM needs simultaneous chunk access. Batch mode is also preferred when relevance scores are close.

## Sufficiency Predicates

The system uses AnswerWithEvidence schema from Article 8A. answer_found (is info present?) and complete_answer_found (is answer complete?) booleans control the loop flow. These structured contracts replace custom heuristics for reliability.

## Cost Analysis

In typical corporate documents (K=5), sequential feeding achieves 1/5 cost for simple queries (80% traffic). Comparison queries favor batch mode. The executable notebook on GitHub compares token costs per question for both approaches.

## Dispatch Table

Clean architecture makes per-question decisions using question_df output from Brick 2. This dynamic routing aligns with Enterprise Document Intelligence's core philosophy, choosing between batch/sequential based on question type.

## Real-World Savings

In a 50k-policy corpus, date query optimization saves thousands in monthly LLM costs. Sequential feeding prevents processing unnecessary chunks like signatures/footnotes. Scalability proves critical for enterprise RAG systems.

## Implementation Scenarios

The technique applies directly to contract management, policy queries, and customer service bots. The four-brick RAG architecture (document parsing, question parsing, retrieval, generation) matures further with this optimization.

## Code Support

The article's runnable notebook is available on GitHub (doc-intel/notebooks-vol1). It tests batch/sequential generation with identical questions, reporting token costs and sufficiency booleans while enabling dispatch table reproduction.

## Paradigm Shift in AI Efficiency

This technique challenges 'more data is better' dogma in enterprise LLM use. Smart data feeding strategies form the new frontier of AI cost optimization. RAG systems achieve industrial-scale sustainability through such fine-tuning.

## Sources covering this story

- https://towardsdatascience.com/loop-engineering-for-rag-generation-when-top-1-is-enough-when-you-need-top-k/ — 

---

_Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._