# New Approaches for Listing Questions in RAG Systems The answer is not in a single passage, but scattered across the entire document. - **Source:** [Towards Data Science](https://towardsdatascience.com) - **Published:** 2026-08-07 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/5f8df556-728a-4044-97c6-f4336cedb299 ## The Challenge of Listing Questions RAG systems are typically optimized for answers found in a single passage. However, questions like 'list all exclusions in this policy' fail when answers are distributed across different parts of the document. These questions break the assumption that the answer is in one passage, leading systems to produce incomplete answers. ## Limitations of Traditional Approaches Traditional RAG systems operate using top-k retrieval. This method retrieves the k most similar passages and expects the LLM to extract the answer from them. However, in listing questions, this approach fails to capture all items and produces incomplete answers. For example, only five out of nine exclusions in a policy might be listed. ## The NIST Example In the NIST Cybersecurity Framework, the question 'list all categories under the GOVERN function' includes six categories. However, top-k retrieval often captures only four or five of these categories and remains silent about the missing ones. This leaves users with incomplete information. ## Limitations of Long-Context Models Long-context models excel at finding a single needle in a haystack. However, listing questions require finding six needles scattered in different places. These models miss items expressed differently and present incomplete lists. For example, they might fail to capture a GOVERN category phrased differently in a passage. ## Structural Issues with Top-k Retrieval In listing questions, top-k retrieval operates with a fixed k value. This either captures too few items or dilutes the LLM's attention with irrelevant passages. When items are similar, their similarity scores are also similar, making it hard for the LLM to distinguish which passages are actual definitions. ## Proposed Structural Solution It is recommended not to use top-k retrieval for listing questions. Instead, an approach that scans the entire document and captures all items should be adopted. This prevents incomplete answers and ensures users receive a full list. ## Completion Signal For listing questions, the system needs to generate a completion signal. This signal indicates that the list is complete and prevents users from encountering incomplete information. For example, a signal confirming all categories in NIST CSF have been listed can be used. ## The Role of Expert Knowledge Expert knowledge is crucial in listing questions. The expert knows that bounded lists exist in their domain and what a complete list looks like. The system uses this knowledge to generate completion signals and allows the expert to ratify the result. ## Code Examples The article provides runnable code examples for three different strategies to handle listing questions. These codes can be used to pull GOVERN categories, scan GV.XX-NN codes, and capture missing items. They are accessible via GitHub. ## Lack of Benchmarks Current RAG benchmarks typically measure questions with a single answer. Listing questions have a shape that these benchmarks do not measure. Therefore, new benchmarks should be developed to measure the performance of listing questions. ## Next Steps New retrieval methods need to be developed to handle listing questions. These methods should scan the entire document and capture all items. Additionally, completion signals and expert knowledge should be integrated to prevent incomplete answers. ## Conclusion Listing questions pose a significant challenge for RAG systems. Traditional top-k retrieval methods fall short in handling these questions. New approaches and benchmarks should be developed to ensure users receive complete and accurate answers. ## Sources covering this story - https://towardsdatascience.com/loop-engineering-for-listing-questions-when-the-answer-is-every-passage-not-the-top-one/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._