# Testing Position Sensitivity in Modern Large Language Models Developer fails to replicate 'lost in the middle' phenomenon on current AI models. - **Source:** [r/MachineLearning](https://www.reddit.com/r/MachineLearning) - **Published:** 2026-11-09 - **Topic:** [technology](https://brifing.si/c/technology) - **Original article:** - **On Brifing:** https://brifing.si/s/b66cce5a-1408-40c8-b79b-782f3b9da9ff ## Revisiting Lost in the Middle The 'lost in the middle' phenomenon suggests language models struggle to retrieve information placed in the center of long context windows. A developer recently attempted to replicate this behavior using modern models like Haiku 4.5 and Sonnet 5. The goal was to verify if these models still exhibit significant positional sensitivity during retrieval. ## Experimental Methodology Used The researcher utilized a simplified testing approach by inserting a single fact-containing document into a pile of distractor documents. By sliding the target document through various positions—front, middle, and back—the study measured if the model's answer consistency fluctuated. This method avoids the need for complex labeled datasets while testing for internal contradictions. ## Initial Findings on Retrieval Testing across 5, 15, and 30 documents yielded no significant performance drops for either model. At the 5 and 15 document levels, the models maintained perfect consistency. Even at 30 documents, the initial error rate was negligible, suggesting that current architectures handle basic single-fact retrieval tasks with high reliability regardless of document placement. ## Addressing False Positives The researcher initially observed a 4% error rate at 30 documents, but closer inspection revealed these were false positives. The models were simply outputting identical answers in different formats, such as '3' versus 'three'. Once the evaluation shifted from exact string matching to semantic meaning comparison, the error rate effectively returned to zero. ## Limitations of the Study The author acknowledges that these tests represent the 'easy case' of retrieval. The experiments focused on single-fact extraction with relatively short documents. The researcher notes that positional sensitivity might still manifest in more complex scenarios, such as multi-hop reasoning tasks or when the relevant information is buried within significantly longer, more dense documents. ## Open Source Verification To encourage community verification, the researcher published the testing script on GitHub under the repository name 'wobbly'. By providing the live_rag.py script, the author invites other developers to run these tests on their own setups. This transparency allows for broader validation of whether current models have truly overcome previous context window limitations. ## Context Length Considerations While the current results suggest that 'lost in the middle' is largely mitigated for simple tasks, the industry remains curious about extreme context lengths. Future testing will likely focus on whether these findings hold up as context windows expand further. Developers are encouraged to test their own specific use cases to confirm model behavior. ## Implications for AI Development The findings suggest that recent advancements in model training have significantly improved information retrieval accuracy. If these results hold across broader benchmarks, it indicates that architectural improvements are successfully addressing previous weaknesses in long-context processing. Moving forward, researchers must prioritize semantic evaluation over simple string matching to accurately measure true model performance. ## Sources covering this story - https://www.reddit.com/r/MachineLearning/comments/1xrn7lo/i_tried_to_reproduce_lost_in_the_middle_on/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._