LLM Reliability Layers Under Test: Accuracy, Yield, and Limits
Testing six reliability layers reveals they sort trustworthiness but fail to correct errors.
1Reliability Layers Theory
Architects often stack reliability layers around language models, assuming they additively improve accuracy. This study tested six distinct layers, including deterministic checks and secondary model judging, on the CADEC adverse drug event corpus. Contrary to expectations, these layers did not increase accuracy but merely sorted answers by their relative trustworthiness levels.
2The CADEC Benchmark
Researchers utilized the CSIRO Adverse Drug Event Corpus, containing 1,250 forum posts about two specific drugs. Each adverse reaction was marked with SNOMED CT codes to establish a gold standard for evaluation. This dataset served as the instrument to measure how much each reliability layer raised the accuracy of open-weight models.
3Extractor Performance Limits
The zero-shot extractor achieved an F1 score between 0.39 and 0.50, significantly lower than supervised systems like CONORM. The model struggled with reading comprehension, often missing mentions entirely. Because reliability layers only process output already generated by the extractor, they cannot recover mentions that were initially missed during the extraction phase.
4Deterministic Code Utility
Deterministic code proved effective at identifying impossible answers, such as nonexistent SNOMED codes. However, it cannot verify if a plausible answer is correct. While it caught missing spans, it failed to improve overall accuracy, demonstrating that deterministic checks are limited to filtering rather than correcting the model's underlying reasoning errors.
5Recognition Versus Generation
The model performed significantly better when selecting from a menu rather than recalling codes from memory. Recalling codes led to 13 to 18 percent nonexistent entries, while menu selection scored ten times higher. This highlights that recognition tasks are inherently more reliable for language models than open-ended generation of specific identifiers.
6Model Consistency Issues
Four of the five tested model families produced byte-identical output at temperature zero. The gpt-oss:20b model, the only mixture-of-experts architecture, showed variance across runs. This inconsistency complicates reliability, as identical inputs produced different results, forcing researchers to measure performance across multiple runs to establish a reliable range of output accuracy.
7Verdicts Do Not Stack
The study found that reliability verdicts nest rather than add value. Most answers endorsed by a free vocabulary check were also endorsed by a secondary judge. Consequently, adding more layers only removed records without recovering missed ones. Each layer represents a different trade-off between error reduction, lost answers, and manual review.
8The Cost of Precision
The most precise rule, ACCEPT, removed 128 errors but cost 49 correct answers and required 177 manual reviews. Because withholding is the only lever these layers possess, accuracy gains are always paid for by reduced yield. Accuracy metrics can be misleading if they do not account for the total number of records.
9Domain Adaptation Challenges
Two domain-adapted models were tested to see if they could improve extraction, but neither provided significant gains. BioMistral-7B failed to return parseable replies, while SapBERT worsened the menu selection. The primary bottleneck remains the model's reading capability, which no secondary reliability layer was able to overcome or effectively compensate for.
10Metric Design Flaws
The study identified a critical flaw in the evaluation metric: voting mechanisms overwrote codes without re-running vocabulary checks. This meant records were shipped as verified even when the code no longer matched the check. The metric failed to distinguish between an unwarranted answer and a wrong one, masking the system's true performance.
11Practical Architectural Lessons
The research concludes that stacking layers does not buy reliability in the way architects expect. Instead of automated correction, the focus should shift to six core practices derived from the study. These practices emphasize that while models can read, the responsibility for verification and decision-making remains with the software architecture and developers.
12Final System Assessment
Reliability layers are useful for sorting and filtering output but fail to fix underlying model errors. The ceiling for accuracy is set by the initial extraction, which remains the most critical failure point. Architects should prioritize robust extraction methods over complex stacking, as the latter provides diminishing returns on overall system reliability.
Original article: InfoQ — LLM Reliability Layers Under Test: Accuracy, Yield, and Limits