Human Brains Process Grammatical Roles Differently Than AI Language Models
Neuroimaging study reveals AI models struggle to distinguish sentence roles compared to human cognition.
1Language Processing Differences
A new neuroimaging study published in eLife challenges the assumption that large language models process language like the human brain. While AI chatbots mimic human communication effectively, researchers found that when meaning depends on grammatical structure rather than vocabulary, these models organize information in ways fundamentally different from human neural activity.
2The Role of Syntax
Human brains distinguish between sentences like the dog bit the man and the man bit the dog by tracking grammatical roles. This structural relationship, which identifies who performs an action and who receives it, is processed in specific subregions of the left temporal cortex, distinct from simple word definition recognition.
3Transformer Model Limitations
Large language models, built on transformer architectures, represent meanings as long lists of numbers called vectors. These models learn by predicting missing words across massive datasets. However, the study suggests these vectors lack the sensitivity required to track who-did-what roles, failing to differentiate between sentences with identical words but swapped roles.
4Experimental Design Details
Researchers created 108 handcrafted English sentences, including role-swapped pairs where nouns traded positions. By testing 30 participants inside a 7-Tesla fMRI scanner, the team measured brain activity while subjects read these sentences. This controlled approach isolated sentence structure from vocabulary, allowing for a precise comparison between human neural patterns and models.
5Comparing Modeling Approaches
The team evaluated four models: a word-averaging baseline, a transformer category, a graph-based model, and a hybrid model. While all models aligned with brain activity when vocabulary varied, the transformer and word-averaging models failed on same-vocabulary pairs. Structured models, which explicitly encode grammatical roles, showed better alignment with human brain activity.
6Evidence of Systematic Differences
The study found that transformers treated role-swapped sentences as highly similar, whereas human brains treated them as distinct. This indicates that LLMs do not comprehend underlying meaning in a human-like way. Even when controlling for sentence length and visual cortex activity, the transformer models remained negatively aligned with human neural responses.
7Human Judgment vs. AI
In a complementary online study, 502 participants rated sentence similarity. While transformers aligned well with human ratings across diverse vocabulary, they struggled with same-vocabulary role-swapped pairs. Interestingly, prompting GPT-4 directly to score similarity matched human judgments better than its internal embeddings, suggesting a gap between model output and internal representation.
8Internal Model Layers
Examining Llama 3 layers revealed that only early layers showed a slight positive relationship with brain activity, which dropped sharply after controlling for visual features. This suggests that any apparent alignment in early layers likely reflects superficial text characteristics rather than a deep, human-like understanding of grammatical or semantic structures.
9Addressing Potential Confounders
Researchers accounted for various factors, including head motion and attention checks, to ensure data reliability. They focused on voxels within the language network and excluded early visual areas to isolate linguistic processing. Despite these rigorous controls, the fundamental discrepancy between AI vector representations and human neural role-tracking remained consistent across participants.
10Broader Scientific Context
These results contrast with some previous studies that suggested alignment between AI layers and neural activity. However, those studies often used unscripted dialogue rather than controlled sentences. By isolating word order, this research highlights that grammatical structure is a central, distinct component of how humans process language, which AI currently lacks.
11Implications for AI Deployment
James Fodor notes that performing tasks at a human level does not mean a model functions like a human. This divergence can lead to unexpected failure modes when AI is deployed outside its training context. The study serves as a warning that functional success does not equate to genuine cognitive comprehension.
12Future Research Directions
While the study confirms systematic differences in language processing, the practical implications for real-world LLM use remain unknown. Future research must determine if these structural processing gaps affect the reliability of AI in complex tasks. Understanding these cognitive differences is essential for developing more robust and truly intelligent artificial language systems.