# Inside the Black Box of AI Models Emmanuel Ameisen explores how LLMs form concepts and plan beyond next-token prediction. - **Source:** [Lenny's Newsletter](https://www.lennysnewsletter.com) - **Published:** 2026-10-18 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/53d16654-fc3b-45b5-8765-99c339b39405 ## Beyond Next-Token Prediction Emmanuel Ameisen argues that framing LLMs merely as next-token predictors is insufficient. His research at Anthropic suggests these models perform complex internal planning and form general concept representations. This perspective shifts the focus from simple probabilistic output generation to understanding the deeper cognitive-like structures emerging within large-scale neural network architectures. ## Conscious and Unconscious Processing Ameisen identifies structural similarities between AI processing and human cognition. His findings indicate that models may distinguish between conscious and unconscious-like states during operation. By tracing these internal pathways, researchers are beginning to map how models manage information, suggesting that AI reasoning involves more than just surface-level pattern matching. ## The Concept of J Space The research highlights the J space as a critical tool for revealing model self-awareness. By examining these internal spaces, developers can observe how models categorize information and maintain conceptual consistency. This insight is vital for interpreting how AI systems navigate complex queries and maintain coherence across long-form text generation tasks. ## Models Are Grown, Not Built Ameisen emphasizes that modern AI models are effectively grown through training rather than traditionally engineered. This distinction explains why capability jumps often appear sudden despite following smooth scaling curves. Understanding this growth process is essential for predicting how model performance evolves as computational scale and data diversity continue to increase. ## Anthropomorphizing as a Strategy Contrary to common technical advice, Ameisen suggests that anthropomorphizing AI can be a valid analytical approach. Treating models as agents with internal states helps researchers hypothesize about their behavior. This framing allows for more intuitive testing of how models plan, prioritize tasks, and handle complex instructions during real-world applications. ## Interpretability and AI Alignment Interpretability is directly linked to the broader goals of AI alignment and safety. By looking inside the black box, researchers can identify potential risks before they manifest in user-facing outputs. This proactive approach to safety ensures that as models become more capable, their internal decision-making processes remain transparent and controllable. ## Tracing Model Thoughts Anthropic’s research into tracing thoughts provides a window into the hidden reasoning of Claude. These techniques allow engineers to see the intermediate steps a model takes before producing a final answer. Such visibility is crucial for debugging errors and ensuring that the model's logic aligns with intended safety guidelines and constraints. ## Future of AI Transparency The work of researchers like Ameisen marks a transition toward verifiable AI intelligence. Moving beyond black-box performance metrics, the field is prioritizing structural understanding of neural networks. This shift is necessary for building reliable systems, as it moves the industry toward a future where AI behavior is predictable, safe, and explainable. ## Sources covering this story - https://www.lennysnewsletter.com/p/a-mind-expanding-conversation-on — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._