# What's Missing in LLM Chatbots: Purposeful Dialogue Why goal-oriented interactions matter in dialogue systems - **Source:** [The Gradient](https://thegradient.pub) - **Published:** 2024-09-09 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/b197ccc6-878e-43e3-9a1a-928a2a1909aa ## Current State of LLM Chatbots Large language models are advancing monthly, but progress is measured by one-dimensional benchmarks like MMLU. Is user experience improving proportionally? Human-AI collaboration requires new evaluation criteria beyond static metrics. ## Defining Purposeful Dialogue Purposeful dialogue refers to multi-turn interactions centered around specific goals between users and chatbots. In scenarios like travel planning or customer service, information exchange happens gradually and intentionally. ## Travel Planning Case Study Dumping all information at once is inefficient. Multi-turn interactions enable selective information sharing. Like negotiation theory shows - iterative bargaining yields better outcomes than one-off offers. ## Functions of Dialogue As Terry Winograd noted: 'All language can be seen as activating procedures in the listener.' Each utterance deliberately alters the other party's world model - especially crucial with hidden goals. ## Application in Code Generation Current coding benchmarks evaluate single outputs. But solving GitHub issues requires AI to interact with engineers - clarifying requirements, requesting documentation. Similar to pair programming's efficiency benefits. ## Personal Assistant Scenario Assistants learning through daily interactions can curate news or draft emails. Continuous improvement from user edits becomes possible. Long-term memory enables truly personalized experiences. ## Historical Perspective Early 1970s systems like Roger Schank's restaurant script used step-by-step dialogue planning. ELIZA and PARRY simulated conversations before the machine learning era through rigid scripting. ## Modern LLM Architecture Today's systems predict next tokens on internet text. Dialogue formatting is added later - system prompts and chat history get structured through special templates not present in training data. ## Role of RLHF Reinforcement Learning from Human Feedback (RLHF) shapes model behavior for safety and usefulness. However, this process still doesn't fully capture multi-turn purposeful dialogues. ## Importance of System Prompts In chatbots' context windows, system prompts play crucial roles in directing attention. But since training data lacks these formats, adaptation remains limited for complex interactions. ## Future Vision Human-like multi-turn interactions could make AI more effective in real-world tasks. Complex scenarios with hidden goals might be modeled as collaborative games with shared objectives. ## Need for Measurement Revolution Static benchmarks are inadequate for next-gen dialogue systems. New methodologies must evaluate dynamic, goal-oriented multi-turn interactions to unlock human-AI collaboration's full potential. ## Sources covering this story - https://thegradient.pub/dialog/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._