# Analyzing 'Trump' Embedding Geometry in GPT-2 Small Comparison of Discrete and Continuous approaches. - **Source:** [r/MachineLearning](https://www.reddit.com/r/MachineLearning) - **Published:** 2026-07-18 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/3dbc340b-8f80-4a28-a880-39081e3948e5 ## Token Embeddings in GPT-2 Small This analysis examines the 'Trump' token in GPT-2 Small's static embedding table. The model's learned token embeddings are considered before attention or context is applied. ## Visualization with t-SNE Projection 32,070 alphabetic tokens, each with at least two characters, are visualized using t-SNE projection. This method makes high-dimensional data understandable in two dimensions. ## Discretized Approach In the discretized approach, each coordinate is thresholded before neighbors are calculated. This method reveals general political terms like Mitt, Hillary, Pelosi, and Blair. ## Continuous Approach In the continuous approach, original coordinates are retained to calculate neighbors. This method reveals a more specific group including Obama, Clinton, Bush, and Eisenhower. ## Family Members and Rivals The continuous approach reveals more specific relationships, such as Trump's family members, staff, and rivals. This shows how detailed the embeddings learned by the model are. ## Model Understanding and Insights This analysis provides important insights into how embeddings learned by models like GPT-2 Small work. Understanding the inner workings of models can help develop better models. ## Sources covering this story - https://www.reddit.com/r/MachineLearning/comments/1v07xai/gpt2_smalls_embedding_geometry_around_trump/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._