Interactive Map to Explore GPT-2's Token Embedding Space
Tap any token to explore GPT-2's language model.
1Introduction
This interactive map visualizes GPT-2's token embedding space. Users can tap any token and explore its nearest connections to better understand how the language model works.
2How It Works
The map displays 32,070 alphabetic tokens from GPT-2-small's WTE. It is organized using the t-SNE algorithm over a compressed representation, with each line representing a real nearest relationship.
3User Experience
Users can zoom in, tap any token, and explore its neighbors. The search box allows them to quickly jump to any token they are interested in.
4Mobile Compatibility
The map works seamlessly on mobile devices. Users can pinch to zoom and navigate between tokens with ease.
5Technical Details
The map is created using the minimum spanning tree method, ensuring that each line represents a real proximity relationship between tokens.
6Significance
This tool is a valuable resource for those looking to understand how language models work. It sheds light on GPT-2's inner workings by visualizing relationships between tokens.