Language models work well because they learn useful representations of language in latent space, beyond raw words and tokens.
2
Sparse autoencoders and related interpretability methods can identify human-interpretable features in embeddings and let users edit those features during generation.
3
Semantic interfaces could let people read, write, copy, and share text by manipulating properties such as topic, tone, style, and technicality.
Summary
Linus Lee argues that language models may be easier to work with when represented through the features they learn internally, rather than through tokens alone. He compares this with binary numbers, color models, and spectrograms, where a different representation makes new operations possible. Using sparse autoencoders and automated interpretability, Lee shows features that correspond to topics, grammar, dates, questions, dialogue, and other properties of text. His demos turn features on and off inside an embedding model, changing a paragraph's topic or phrasing it as a question. He also shows a heat map for exploring a novel by features such as dialogue, narration, movement, and negation. Lee proposes semantic copy and paste, writing interfaces with separate tracks for topic and tone, and shareable style patches. The aim is to give text more visible structure and physical affordances, so people can understand and manipulate ideas without inspecting every token.
A useful representation changes what people can do with information
Lee begins with examples where the representation determines the available operations. The IBM 1401 used decimal numbers, which helped with financial work but added hardware and algorithmic complexity. Modern computers use binary representation because it is simpler and more compatible. Color can be represented with RGB or HSL, and HSL makes operations such as increasing brightness or saturation easier to express. A spectrogram changes sound from a waveform over time into a frequency view, making it easier to work with bass, mids, and upper frequencies. Lee's point is that useful representations come from understanding the thing being represented, then expose operations that make sense for users.
Language models may reason through latent representations rather than raw tokens
Lee's hypothesis is that neural language models work well because they learn a more useful representation of language in latent space. Humans interact with words and tokens, while the model may turn complex thought into simpler operations in a geometric or spatial representation. Interpretability research gives access to some of this internal structure. Earlier vision models showed neurons responding to textures, color contrasts, brightness, and other recognizable properties. Similar work in text and image models has found directions related to art style, location, content, sentiment, and topic.
Sparse feature discovery can expose the terms a model uses to understand text
Lee describes a technique based on two assumptions: models learn many features during training, and each example activates only some of them. A secondary model can use internal activations to find a small set of features that reconstructs the original output. In his embedding-model tool, the features include biology, medicine, law, transitional phrases, first-person narration, and the presence of specific numbers. A language model generates labels for the features by examining their strongest examples. Larger models and more expressive embedding spaces can encode increasingly detailed properties.
Internal features can be edited to change generated text
Lee demonstrates more than feature inspection. He takes a paragraph about Taylor Swift and asks which features activate when the model processes it. The model identifies lists, dates, timestamps, and female pronouns. He then turns a feature on inside the model. The paragraph stays mostly the same while its subject shifts from music toward a mixture of music and law. Turning on a question-related feature causes the text to be phrased as a question. Lee compares this with editing music in the frequency domain, since the text is changed through semantic features rather than individual tokens.
Feature heat maps could make long texts easier to inspect
Lee shows a tool applied to the opening of Neuromancer. It identifies prominent features across the sentences and highlights the passages that activate them most strongly. A dialogue feature marks dialogue passages, while first-person narration appears as a heat map. Other features show physical interaction, movement, questions, and negation. He suggests that this could let readers zoom out from individual words and see where different kinds of meaning appear in a document. A scroll bar could carry a semantic spectrogram that marks areas focused on opinion, facts, politics, finance, or other features.
Semantic editing could separate topic, tone, and style
Lee proposes interfaces that manipulate text through semantic properties. Semantic copy and paste could copy only the tone of one passage, or only its topic, and apply that property to another passage. A writing tool could borrow the track-based design of music software, with separate tracks for topic, tone, and technicality. Compiling those tracks could produce an essay that follows the selected settings through a passage. Users could also share style patches containing feature settings that others apply to their own writing. These ideas move text editing away from selecting and moving words.
Interpretability methods can reveal model features without assuming them in advance
Lee says this approach differs from methods that begin by demanding a capability from the model. Instead, it asks what the model sees in an input. In the question session, he explains that sparse autoencoders or dictionary-learning methods train on internal activations and find a small number of features that reconstruct the output. The discovered features are initially unlabeled. Automated interpretability then uses a language model to inspect the examples that activate each feature and suggest what the feature controls. He also mentions representation engineering and inference-time intervention as related approaches.
Better representations could give text the visible affordances of physical materials
Lee wants text to communicate more about its structure and possible uses at a glance. Physical materials have texture and visible properties that show how they can be handled and combined, while text usually requires reading every token. A paragraph might instead reveal its topic, style, and similarity to other passages through color, shape, or other visual cues. Lee connects this to bringing thought outside the head, so concepts can be seen and manipulated directly. He expects interpretability research to develop from early methods toward techniques that work with larger models, creating room for more interfaces for reading and writing.
"The representation that you use to represent color kind of impact the space of operations that you can imagine and the kinds of operations you can intuitively expose to the user who wants to really understand and work with color."07:09
Who should watch
You are building interfaces for reading, writing, or knowledge work and want interaction patterns beyond chat.
You work with embeddings or language-model internals and want a concrete explanation of feature discovery and control.
You are exploring interpretability and need examples of how internal model features could become user-facing controls.
