LLM teams should define the language goal they need before optimizing metrics such as precision, recall, or F1.
2
Linguistic structure gives practical ways to improve tokenization, embeddings, context handling, and speech generation.
3
In one biology-question example, pragmatic prompting raised Falcon 7B Instruct from 7 correct answers out of 20 to 17, while reducing runtime to about two seconds locally.
Summary
Chris Brousseau argues that large language models should be designed with language itself in mind. He uses syntax, morphology, semantics, pragmatics, and phonetics to explain where current systems work and where they lose information. He considers syntax largely solved by generative models, while tokenization and embeddings are partly solved. Pragmatics can improve inference by giving a model context, rules, and opportunities to reason through a task. In his biology example, Guidance applied pragmatic instruction to Falcon 7B Instruct, increasing the score from 7 out of 20 to 17 out of 20 and cutting the local runtime to about two seconds. Brousseau also explains how frequency-based tokenization can mishandle numbers, newly popular words, and borrowed words. The final example concerns speech: text alone loses emphasis and melody, so systems that combine text with phonetic and audio information can preserve more of the intended meaning.
LLM projects need a language goal before they optimize metrics
Brousseau compares training an LLM with growing a beard. A person can measure length, but that does not say whether the result matches the intended style. He makes the same point about precision, recall, and F1. If a KPI is treated as the goal, a team may be late in the process or may have no clear destination. The model should be built toward a use case and a desired language behavior, with metrics helping measure progress toward that goal.
Language has several structures that models need to handle
Brousseau divides language into syntax, morphology, semantics, pragmatics, and phonetics. Syntax concerns grammar and structure. Morphology concerns how smaller units of meaning form words. Semantics concerns literal definitions. Pragmatics concerns the surrounding situation and how context changes interpretation. Phonetics concerns the production of speech, including articulation, sounds, and the relationship between writing and speech. These categories give teams a way to locate the source of a model's failure.
Current LLM methods handle syntax well and morphology partly well
Brousseau says generative LLMs can produce infinitely many combinations, which makes him think much of the syntax problem is already handled. Tokenization gives models a representation of word structure, while embeddings turn vectors into representations with meaning for the model. He estimates that morphology and embeddings are roughly 75 to 80 percent solved. He places pragmatics and phonetics further behind, since they require context and information that ordinary text may omit.
Dictionaries describe changing usage rather than permanent definitions
Brousseau presents the dictionary problem as a warning against treating dictionaries as final authorities on meaning. He describes Dictionary.com, Merriam-Webster.com, and the Oxford English Dictionary as updating their vocabularies over time to reflect current usage. That creates a design question for an LLM: should it describe language at one point in time, or should it keep adapting? A financial model may need less frequent language updates than a general-purpose model because the language in its domain changes more slowly.
Frequency-based tokenization can hide useful word structure
Brousseau connects the yeet problem to morphology and predictability. Byte pair encoding, SentencePiece, and ChatGPT's encoding rely on statistical patterns and frequency. He argues that this can produce bad representations for numbers and newer words. English has constraints on which sounds and letters can appear together, and those constraints can make a word predictable even when the word itself is new or uncommon. Using linguistic structure can therefore improve tokenization and the model's view of the input.
Multilingual training can help models segment borrowed words
The kimono example shows how a tokenizer trained around English can assign misleading meaning to parts of a borrowed word. Brousseau says an English-oriented tokenizer may see 'mono' as meaningful while failing to treat the rest of 'kimono' appropriately. He connects this to the observation that multilingual models can outperform monolingual models on the same tasks. Exposure to more languages gives the tokenizer more evidence about how words are formed and borrowed.
Pragmatic instructions can improve both answers and runtime
For a set of 20 college-level biology questions, Brousseau says a vanilla ChatGPT setup answered 7 correctly and initially took about four minutes, or about one minute after the connection warmed up. Running Falcon 7B Instruct locally with Guidance added pragmatic instruction and chain-of-thought prompting. The system prompted itself several times and used system prompts to draw out the model's knowledge. It answered 17 of 20 correctly and took about two seconds. He also mentions LangChain and vector databases as ways to add retrieval and more context.
Text-only speech systems lose emphasis that changes meaning
Brousseau uses the sentence 'I never said I loved you' to show how emphasis changes meaning while the written words stay the same. He compares text-to-speech systems such as Tortoise and 11 Labs with a speech-to-speech model and a model that combines phonetic information with an audio reference. He says the text-to-speech systems lose melody and phonetic information, while the combined approach sounds much better. His point is that reducing speech to text removes information that language models may need.
"It increased by 10 which increased by 50% accuracy and increased in speed like almost a hundredfold just from using some pragmatic instruction on the inference side."16:09
Who should watch
You are choosing evaluation metrics for an LLM but have not written down the language behavior the system should produce.
Your tokenizer performs poorly on numbers, borrowed words, newly popular terms, or multilingual input.
You work with speech or text-to-speech systems and need to understand what is lost when audio becomes text.
