LLMs can replace purpose-built text classifiers for some straightforward tasks, reducing development time, infrastructure work, and operating cost.
2
Canva used LLM embeddings on plain page text to group content pages, and this performed better than several combinations of traditional text feature extraction and open-source embeddings in the tested case.
3
Few-shot prompting is useful for a fast first release, while fine-tuning or open-source models may become more suitable when data volume, custom logic, or API cost grows.
Summary
Xin Lian describes two Canva use cases for LLM-based feature extraction. The first categorizes user search queries into branches of Canva's content architecture. A traditional classifier took about four weeks to develop, while an LLM API approach took about a week, used fewer than ten examples, and cost less in the reported workload. The second groups content pages by semantic similarity, even though the pages range from short metadata to long articles. Canva compared keyword extraction, summarization, key point extraction, and embedding approaches. LLM embeddings applied directly to page text produced the best results in the tested setup, grouping 89% of pages into relevant categories. Lian also explains the limits. LLM outputs can vary between runs, APIs have rate limits and downtime, and cost depends on scale. Aggregating inputs can reduce volume, while open-source LLMs may make in-house processing practical for larger workloads.
LLMs can act as a middle layer for existing machine learning systems
Xin Lian focuses on using large language models between existing inputs and downstream tasks. Instead of building a new application entirely around generated text, Canva uses LLMs to extract features that other systems can consume. This includes text intent classification, keyword extraction, summarization, key point extraction, and embeddings. She says the approach can improve accuracy and reduce development effort for suitable language tasks, although the results depend on the use case and operating scale.
An LLM API simplified Canva's search-query classification workflow
Canva organizes content in a tree structure, so user search queries must be routed through several classification steps. A traditional classifier required a full development workflow, including tens of thousands of labeled examples, training infrastructure, deployment infrastructure, and inference. The LLM approach used fewer than ten examples, a prompt structure with predefined classes, and few-shot learning at inference time. This removed the need to train and deploy a separate classifier for the first version.
The LLM approach shortened development and reduced reported monthly cost
For the search-query intent task, Xin reports about four weeks to develop the single-purpose classifier and about one week using an LLM API. In the reported weekly scheduled workload, which had hundreds of thousands of inputs, the classifier cost about $100 per month while the LLM API cost less than $5 per month. She also says the LLM API produced higher accuracy in this case, without fine-tuning.
Prompting is enough for an initial classifier, while fine-tuning can help at scale
Xin recommends LLM APIs for straightforward text tasks and early prototypes, where few-shot prompting can encode custom logic. She advises standardizing the completion format, such as using JSON or constraining the prompt to a fixed set of answers. Rate limits require throttling and a fallback for API downtime. When the data volume or custom logic grows, she says fine-tuning may improve performance and cost, and her use case needed roughly 50 to 100 training examples per class for acceptable results.
Content-page grouping needs a common representation across very different page types
Canva's content pages range from collections of templates with short metadata to informational articles with much longer text. The goal is to group pages by relevance and proximity in the information architecture. Before using LLMs, the team considered keyword extraction, text summarization, and key point extraction, followed by conversion into embeddings. Xin describes this as a scattered workflow with different libraries and methods for different text variations.
Plain page text with LLM embeddings performed best in the tested comparison
Canva compared different text transformations and embedding combinations using balance, completion, and coherence metrics. Balance measured how evenly pages were distributed across groups. Completion measured how many pages were assigned to relevant topics rather than left as outliers. Coherence measured whether grouped pages made sense together. LLM embeddings applied directly to plain page text gave the best outcome, grouping 89% of pages into relevant categories and achieving the highest coherence score among the tested methods.
The embedding approach reduced feature-extraction effort and reported cost
The traditional feature-extraction approach with open embeddings took about two weeks of development and cost about $3 per month in the described setup. LLM embeddings took about three to five days and cost about one third as much. Xin attributes the shorter process to using one foundation model for several text transformations instead of assembling separate methods for keywords, summaries, and key points.
LLM use becomes harder when output variation and workload scale matter
Xin cautions that LLM feature extraction is non-deterministic, so the same input can produce slightly different outputs depending on configuration and settings. She says this was acceptable for Canva's use case. For workloads involving millions of samples, she recommends aggregating or preprocessing inputs when possible. If reducing the volume is not practical, an open-source LLM hosted within the organization's infrastructure could avoid some external API costs, at the price of additional setup.
"The format and the length of the text inputs doesn't seem to affect the semantics understanding much in the LLM embedding space."17:07
Who should watch
You are deciding whether a simple NLP classifier is worth building and want a concrete comparison with few-shot LLM prompting.
Your product contains varied text fields and you need to turn them into embeddings or categories without maintaining several separate feature-extraction pipelines.
You operate LLM workloads at high volume and need to think through aggregation, rate limits, fallback behavior, fine-tuning, or self-hosted models.
