Salesforce uses small language models in Data Cloud to automate data harmonization at large scale.
2
Identity resolution first uses a 300-million-parameter embedding model to select candidates, then a 66-million-parameter distilled BERT model to score matches.
3
Small models and quantization reduce serving cost, hardware use, latency, and deployment requirements for focused data-processing tasks.
Summary
Shelby Heinecke explains how Salesforce uses AI inside its Data Cloud platform. Data arrives from many sources, but ingestion alone is not enough. The platform must harmonize records so that information about the same customer can be brought together and used by downstream AI. Salesforce handles identity resolution in two stages. A 300-million-parameter embedding model selects likely candidates from very large datasets. A 66-million-parameter distilled BERT model then scores pairs of records to decide whether they match. Heinecke argues that small language models are a practical fit for focused, high-volume operations because they are cheaper and faster to serve and can be fine-tuned for a defined task. Quantization reduces model precision and makes the footprint and latency smaller while retaining most performance. In the discussion, she says larger models are better for broad, flexible tasks and can be more tolerant of noise, while hardware and deployment constraints also determine model choice.
Heinecke says Salesforce customers bring data from many sources into Data Cloud, including sales, marketing, service, and other systems. Ingestion creates a large pile of records, but the value comes from organizing those records into a single source of truth. A customer record should bring together all data linked to that person. Harmonized data also makes it easier to fill prompts and produce useful analysis because the AI can read from data that has already been mapped and organized.
Identity resolution links inconsistent records to one person
The identity-resolution problem occurs when one customer appears differently across social media, email, purchase, and customer-service data. Names may change, addresses may reflect a move, and records may contain typos or human errors. Salesforce uses AI to determine when these separate records refer to the same person. The process has to handle differences in text while still merging the relevant data pieces.
Candidate selection prevents comparisons across the entire dataset
At Data Cloud scale, a target person could otherwise be compared with millions, billions, or trillions of rows. Heinecke describes candidate selection as the first step: the system narrows the data to records that are likely to match the target. Salesforce uses an embedding model with 300 million parameters for this stage. The model is already being used by customers for the candidate-selection step.
A second small model makes the final identity decision
After candidate selection, the system needs a finer-grained comparison. Salesforce uses a distilled BERT model with 66 million parameters. It receives a target and a candidate as a pair and produces a score. That score helps determine whether the candidate is close enough to count as a match. The two-stage design separates broad retrieval from the more careful matching decision.
Small models fit high-volume, well-defined operations
Heinecke contrasts the small models used in this system with large models that have hundreds of billions of parameters. She describes models below 13 billion parameters as small language models for this discussion. Their smaller size reduces serving cost and hardware consumption, and self-hosting avoids paying an external API to run the model. They also process requests faster, which matters when a data platform must handle very large numbers of records.
Small models are useful when the data platform knows exactly what operation needs to happen. Heinecke says fine-tuning can make a small model exceptionally good at a clearly defined task. In Salesforce's case, the models focus on candidate selection and identity matching rather than trying to solve every language task. Their limited scope is an advantage when speed and cost matter.
Quantization reduces the precision of model weights. Heinecke says weights commonly use 32-bit or 16-bit floating-point values, while quantization can reduce many of them to 8, 4, 3, or 2 bits. The result is a smaller footprint and better latency while retaining most of the model's performance. She presents quantization as an additional step after choosing a small model.
Model choice depends on task breadth and available hardware
In the discussion, Heinecke recommends small models for focused use cases with a limited scope, especially when they can be fine-tuned for that use case. Larger models are more suitable for flexible and broad tasks because they generalize better, handle noise more robustly, and offer more flexibility. Deployment hardware also matters. A phone requires a small model, while access to more GPUs may allow a medium-sized model.
"The result is for the most part most of the performance is going to be retained but the footprint is going to be even smaller, the latency is going to be even better."09:49
Who should watch
You are building a data platform that must link customer records from several systems before AI can use them.
Your team is deciding whether a small, fine-tuned model can replace a larger model for a defined production task.
You need to run models on limited hardware or process very large record volumes with low latency.
