Foundation models can automate structured data work such as SQL generation, data cleaning, and data linkage by turning tasks into prompted generation problems.
2
Production systems need to control the cost, latency, and brittleness of foundation models, especially when processing millions or billions of records.
3
Numbers Station combines simple rules with foundation models and adds domain knowledge through distillation, prompt aggregation, continual pre-training, or external memory.
Summary
Ines Chami explains how foundation models can handle structured data tasks in the modern data stack. She covers natural-language SQL generation, cleaning records with prompts, and linking records across systems such as Salesforce and HubSpot. The talk is clear about where these approaches fail. Large models become too slow and expensive for whole databases, prompts can produce different answers after small formatting changes, and public training data does not contain an organization's definitions or metadata. Chami describes practical responses: use rules for simple cases, call the model for harder examples, distill a large model into a smaller one, aggregate outputs from multiple prompts, and provide enterprise knowledge through documents, logs, metadata, knowledge graphs, semantic layers, or search indexes. The central engineering choice is deciding where a foundation model adds value instead of applying it to every row or every task.
Foundation models generalize across tasks through prompted generation
Chami defines foundation models as large neural networks trained on massive amounts of unlabeled text or image data with self-supervised learning. Autoregressive language models learn to predict the next word from previous words. Their scale enables in-context learning: a task can be expressed as a prompt, and the model completes that prompt. The same underlying model can then be reused for many examples and for different tasks without building a separate model for each one.
The modern data stack still contains substantial manual work
Chami describes the modern data stack as the tools used to process, store, and analyze data. Data often starts in applications such as Salesforce or HubSpot, moves into warehouses such as Snowflake, gets transformed with dbt, and is then used for visualization. These tools have improved scalability and knowledge sharing, but people still spend significant time handling work across the pipeline. Numbers Station applies foundation models to this work with the goal of reducing the time needed to reach useful insights.
Foundation models can generate SQL from business questions
Business users often send requests to data engineering teams when they need an answer, and the resulting back-and-forth can take time. A foundation model can translate a natural-language request into SQL and avoid some of those iterations. Chami says this works very well for simple queries. Complex queries still require domain knowledge the pretrained model may not have. If a table contains several date columns, for example, the model may not know which date the business intends to use.
Prompted models can clean and link records, with rules handling the easy cases
For data cleaning, teams commonly write SQL rules to fix typos and missing values. Writing enough rules takes time, and an unhandled edge case can cause a rule to fail. A prompt with examples lets a foundation model infer correction patterns across records. The same approach can compare records from systems without a shared identifier, such as Salesforce and HubSpot, and ask whether two records refer to the same thing. Chami recommends composing rules with the model: use a basic rule for the large, easy part of the task, then call the model for complex last-mile cases.
Large models are too expensive for many full-database workloads
A foundation model can fit a human-in-the-loop SQL copilot because latency matters more than processing an entire database. The economics change when the model must predict on millions or billions of rows. In that setting, a large model can be extremely slow and expensive compared with rules. Chami proposes using the large model for prototyping and then teaching a smaller model through distillation. She also recommends using the foundation model to derive rules from data when a rule can solve the task, rather than calling the model for every record.
Prompt formatting and examples can change the result
Chami describes prompt brittleness as a serious issue for data applications. Changing the prompt format can change the prediction, and the examples placed in the prompt matter as well. In an experiment on the cleaning task, manually selected demonstrations produced a large performance difference compared with random demonstrations. This is uncomfortable in the modern data stack, where users expect deterministic outputs and may not accept unpredictable errors even when automation saves manual work.
Multiple prompts and better prompt construction can reduce brittleness
One method from the AMA paper applies several prompts to the same input and aggregates their predictions into a final result. Chami says this improved performance over traditional prompting in their experiments. She also names prompt decomposition with chains and more careful sampling of demonstrations as other ways to address brittle behavior. These methods do not remove the need to evaluate outputs, but they can make a prompted system less dependent on one exact wording or one small set of examples.
Enterprise data needs domain knowledge at training time or inference time
Public training data does not contain every organization's definitions. A request for the number of active customers may be impossible to answer correctly if the database has no direct is-active field and the organization has its own definition of an active customer. Chami describes training-time approaches that continually pre-train open models on internal documents, logs, and metadata. Inference-time approaches add external memory through a knowledge graph, semantic layer, or search index over internal documents. Both approaches give the model information that was absent from public data.
