Verticalized language models can handle enterprise data tasks more accurately because they are given the data schema, query history, and business context that general models lack.
2
Smaller, task-specific models can reduce hosting costs, run inside an enterprise VPC, and support high-volume jobs such as entity resolution.
3
Model evaluation should measure the result a customer cares about, such as the output table produced by SQL, rather than comparing generated SQL strings alone.
Summary
Venky Ganti and Laurel Orr explain why general-purpose LLMs often struggle with enterprise data. A public model can write SQL for familiar schemas, but real companies have obscure column names, many tables, private query logs, and business rules that the model has never seen. Numbers Station focuses on private data models for conversational analytics, entity resolution, data wrangling, and related work. The guests argue for verticalization, with model size chosen for the workload. A larger model may suit occasional questions from business users, while a small tuned model is better for repeated production jobs. They discuss the practical work of collecting and labeling customer data, building evaluations, and keeping models inside a customer VPC. Laurel expects more local models and longer context windows, while Venky expects enterprise demand for private models to grow. Both describe model selection as an ongoing tradeoff between accuracy, cost, latency, and system complexity.
Number Station builds private models for enterprise data work
Venky Ganti describes Number Station's focus as private data LLMs for enterprises. One application lets business users ask questions of company data in a conversational way, with the model using enterprise context to produce more accurate answers. He also names entity resolution and data wrangling as customer needs. Laurel Orr joined after working with Chris Re at Stanford and becoming convinced that large models would change how engineers build applications and interact with data.
Data wrangling still needs engineers and domain experts
Venky explains that wrangling includes schema and data transformations, cleaning, standardizing column names, filling null values, and matching records that refer to the same entity. Customers in insurance and real estate are interested in replacing rule-based entity-resolution systems that take time to build and maintain. He is clear that LLMs do not remove data engineers. Engineers still tune the models, and domain experts guide them toward the right behavior.
General models know SQL syntax but miss private business context
Laurel says models trained on public sources can learn general SQL, including common schemas and query patterns. They struggle when a company has hundreds of tables, columns with unexplained short names, private pipelines, and business logic known only by a few engineers. The model has not seen the company's data or query logs, so its answer may be a plausible guess rather than a correct interpretation. This is why enterprise text-to-SQL requires access to the customer's own context.
Verticalization improves accuracy and gives models useful guard rails
Venky identifies two steps for complex enterprise tasks. The model must first be specialized for a data task such as text-to-SQL or entity resolution. It must then learn the customer's schema, values, query patterns, reports, and the purpose behind those queries. A private data model can also reject questions outside its scope, such as a weather question, or say when it lacks enough information to answer accurately. Those guard rails are harder to define in a general-purpose chatbot.
Smaller models make private hosting and customization more practical
Laurel argues that a model focused on text-to-SQL does not need most of the information stored in a broad model that also handles recipes, document summaries, and many other tasks. A smaller model can run locally with more reasonably priced hardware, which lets an enterprise host and customize it inside its own VPC. Venky adds that reducing the model's footprint can lower cost while making it easier to provide the specific context needed for accurate answers and to limit hallucinations.
Laurel prefers tighter task isolation when engineers need to customize models, since changing one behavior in a large multitask model can hurt another task. Venky separates human-driven questions from system-driven jobs. A business user asking for a sales aggregation can tolerate a larger model and more latency. Entity resolution may run millions of times, so it needs a small, efficient model tuned to a specific customer's data. Related tasks may share a model when transfer learning can carry useful information between them.
Laurel says that companies rarely have a ready-made collection of text-to-SQL examples. Teams must curate logs and data dumps, create evaluation sets, label examples, and deal with privacy restrictions. Fine-tuning an existing model generally needs less data than training from scratch, but the quality of the available customer data still determines the result. Venky says Number Station starts with a verticalized model and adds the customer's schema, queries, and reports in a process he describes as taking weeks, followed by continued fine-tuning as the context changes.
Evaluation should follow the customer's actual outcome
Laurel says evaluation is custom because LLM tasks can combine free-form text with classifications, actions, or API calls. Standard metrics are easier for classification and less useful for free-form outputs. For text-to-SQL, comparing two SQL strings misses the point because many different queries can produce the same answer. The more useful test is whether the resulting table matches what the user needs. For some constrained classes of SQL, that comparison can be automated.
"The single most important factor to getting a high quality model is the data that comes in."Laurel Orr25:26
Who should watch
You are deciding whether a general LLM is suitable for text-to-SQL or enterprise data cleaning, and need to understand where private schema and business context matter.
Your team is choosing between one large model and several smaller models for human questions, batch jobs, or high-volume entity resolution.
You need a practical approach to collecting customer data and evaluating generated SQL by its result rather than by string similarity.
