Machine learning pipelines begin with how a business collects, structures, and maintains data, not only with feature extraction and model training.
2
Thoughtful data collection and storage can reduce pipeline costs because every event creates later compute, storage, and processing work.
3
Data scientists need business context and clear data semantics so they can produce actionable insights without spending days tracing undocumented data sources.
Summary
Satish Chandra Gupta explains machine learning from a software engineering and data infrastructure perspective. He separates the pipeline into collection, injection, preparation, computation, and presentation, while stressing that these stages overlap across data engineering, machine learning, software engineering, and business roles. The data pipeline starts when an application generates events. Poorly designed collection creates storage, transformation, and debugging costs, while a useful schema and clear catalog make data easier to access and trust. Satish describes a taxi use case with streaming events, strict latency needs, batch processing, and model outputs that reach customers as notifications. He also discusses how data lakes and warehouses have begun to converge, using BigQuery as an example. In the final section, he describes Slang Labs' attempt to provide voice assistants inside applications. The team learned that a technically elegant SDK still required too much NLP knowledge from ordinary application developers, so they moved toward domain-specific assistants and more carefully designed voice interactions.
Satish moved into machine learning after seeing program analysis solved with data
Satish began as a software engineer building compilers, program analysis tools, development tools, IDEs, and memory managers. At Microsoft Research, he saw a colleague use machine learning to solve a difficult program analysis problem. After spending about ten years mastering conventional techniques, Satish was surprised that mining data could solve at least part of the same problem. AlexNet was another turning point because it improved computer vision results by more than ten percent, which he described as unusually large for research. He taught himself machine learning and later moved into microservices, ML applications, and data pipelines.
A machine learning pipeline starts with the data a business already generates
Satish distinguishes between preparing data for training and making sure useful data reaches the data lake in the first place. Businesses often already collect data through applications, analytics systems, and warehouses, yet much of it cannot be trusted or understood. He calls this an "island on the lake": a small portion of a large data store that people can actually use. The rest may be only bytes occupying storage. Satish says application teams can help future data scientists by recording meaningful, semantically complete information rather than dumping every event as it happens.
The strongest pipeline work crosses engineering, machine learning, and business boundaries
Satish's five stages are collection, injection, preparation, computation, and delivery. He groups them around data engineering, machine learning or analytics, and presentation, but says this does not require three separate people. People who cross into adjacent disciplines understand the circumstances more fully. Data engineers focus on pipeline discipline, data hygiene, and cost. Data scientists focus on statistics and models. Business-facing work asks whether the result changes a business decision. Satish says innovation often happens at these intersections, and that more engineers are moving beyond strict professional silos.
Data access, scale, efficiency, and monitoring describe different pipeline needs
Accessibility means that data scientists can find, understand, and query data quickly. Catalogs and organized data reduce the need to inspect generating code or ask another engineer what a dataset means. Scalability concerns the volume of data and how quickly it must become available. Satish describes a taxi system receiving a beacon from every cab every five seconds and needing to act within thirty seconds. The system processed about a billion events per day on a three-node cluster. Monitoring answers whether the pipeline and model continue to work, rather than leaving teams to discover days later that a job failed or a model became stale.
Data collection should capture meaningful states instead of every control-flow event
Satish says programmers often think in control flow, while data scientists need to reason about data flow. When applications dump small pieces of data whenever code reaches a step, later analysis has to reconstruct what happened. His approach is to identify "sentinel states", points where the application's state is semantically complete, and emit data there. This reduces the amount of computation and storage required downstream. He gives an example where a deployment's monthly pipeline bill fell from thirty thousand dollars to five thousand dollars after the team changed what it collected. He considers twenty to twenty-five percent savings a reasonable expectation in many cases, though he presents that as an estimate from experience.
Real-time processing should be limited to decisions that need immediate action
Satish separates streaming data from batch data and says teams should decide which business needs truly require real-time processing. In the taxi example, an empty cab can represent lost revenue, so the system must make a decision within thirty seconds. A report about the previous day's business can wait until a scheduled batch job. He describes a lambda architecture in which a real-time path handles urgent decisions and a batch path later adds fuller processing. Outputs can include reports, interactive data access, and streaming notifications to customers. Fraud detection is another example where the output may need to be delivered immediately.
Data lakes and warehouses are becoming less distinct as storage and query options change
Satish explains that warehouses brought data from separate systems into a central structure, while data lakes emerged when businesses needed to store much larger volumes of structured, semi-structured, and unstructured data. The flexibility of lakes also made them easy to pollute with poorly documented data. He says BigQuery changed the trade-off by making storage relatively inexpensive and charging mainly for query computation, with serverless execution. Materialized views, flexible columns, and SQL access to JSON-like fields further narrow the difference between lakes and warehouses. Teams can keep frequently needed data in a hot warehouse and less time-sensitive data in a cheaper lake, provided they maintain a catalog.
Useful schemas begin with business decisions and available levers
Satish argues that teams should start with the business goal, identify the action or lever that can change it, and then determine what data is needed. A statistic such as the number of sign-ups matters only when it supports a decision. For a taxi business, the relevant question might be where to place vehicles or how to influence demand. This reasoning determines what to collect, how to structure it, and whether it needs real-time access. Satish recounts spending three days investigating a supposedly broken model before learning that the live system used a different stream from the curated data. A defined schema and data ownership would have prevented that surprise.
Slang Labs learned that a simple voice SDK still demanded too much specialist knowledge
At Slang Labs, Satish and his co-founders wanted programmers to add an in-app voice assistant without implementing speech recognition and NLP themselves. Their first design mapped intents to application functions and passed collected entities to those functions. Although this made development much easier for the founders, ordinary application developers still needed to understand concepts such as intents, entities, and NLP training. The team therefore moved toward domain-specific, drop-in assistants for areas such as travel, groceries, and pharmaceuticals. Satish also learned that voice interfaces need deliberate prompts and recovery flows because users have no established interaction primitives to follow.
