AI platforms must handle much larger and less structured data while keeping performance, cost, and scale in balance.
2
Data engineering remains most of the work, while AI adds feedback loops, uncertain transformations, and a stronger need for checks and observability.
3
Small teams should choose AI projects based on business value and optimize for time to learning before building custom infrastructure.
Summary
The panel argues that AI platforms should grow from existing data engineering practice instead of becoming a separate stack. Structured and unstructured data need to work through a shared architecture, with intermediate states available for different applications. AI changes the shape of pipelines because models create feedback loops and transformations that are harder to inspect. The panelists discuss embeddings as reusable representations for downstream machine learning and stress that metadata and relevant signals must be included in those representations. For small teams, the advice is practical: tie projects to revenue, savings, or another clear business outcome; use managed services when they reduce time to learning; and start with a small application that can be launched and improved. The panel closes with a prediction that data, product, application, and operations teams will need to work more closely as AI becomes part of the full user experience.
AI platforms must account for the scale and messiness of unstructured data
Colleen Tartow says data platforms have long balanced performance, scale, and cost. AI changes the scale because unstructured data is much larger and more complex than tables and SQL data. A platform that worked for structured workloads may deliver poor performance or unacceptable cost when applied to documents and other unstructured material. She argues that organizations should first understand the value they expect from an AI use case, then decide what infrastructure to build. Daniel Svonava adds that unstructured data should not become a separate project. Logs, documents, and usage signals can provide context to one another.
AI data work should stay connected to the rest of the organization's data
Svonava describes data types as a spectrum rather than a clean split between structured and unstructured data. A document is more useful when combined with information about whether people access it and which parts they use. That context can come from other first-party systems. He also says AI transformations are harder to inspect than traditional semantic-layer definitions, so platforms need more observability, assertions, and checks. Treating language models as systems that simply consume PDFs loses useful organizational context. The panel's recommendation is to integrate AI data work with the broader data effort instead of isolating it.
Data engineering still accounts for most of the work
Tartow says data engineering remains about 90% of the work in pipelines. Engineers still take data from sources, curate it in a business context, and prepare it for business intelligence, AI, or traditional machine learning. The goal of producing clean, usable data has not changed. Tools and data types may change, including synthetic data, but teams still need shared standards and consistent curation. Svonava suggests that the boundaries between data engineering and machine learning engineering may continue to move as machine learning work becomes more standardized. The platform responsibilities do not disappear when the model becomes easier to use.
AI pipelines need feedback loops and access to intermediate states
Tartow contrasts traditional business intelligence pipelines with AI systems. A traditional pipeline can run from a source through transformations to a consumption layer, while AI systems receive feedback that can affect later steps. That means data engineers may need to work at both ends of the pipeline. Tobias Macey connects this to a broader architectural change: teams need a hybrid warehouse and lake approach that can handle structured and unstructured data through one control plane. Intermediate states matter because different AI applications may need different representations. Rebuilding the architecture for every new use case would be expensive.
Embeddings can give downstream projects a shared representation of an entity
Svonava describes a customer embedding as a way to combine many signals about a customer into one numerical representation. Instead of beginning every machine learning project with raw logs and separate feature engineering, a team could start from a reusable customer vector. The representation might include purchases, behavior, website context, and other signals. It could then support personalization, grouping customers by behavior, or later model training. He extends the same idea to knowledge-base documents and their usage. The embedding becomes a shared point where signals are integrated before they feed different downstream applications.
Metadata must carry meaning without making vector search a sequence of disconnected filters
The panel agrees that vector stores need metadata such as tags and other context. Macey says metadata makes vectors easier to filter, enrich, and reuse across applications. Svonava adds that important metadata should also be encoded into the embedding. He warns that a system can lose much of the value of vector search when it first applies many filters and then searches only within the narrowed result set. At YouTube, he says, signals such as popularity, user types, content, moderation scores, and outputs from other models were included in the embedding. The vector therefore reflected more of the recommendation problem itself.
Small teams should optimize for business value and time to learning
Tartow advises small teams to keep their attention on the business outcome because they cannot afford open-ended AI research. She points to managed tools, open source, and GPU cloud services as ways to avoid building every component themselves. Svonava says the best AI metric is often money saved or made, and teams need to work closely with the product manager who understands the use case. Tartow adds that small teams should optimize for time to learning rather than maximum flexibility. A paid platform may be worthwhile if it helps a team launch quickly. Custom infrastructure can follow once the team understands its actual requirements.
AI work should reframe existing business problems instead of adding fashionable features
Svonava advises teams to apply creativity to known business problems rather than inventing new AI features without a clear need. If user retention is poor, the team can examine the user experience, identify where intervention could help, and then decide whether AI fits that problem. A chatbot added to an otherwise healthy product may be harder to justify than an AI system tied to an existing retention or operational issue. Connecting the project to an established business problem also makes it easier to secure budget and agreement from managers. The panel sees this as a practical way to reduce unnecessary complexity.
"The best metric to evaluate for AI system, you know, there are all these sophisticated ways, but favorite one is USD."Daniel Svonava23:43
Who should watch
You are responsible for a data or AI platform and need to decide whether data engineering and machine learning infrastructure should be combined.
Your team is small and needs to choose between managed services, open source, and custom infrastructure without losing sight of the business outcome.
You are designing systems with embeddings, vector stores, or unstructured data and need guidance on metadata, feedback loops, and reusable representations.
