Build or buy decisions should start with the business problem, then work backward to the required model, data, infrastructure, and controls.
2
Commercial models and products can provide speed and managed safeguards, while open models offer more control, customization, and local hosting at the cost of additional skills and infrastructure.
3
Gen AI adds to traditional ML rather than replacing it, so production systems still need analytics, data platforms, testing, governance, and model evaluation.
Summary
The panel separates several decisions that are often mixed together. An organization might buy a product with an embedded model, build a custom application around a commercial API, or host and adapt an open model. Buying can provide speed and managed controls, while building can improve customization, data control, and fit with the organization's policies. The panel argues that the choice should begin with the problem and the sensitivity of the data. Enterprise offerings may keep data inside an organization, but users often cannot see how a model was trained or which model is operating behind a product. The speakers also discuss hallucinations, regulatory debt, regional language and cultural needs, and the difficulty of moving between models. Gen AI does not remove traditional ML. Recommendation systems, customer data platforms, analytics, and other existing components still have work to do. A chatbot proof of concept can start with a managed API, then move to self-hosting if production risk and control requirements grow.
The model market divides into closed APIs and open models
Nayur Khan describes closed models as commercial systems accessed through an API, such as OpenAI or Google Gemini. Their inner workings are opaque and the infrastructure belongs to someone else. Open models, including Mistral, Mixtral, Falcon, and Meta's Llama 2, can be taken off the shelf and hosted or adapted by an organization, although their licenses and restrictions differ. Hugging Face contains a very large collection of models. This distinction frames the later choices between buying a ready-made capability, building around a model, and choosing commercial or open technology.
Buying can mean purchasing a whole product or only accessing a model
The panel separates products that already include generative AI from custom applications that an organization builds itself. Microsoft, Salesforce, SAP, and GitHub are examples of existing products adding LLM capabilities. With an off-the-shelf product, the user may not choose the underlying model and may have limited visibility into how it works. A custom application gives the organization more choices about the model, integration, data, and controls, but requires more engineering work. This distinction matters because build versus buy is not one decision. It can involve the user-facing product, the model engine, and the surrounding infrastructure.
Data boundaries and model behavior should guide the risk decision
Ilona Logvinova describes an enterprise model as a closed circle. Data can remain on premises or in the organization's cloud environment, with insights generated from that internal data. A second arrangement keeps organizational data inside the boundary while allowing the tool to draw information from external sources. The trade-off includes confidentiality, data use, and the quality of generated answers. Logvinova explains that generative systems can hallucinate because they are designed for creativity, while multi-step reasoning, including some mathematical tasks, is harder for them. Organizations must decide how much creativity they need and how much correctness and containment their use case requires.
The right build decision starts with the problem and the organization's constraints
Mohamed Abusaid says buying can provide a capability quickly, but usually limits customization and control. Building can produce a better fit, yet requires more money, time, skills, data, infrastructure, and often GPUs. He rejects starting with a technology and searching for a problem afterward. The panel recommends defining the problem, understanding why it matters, and then deciding whether an LLM is needed at all. Ilona adds that the market is immature and changes quickly, so organizations should ask whether a capability is core intellectual property, central to the product, or likely to become a standard capability supplied by the market.
Model transparency is still limited, even when organizations want it
The panel does not identify a ready-made product that reliably tells users how a model was trained, which datasets were used, or which model is operating behind an application. Even open models may disclose code without making the complete training data reproducible. Managed services can provide data-use agreements, but that is different from giving an end user a clear view of model provenance. Ilona notes that providers such as Anthropic, Cohere, and OpenAI have different principles and behavior, but users can assess those differences only when they know which provider and model a product uses.
A chatbot can move from a managed proof of concept to a self-hosted system
For an internal chatbot, an organization might keep its existing ServiceNow interface, use a managed model such as Azure OpenAI, and rely on existing cloud contracts and data protections. Production also requires an API, integration work, a vector store, infrastructure, and connections to organizational data. If the chatbot becomes external-facing and handles sensitive customer data, the organization may decide to host an open model itself. The panel presents this as a practical progression: use a commercial API to test value quickly, then consider a self-hosted model when production scale, control, and risk requirements justify the additional work.
Gen AI adds a generative layer to traditional ML systems
Nayur Khan gives a retail example in which traditional analytics uses purchase history and behavior to decide which customer should receive which message, through which channel, and at what time. Gen AI can then turn that decision into a message with an appropriate tone. The analytical model, customer data platform, and MLOps processes still matter. Gen AI is useful for summarizing, extracting, paraphrasing, and generating language, but it is weak at precise mathematics and some financial calculations. Production teams therefore need to combine generative systems with existing analytics, data, testing, and governance practices.
Changing models requires prompt versioning, testing, and an abstraction layer
Abusaid compares changing an LLM to changing a car engine. Prompts can behave differently across models and versions, including between OpenAI model versions. Teams should abstract prompts, store metadata about which model and version each prompt supports, and test when they change providers. A model gateway or similar indirection can make it easier to switch models during evaluation. The panel also recommends recording prompts and responses and continuously checking that outputs remain within acceptable bounds. Managed model behavior can change, so a proof of concept should be made reproducible and testable before it becomes a production dependency.
"Make sure you have written enough prompts and recorded those responses and communicated some sort of mechanism to constantly evaluate what you're running against."Nayur Khan53:35
Who should watch
You are deciding whether to buy a Gen AI product, build a custom application, or host an open model, and need a way to separate those choices.
Your organization handles confidential, regulated, or region-specific data and needs to understand model boundaries, transparency, and vendor questions.
You have a proof of concept built on a commercial API and want to prepare for model changes, production testing, or a move to self-hosting.
