Barak Turovsky says generative AI is entering a second wave because large language models are becoming available beyond large technology companies.
2
Large language models fit creative and workplace productivity tasks better than high-stakes information and decision support, where accuracy, latency, freshness, and user trust matter.
3
MLOps teams will need stronger data, retrieval, evaluation, and product skills because generative AI changes the systems and workflows behind customer-facing applications.
Summary
Barak Turovsky compares the current spread of large language models with the earlier AI wave that brought deep neural networks and custom hardware into Google Translate. He explains that Google had to solve production latency, quality, and hallucination problems before serving the technology to hundreds of millions of users. His framework rates use cases by fluency, accuracy, and the consequences of mistakes. Writing, creative work, and business productivity are suitable early applications because people can review the output. Search, bookings, insurance, and other decisions involving money or time need much higher accuracy and will take longer to gain trust. Turovsky expects customer interactions, entertainment, coding, and education to change substantially. He also argues that companies cannot simply place an LLM on top of existing tools. They need cleaned data, retrieval systems, exception handling, feedback loops, and teams that understand model limits. Machine learning engineers should learn product thinking and challenge unclear use cases.
Generative AI is opening a second wave beyond large technology companies
Barak Turovsky describes the first AI wave as the 2015 and 2016 breakthrough that made deep neural networks practical at scale. Google paired software advances from Google Brain with custom Tensor Processing Units, and Google Translate became the first product he led that used the technology at scale. That work was concentrated in companies such as Google, Meta, Microsoft, and Amazon. He says the current wave is different because startups and other organizations can access large language models through simple interfaces. This broad access will push applications into more industries, although he expects some use cases to move much faster than others.
Production AI required solving hardware, latency, quality, and safety problems
Turovsky explains that academic teams initially presented neural-network systems trained on much smaller data sets, while Google Translate worked with billions of training examples. The software could process those examples, but it was about 100 times slower in production than the existing system. Jeff Dean asked what would be needed, and the answer was new hardware. Google invested $130 million in custom hardware before there was a clear monetization use case. The team expected the first language launch to take three years, then launched about 20 languages in nine months. Turovsky says the production work also involved quality, latency, offensiveness, and hallucination problems.
Turovsky separates fluency from accuracy. Fluency means that a system can produce polished, confident, human-sounding language. It does not mean that the answer is correct. He compares this behavior with people who speak confidently about subjects they do not understand and fill gaps by making things up. That distinction matters when teams choose applications for large language models. A system that writes smoothly may be useful for drafting, while the same behavior is dangerous when users rely on its answer for a business or personal decision. His framework therefore treats fluency and accuracy as different dimensions rather than assuming that one implies the other.
Human review makes drafting safer than automated decision support
Turovsky places use cases on a grid involving accuracy, fluency, and the stakes of an error. Poems, children's books, music, and science fiction need a compelling story, and there may be no single correct answer. A business email or presentation can also benefit from machine-generated fluency because a person can review the facts before sending it. Search and recommendations are harder because a person cannot validate every query at scale. He groups creative and workplace productivity applications as better early fits. Information and decision support, such as choosing a hotel, appliance, insurance product, or business action, needs higher accuracy and will take longer.
Trust in high-stakes assistants will grow more slowly than creative use
The conversation turns to voice assistants and automated purchasing. Turovsky says Amazon already had the technical ability to identify products and ship them, yet people mainly used voice assistants for low-stakes tasks such as alarms and music. He sees this as a user adoption problem as well as a technical one. People may first use LLMs to write reviews, thank-you notes, memos, and presentations. He estimates that moving from those tasks to trusting a machine with restaurant bookings, insurance choices, or other actions involving money and time could take three to five years. He expects search adoption at meaningful scale to take five to seven years.
Feedback inside the workflow can improve an AI product
Turovsky agrees with Demetrios Brinkmann that user feedback can become an important source of product advantage. An application might suggest a text change, then learn from the user's adjustment. It could also ask the user to correct an underlined passage instead of supplying its own replacement. Grammarly is used as an example of a workflow where users naturally edit generated text. The important design choice is to collect those corrections as part of normal use. This gives the company information about how people want the output changed and can provide data for improving future models or product behavior.
Customer interactions are a broad early area for generative AI
Turovsky names entertainment as an early area of change because that industry adopts new technology quickly and is already dealing with deepfakes, generated work, and disputes with platforms. He expects a larger effect from customer interaction use cases across sales, marketing, customer service, legal services, transportation, utilities, healthcare, insurance, and financial services. Companies with large customer bases could connect internal knowledge bases to LLMs across email, documents, chat, and calls. A model could explain technical information in language that makes sense to a customer and respond to their emotional state. He says this could reduce costs and improve the customer experience, but only if the supporting systems are rebuilt properly.
LLM applications need new data and operational systems behind the interface
Turovsky rejects the idea that companies can simply add GPT-4 to an existing tool or replace support workers with a model. Teams will need data cleaning, data processing, retrieval, indexing, embeddings, and systems that decide which private and public information reaches the model. They will also need people to handle exceptions, especially refunds and other monetary decisions. He expects a mix of proprietary, open-source, and custom fine-tuned models rather than one dominant provider. MLOps and DevOps teams will need a deeper understanding of model strengths and limits, plus systems that let smaller models and conventional back-end tools handle retrieval and other tasks.
Machine learning engineers should learn both model limits and product use cases
Turovsky gives engineers two areas to develop. On the technical side, they should follow model comparisons, understand how performance is measured, learn the basics of embeddings and vectorized databases, and prepare internal data for model use. On the product side, they should ask questions about the user, the business problem, and how widely the proposed application will be used. Calling a model with an input and displaying its response may be a demo, but it is not automatically a product. Turovsky says good ideas and bad ideas can come from anyone, so engineers should challenge vague proposals and help clarify the use case.
"There is a big difference between toying and playing and doing something cool and launching it at scale to billions, hundreds of millions, billions of users."Barak Turovsky13:12
Who should watch
You are deciding whether an LLM belongs in a product and need a way to weigh fluency, accuracy, and the cost of mistakes.
Your team is moving from a demo to a production system and needs to plan for latency, retrieval, data quality, privacy, security, and human review.
You work in MLOps, DevOps, data engineering, or product management and want to understand how customer-facing LLM systems change the work behind the interface.
