# Law of Diminishing Returns for Running AI Proof-of-Concepts

Oguzhan Gencoglu, Top Data Science | MLOps Meetup | Episode 62 | 56:19
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=j09xbtudJgs
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/law-of-diminishing-returns-for-running-ai-proof-of-concepts
Published: 2021-05-10
Tags: cost, deployment, governance, latency

## TL;DR
- A successful AI proof-of-concept can still fail when it reaches production because its success measures ignore operational constraints.
- Production requirements such as latency, hardware, cost, privacy, maintenance, fairness, and regulation need to shape the PoC from the start.
- Companies reduce repeated PoCs by building a broad backlog of business problems and choosing cases where machine learning can create real value.

## Summary
Oguzhan Gencoglu argues that companies often get trapped in repeated AI proof-of-concepts. A PoC that fails because the model is inaccurate has still done its job. A PoC that works in a presentation but never reaches production exposes a deeper problem. Teams may lack executive commitment, or they may judge the model with metrics that do not match the production setting. A real system may need to run in real time, on limited hardware, without reliable internet, while meeting budget, privacy, fairness, and regulatory requirements. Oguzhan recommends discussing those constraints before writing code and involving problem owners, data scientists, and engineering teams early. He also advises companies to create a backlog of possible machine learning problems instead of repeatedly choosing the most obvious one. The talk treats problem translation as a learned skill, built through repeated work with real-world data and business processes.

## Key ideas
### Machine learning projects usually begin with repetitive work that people want to automate
[02:18](https://www.youtube.com/watch?v=j09xbtudJgs&t=138s)
Oguzhan says that client expectations changed over five years, but one pattern kept returning. Companies wanted to automate or semi-automate tasks that were extremely repetitive. Automating a repetitive task creates scalability because the work no longer depends on people repeatedly performing the same steps. The boredom of the work also matters. Examples include checking certain fields, looking up text, or watching camera feeds. These tasks often enter a company backlog first because they are tedious. Cost savings then become another filter. Oguzhan describes these repetitive tasks as common low-hanging fruit where machine learning projects can produce some result.

### The industry has moved from exploratory PoCs toward production-level work
[05:27](https://www.youtube.com/watch?v=j09xbtudJgs&t=327s)
Earlier in his career, Oguzhan saw companies use small proof-of-concepts to understand how machine learning might work for them. Production was often not planned at that stage. Teams started with a model, learned a few things, and only later considered what might happen next. Over time, companies began asking teams to productize systems or build something closer to a product from the beginning. Proof-of-concepts remain common, but the expectation is changing. Oguzhan connects this shift with a better understanding of how companies get value from machine learning. A model that stays inside an experiment does not provide the same value as one used in the business.

### A successful PoC can fail because the organization never committed to production
[10:50](https://www.youtube.com/watch?v=j09xbtudJgs&t=650s)
Oguzhan distinguishes between a PoC that fails and one that succeeds but never reaches production. Failure can be useful because it answers the question the experiment was designed to test. The more interesting failure happens when a working model receives praise in a presentation and then stops. One reason is that the project began with enthusiastic mid-level managers rather than an executive or company-level strategy. Those people may understand the opportunity, yet lack the authority, budget, IT support, or talent needed to continue. Oguzhan says that nearly all projects his team put into production had a company-level, executable strategy behind them. Commitment from the top moved into the work.

### PoC metrics need to match the conditions of the final system
[12:48](https://www.youtube.com/watch?v=j09xbtudJgs&t=768s)
A high accuracy score and a successful data science experiment do not prove that a system is ready for production. Oguzhan gives the example of computer vision for defects on a production line. The model may work on annotated data, then face a very different problem when it must run in real time, perhaps in a remote location without reliable internet or a GPU. The team may need to quantize, prune, or compress the model and optimize it for specific hardware. These requirements affect the model and the project plan before deployment. He says teams should introduce production success metrics into the PoC instead of treating the PoC, pilot, and product as unrelated stages.

### Production constraints change the way teams should build and communicate
[16:57](https://www.youtube.com/watch?v=j09xbtudJgs&t=1017s)
Thinking about deployment from the beginning changes the technical choices. Oguzhan lists modest hardware, near-real-time operation, false-negative rates, users, and cloud cost as factors that should be in the team's mind. A state-of-the-art large model may be a poor choice if its production cost is too high. He also recommends building something that works quickly and showing it to stakeholders. In consultancy work, clients may mistake data inspection and attractive plots for progress, then become anxious when model development produces little visible output. A quick working version, including a mock-up of production, gives stakeholders a clearer view of the system and reduces uncertainty during the project.

### Teams should agree on the production vision before writing code
[20:41](https://www.youtube.com/watch?v=j09xbtudJgs&t=1241s)
Oguzhan does not use one decision tree for every PoC. His first step is to make the problem owner think about the intended production system before the team writes a line of code. That conversation covers the available budget, maintenance and support, speed, fairness, regulation, personal-data privacy, and whether the model could leak data. The answers give the team a fuller picture of the problem. They also prevent a project from being judged only by an accuracy target chosen at the start. Oguzhan's point is that expectation management belongs before the experiment, because many expensive surprises appear only after a model has already been declared successful.

### A company needs a backlog of machine learning problems before starting more PoCs
[28:21](https://www.youtube.com/watch?v=j09xbtudJgs&t=1701s)
Oguzhan says organizations can enter PoC hell because they start with whichever problem happens to be visible instead of deciding which problems are good matches for machine learning. He recommends creating a backlog by asking people across the organization what processes or business problems might be improved. This can produce many ideas, including opportunities that are not immediately obvious. The first five ideas people mention are not necessarily the best choices. Teams should step back, examine the business value, and compare possible projects before committing. Oguzhan also says that stakeholders may not know what machine learning can do, so useful conversations require examples that make successful applications concrete.

### Translating a business problem into a machine learning problem is learned through practice
[44:38](https://www.youtube.com/watch?v=j09xbtudJgs&t=2678s)
Oguzhan calls translation the most important part of the modern machine learning scientist or engineer role. It means turning a real-world process into a precise machine learning problem, while also deciding whether machine learning is needed at all. Formal courses often provide a prepared dataset, so learners do not practice defining the problem or collecting the data. Oguzhan looks for candidates who start with a real problem, collect their own data, build a model, and put it somewhere usable. He gives the example of a colleague who collected data to predict where to pick blueberries in Helsinki. That end-to-end experience gave the person empathy for clients and an understanding of the difficulty hidden inside data collection and problem definition.

## Notable quotes
- Oguzhan Gencoglu: "When you basically solve a repetitive task in an automatic way, that's scalability basically." (03:16)
- Oguzhan Gencoglu: "A proof of concept being successful and not making it into production, that's very interesting to me." (09:36)
- Oguzhan Gencoglu: "If you don't plan during the proof of concept it will never make it to production." (14:53)
- Oguzhan Gencoglu: "The most crucial part is translating a real life problem into a machine learning problem." (44:40)
- Oguzhan Gencoglu: "You can't learn it in a textbook right." (45:38)

## Tools & references mentioned
- Top Data Science
- LinkedIn
- Slack

## Who should watch
- You are running PoCs that produce impressive demos but lack a clear route to deployment.
- Your data science and engineering teams receive models without agreeing on hardware, latency, cost, maintenance, or support.
- Your organization keeps choosing obvious automation ideas and needs a better way to find machine learning problems with business value.

## Related talks

- [Managing Data for Effective GenAI Application](https://mlopstalks.com/talks/managing-data-for-effective-genai-application) (Anu Arora & Anass Bensrhir, QuantumBlack AI by McKinsey, 51:01)
- [Productionizing AI: How to Think From the End](https://mlopstalks.com/talks/productionizing-ai-how-to-think-from-the-end) (Annie Condon, 11:11)
- [Building Effective Products with GenAI](https://mlopstalks.com/talks/building-effective-products-with-genai) (Faizaan Charania, LinkedIn, 53:14)
- [AI Operations Without Fundamental Engineering Discipline](https://mlopstalks.com/talks/ai-operations-without-fundamental-engineering-discipline) (Nikhil Suresh, Hermit Tech, 49:28)
- [Building Production Copilots](https://mlopstalks.com/talks/building-production-copilots) (Tristan Zajonc, Continual, 19:44)