# How Synthetic Data Will Transform Publishing Forever

Robert Caulk, Emergent Methods | Agents in Production 2025 | 23:51

Source: https://www.youtube.com/watch?v=9R84L6Jp7dE
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/how-synthetic-data-will-transform-publishing-forever
Published: 2025-08-13
Tags: context-engineering, synthetic-data

## TL;DR
- Publishers need a way to earn money when AI agents use their reporting without visiting pages that generate advertising revenue.
- Grounded synthetic news data removes original phrasing while preserving facts and adding entities, relationships, sentiment, locations, dates, and other context for AI systems.
- Ask News processes publisher data on premises, pays 50% of royalties to publishers, and supplies cleaned information to agents making high-stakes forecasts.

## Summary
Robert Caulk argues that publishers need a different business model as AI agents retrieve information directly instead of sending users to websites. Direct licensing with every AI developer is difficult, while raw HTML exposes publishers to scraping costs and gives agents poorly structured data. Ask News addresses this by turning news articles into grounded synthetic data. The process removes narrative voice and original phrasing, then adds structured context such as entities, statements, dates, locations, sentiment, classifications, translations, and knowledge-graph relationships. Caulk says this gives agents cleaner material for tasks such as geopolitical forecasting and fact-checking while protecting publishers' original expression. Ask News uses small models for high-throughput processing and does the publisher data processing on premises. The company says half of royalties go to publishers, weighted by how often their articles surface. Caulk is direct about the tradeoff: on-premises infrastructure reduces dependence on cloud providers but creates operational risks of its own.

## Key ideas
### AI agents break the advertising model that funded publishers
[00:56](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=56s)
Robert Caulk says publishers built their internet business around users visiting pages and seeing advertisements. That model does not work when ChatGPT or another agent visits a website on a user's behalf and returns an answer without creating an ad impression. Publishers also face low subscription sales, short-form summaries from search engines, and disputes over whether their reporting and original expression were used to train language models. Caulk argues that keeping journalists and publishers alive matters because agents need current, accurate information about the real world.

### Existing licensing and ad-based approaches leave gaps
[02:04](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=124s)
Caulk describes direct licensing as difficult to scale because a publisher would need agreements with many AI developers and agents. He says Cloudflare's AI content market plans have potential, while ad-filled AI chatbots may work for consumer use. Those approaches do not solve the needs of agents used for geopolitical risk analysis, forecasting, or other business tasks. They also fail to preserve control over a publisher's original text and expression. Smaller publishers face the greatest difficulty because their information often remains as HTML on a server with no simple route into an AI system.

### Synthetic news data preserves information while removing original expression
[06:08](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=368s)
Ask News takes an original article and removes its narrative voice, journalistic style, and original phrasing. It keeps the important information and enriches it with sentiment, classifications, geographic coordinates, entity relationships, and other context. Caulk says agents need this structured information more than they need a publisher's exact wording. The approach gives publishers a way to monetize the underlying information without handing over the expression they paid to create. It also gives developers a cleaner feed than repeatedly scraping and cleaning web pages.

### Context engineering turns news into structured material for language models
[06:49](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=409s)
Caulk defines context engineering as building an ideal synthetic representation of news for an LLM. He starts with the traditional who, what, why, where, and when, then adds people, organizations, events, statements, attributions, publication dates, motivations, and locations. A date can change the meaning of a fact for forecasting, such as whether Joe Biden withdrew from the race 24 hours before an election or three months before it. Caulk also wants source origins, geocoordinates, and knowledge graphs, producing token-optimized information instead of raw HTML that can create hallucinations.

### The pipeline needs editorial judgment and specialized models
[09:26](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=566s)
The system includes entity extraction, sentiment analysis, topic classification, storage choices, translation, and the extraction of statements, evidence, and attribution. Caulk says Ask News relies on an editor-in-chief with 45 years in journalism to preserve journalistic integrity in each synthetic data point. Reporting voice helps an LLM assess whether a source is provocative, subjective, or objective. Ask News trained a small entity extraction model called GLiNER for high throughput and worked to reduce the Western-name bias found in some basic extractors. Caulk says extracting names accurately across regions matters for geopolitical forecasting.

### Knowledge graphs and bias analysis add context beyond article text
[11:50](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=710s)
Caulk shows how relationships connect people, countries, industries, and other entities across sources. For a forecast about tariffs, an agent needs those connections rather than one isolated article. Ask News also researched model bias, including an analysis of DeepSeek run locally rather than through its chat interface. Caulk says that version was not biased in the way people had claimed, although he described it as leaning toward the Western side. Ask News released a model called BiasExpert to identify bias and recommend ways to improve the data.

### Ask News ties publisher payments to how often their information is used
[16:46](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=1006s)
Caulk describes a bridge in which publishers sell high-value, human-created information while developers buy clean information at the right time and in a usable form. Developers avoid maintaining fragile HTML scrapers, and publishers retain control of their original expression. He says Ask News has more than 700 publishers, including Agence France-Presse. Half of royalties go to publishers and are weighted by how often an article from each publisher surfaces. The remaining half supports overhead and research into areas such as bias and entity extraction.

### On-premises processing protects source text but creates infrastructure tradeoffs
[21:18](https://www.youtube.com/watch?v=9R84L6Jp7dE&t=1278s)
Ask News processes articles around the clock with multiple smaller LLMs on each item. Smaller models reduce operating costs and allow more articles to move through the pipeline. Publisher agreements require processing on premises, so the company does not send source data to OpenAI or Anthropic. Caulk says some front-end components used DigitalOcean, although the company later moved much of its infrastructure back on premises. He is candid about the downside: if the data center catches fire, the company has a serious problem.

## Notable quotes
- "The publisher wants to monetize the information. The AI agent just needs the accurate information." (03:45)
- "Take an original news article, remove the narrative voice and journalistic style and original phrasing, but preserve the most important information and enrich the most important information." (06:08)
- "We want token optimized extra content, LM ready, objective information." (08:47)
- "The beauty is that the AI developer doesn't even want the original expression in these high stakes decision-making systems. They just want objective accuracy." (16:46)
- "Part of our agreement with publishers is that it all has to be done on premise." (21:58)

## Tools & references mentioned
- Emergent Methods
- Ask News
- MLOps Community
- Databricks
- ChatGPT
- The New York Times
- OpenAI
- News Corp
- Perplexity
- Cloudflare
- Prada
- Gemini
- DeepSeek
- GLiNER
- spaCy
- Qdrant
- BiasExpert
- Metaculus
- Agence France-Presse
- Columbia University
- George Mason University
- Texas
- AMS.ext
- Riley Risk
- Prepper.ai
- DigitalOcean
- Anthropic

## Who should watch
- You are building an agent that needs current news for forecasting, risk analysis, fact-checking, or other high-stakes work, and raw web scraping is becoming a liability.
- You work for a publisher and need a way to license information to AI systems without giving away the publication's original wording and voice.
- Your data pipeline needs to process many articles with several models while keeping source material away from external model providers.

## Related talks

- [Navigating the AI Frontier: The Power of Synthetic Data and Agent Evaluations in LLM Development](https://mlopstalks.com/talks/navigating-the-ai-frontier-the-power-of-synthetic-data-and-agent-evaluations-in) (Boris Selitser, Okareo, 57:22)
- [How AI Will Change Gaming Forever](https://mlopstalks.com/talks/how-ai-will-change-gaming-forever) (Pietro Gagliano, Transitional Forms, 26:25)
- [Innovative Gen AI Applications: Beyond Text](https://mlopstalks.com/talks/innovative-gen-ai-applications-beyond-text) (Diana C. Montañes Mondragon & Nick Schenone, QuantumBlack, 54:45)
- [How AI is Revolutionizing Marketing Forever](https://mlopstalks.com/talks/how-ai-is-revolutionizing-marketing-forever) (Fausto Albers, AI Builders Club, 49:41)
- [A New Way of Building with AI](https://mlopstalks.com/talks/a-new-way-of-building-with-ai) (Jiquan Ngiam, Lutra AI, 1:04:46)