Podcast

Micro Graph Transformer Powering Small Language Models

Jon Cooke, DataceptionEpisode 208 · 57:11 · Jan 2024 · 351 viewsHosted by Demetrios Brinkmann
Thumbnail for Micro Graph Transformer Powering Small Language Models Watch on YouTube
TL;DR
  1. 1

    Data products should start with a business decision and its expected value, then bring in the data and technology needed to support it.

  2. 2

    Teams can test data in a sandbox before fully onboarding, curating, and modeling it, which exposes poor quality or insufficient data earlier.

  3. 3

    A specialist language model can interpret business requests while separate analytics systems calculate metrics and forecasts, reducing the risk of hallucinated numbers.

Summary

Jon Cooke argues that data and analytics projects take too long because organizations start with platforms, schemas, and data requests instead of the business decision they need to support. He describes Dataception's data product process, which frames the business problem, identifies the value, examines source data in a sandbox, prototypes an analytic product, and then operationalizes it. Cooke favors small, multidisciplinary teams with business, product, data, and technical skills. He also describes work on a specialist language-model system that could let business users create metrics, forecasts, and other analytics through natural language. The language model would route requests to existing infrastructure rather than calculate numbers itself. Cooke is direct about adoption: a technically correct data product has little value if nobody uses it. Engineers need enough business understanding to question requests, clarify the desired outcome, and build trust with stakeholders.

Key ideas
10:44

Data work should begin with the decision the business needs to make

Cooke says he starts with business leaders by asking what decision they need to make and what outcome would create value. He usually avoids beginning with technology or even data. A request such as a dashboard full of KPIs may hide a smaller need, such as identifying the two or three things that would improve someone's day or support a decision. He describes this as closing the gap between a business conversation and something people can actually use. Dataception combines business knowledge, product management, technology, and data to shorten that path.

15:04

Sandboxing source data prevents teams from spending months on the wrong foundation

Cooke recommends examining data in a sandbox before fully onboarding, curating, or modeling it in a warehouse or platform. Teams may not yet know whether the data is useful, whether its quality is sufficient, or whether it contains enough observations. He gives the example of an operational-risk project with about one outage per year, which could not support machine learning. Looking at the data earlier would have exposed that limitation before the team waited for the formal warehouse process.

20:41

A data product links the data, analytics, interface, and business use case

Cooke describes Dataception's data product pyramid as an agile-style process for analytics. The work includes a product canvas, an ideation session, data discovery, value prioritization, wireframes, and analytics development with the data team. He wants data and analytics infrastructure to work as an ecosystem rather than forcing every use case through one engine. A product might use Python and machine learning, SQL for a metric, or another analytic tool, while still being delivered as one business-facing product.

29:39

Small multidisciplinary teams can replace the order-taking model

Cooke proposes small teams organized around a particular business-facing product. A team might include a business product manager, a data product specialist who combines business, data, and technical skills, an engineer, and a technical or data architect for more complex work. A user-experience person may also help put the result in front of the business. Platform teams can build shared infrastructure, while these product teams own the analytics and pipelines needed for specific use cases.

33:03

Specialist language models can turn natural-language requests into analytics products

Cooke explains that Dataception is developing technology for creating data products through natural language. A user could request a metric, choose the relevant dataset and presentation, or ask for a forecast, with the system creating and deploying the product. The approach uses a deconstructed encoder-decoder Transformer and a smaller specialist model rather than a very large general model. Cooke says the aim is to support design, prototyping, pipeline onboarding, and deployment with much less manual interaction.

36:46

The language model should call analytic infrastructure instead of inventing the numbers

When Demetrios Brinkmann asks how the system will keep results reliable, Cooke says the language model will not generate the numeric answers itself. It will interact with Dataception's infrastructure, using a RAG-like approach and separate models or components to produce the results. This avoids asking a probabilistic language model to perform calculations it was not designed to do. The model interprets the request, while the appropriate metric, regression, or forecasting system does the computation.

40:03

Prescriptive routing is faster when the desired end state is already known

Cooke agrees that the architecture is related to agents because one component can understand a request and ask other components to perform work. He distinguishes this from an agent that must discover its own plan. Dataception works with business teams to define the desired outcome in advance, so the system can follow a more prescribed path. That removes the need for a reward mechanism and much of the guesswork associated with agent planning.

48:00

Adoption must be established before a data product is built

Cooke says analytics is a business-change process, and adoption is a practical test of whether an organization actually wants to become data-driven. Teams can build the right technology and still fail because the business does not use it. He recommends testing stakeholder appetite by showing how a proposed change could affect a business result and asking whether that result matters. His preferred cycle is to observe, orient, decide, act, and repeat by measuring outcomes and speaking with users.

"If you build them and you don't know that they've been adopted, in my mind this is going to be a challenge."Jon Cooke50:36
Who should watch
  • Data engineers who receive vague requests for dashboards, metrics, or forecasts and need a way to clarify the business outcome before building.
  • Engineering leaders deciding whether data and machine learning platforms should be organized around shared infrastructure or business-facing product teams.
  • Teams exploring natural-language analytics who need to separate language-model interpretation from numeric calculation and validation.