# Monzo Bank - An MLOps Case Study

Neal Lathia, Monzo Bank | MLOps Coffee Sessions | Episode 20 | 1:03:54
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=EyLGKmPAZLY
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/monzo-bank-an-mlops-case-study
Published: 2020-12-07
Tags: deployment, governance, model-serving, platform-teams

## TL;DR
- Neal Lathia says Monzo starts with rule engines, then adds machine learning when it can make a large difference.
- Monzo's small machine learning team builds the tooling and production services needed to ship its own models instead of throwing models over the wall.
- Neal advises teams to get a machine learning system into production before trying to solve every MLOps problem around it.

## Summary
Neal Lathia describes how Monzo uses machine learning across customer service, fraud detection, credit scoring, and features in its banking app. The team starts with a rule engine where possible. This gives the business a useful baseline and creates logs that can later become training data. Monzo's machine learning team has three people inside a wider data science organisation, so it builds reusable templates and services instead of handling every deployment by hand. Models are trained in Google Cloud, stored in a model store, and served through small Python microservices running alongside Monzo's AWS and Kubernetes infrastructure. New models can run in shadow mode before their predictions affect customers. Neal also explains why different business areas need different levels of oversight. A text classifier for agent recommendations is low risk, while credit scoring requires a much stricter validation process. He is direct about the trade-off between buying MLOps products and building internal tools, and recommends shipping something useful before becoming paralysed by tooling requirements.

## Key ideas
### A bank contains many separate machine learning problems
[05:12](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=312s)
Neal says a bank is effectively around ten different companies under the hood. Its areas include customer service, loans and overdrafts, credit scoring, financial crime detection, and features that improve the app. Monzo has focused heavily on customer service because customers contact agents through chat, creating a large collection of conversations and agent responses. The team uses this text to help agents respond and to direct customers towards useful information. During the coronavirus lockdown, Monzo also spent substantial effort improving controls for payment fraud, including scams where callers claimed that customers should transfer money to a safe account.

### Human decisions create a path towards useful machine learning
[08:33](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=513s)
Neal looks for situations where a person can make a decision in one or two seconds and the decision is recorded. That pattern can provide both an automation opportunity and a source of labelled data. In customer service, agents choose from roughly a thousand pre-filled responses after reading a customer's message. Monzo built a system that recommends relevant shortcuts to the agent. The agent still personalises and sends the response, while the customer's message and the selected shortcut provide training data for later improvements.

### Rule engines are Monzo's starting point for most problems
[11:52](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=712s)
Because Monzo has a small machine learning team and many more software engineers, Neal's first recommendation is usually to build a rule engine. A rule engine may solve the problem well enough, and it does not require a machine learning specialist. It also forces a team to record outcomes, which creates the data needed for a future model. One example routes a customer to an account management specialist when the customer starts a chat from an article about changing their legal name. Machine learning is added only when it can make a substantial difference.

### The team avoids one person owning one project
[22:02](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=1322s)
Monzo has around 40 people across data science, with separate areas for product analytics, banking domain work, data engineering, and machine learning. Neal says the machine learning team had three people at the time of the conversation. He avoids a one-to-one mapping between people and projects because a project can stall when its owner leaves or goes on holiday. For a group of text classifiers, the team split one large multi-class problem into around 20 binary models. Each person could own a related model while sharing work on text cleaning, datasets, and model pipelines.

### Reusable tooling turns repeated deployment work into a single command
[25:01](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=1501s)
Monzo looks for work that the team repeats and replaces it with a tool. Instead of manually putting model artifacts into S3 and managing credentials each time, the team built a model store service and a command-line tool. A scientist uploads a model and supplies information about its purpose. The tool handles storage and makes the model available to other services. It also records the upload so the team can display model information on a dashboard, including the latest model uploaded for each domain.

### Monzo keeps model services narrow and understandable to production engineers
[40:51](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=2451s)
Neal describes a workflow that starts with notebooks for exploring data and testing ideas. The team then writes Python pipelines that create datasets or train models in the Google AI Platform using Docker containers. Cookie-cutter templates provide the Docker and pipeline boilerplate. Once a model is selected for production, a command sends it to the production model store, and another template creates a Python microservice. The machine learning scientist mainly fills in the predict function. The service does prediction and little else, so Monzo's Go engineers can call it without needing to understand the Python implementation.

### Shadow mode separates production testing from customer impact
[45:45](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=2745s)
A new or retrained model can run in production while its predictions are ignored. Neal calls this shadow mode. The predict function receives real calls and logs its outputs, but the system does not act on them. Those logs flow into Monzo's analytics stack, where SQL and Looker can be used to compare behaviour and performance. This gives the team a way to check whether production behaviour matches expectations before allowing the model to affect customers.

### Risk determines how much validation a model needs
[47:07](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=2827s)
Neal distinguishes between models that recommend saved responses to internal customer service agents and models that make decisions about credit or financial crime. The first category is lower risk because a human agent makes the final decision. Credit scoring has a much stricter process, partly because feedback on the outcome may take 12 months. The process includes replication, validation, reporting, several lines of defence, and monitoring. Neal says the goal is less about rerunning code exactly and more about showing that the team followed a controlled process and can detect problems.

### Teams should ship a useful model before solving every MLOps problem
[1:00:48](https://www.youtube.com/watch?v=EyLGKmPAZLY&t=3648s)
Neal's closing advice is to avoid analysis paralysis caused by MLOps requirements. He sees companies telling teams that they need feature monitoring, feature stores, or other capabilities before they can be doing machine learning correctly. His experience at Monzo started with a complicated neural network that was difficult to deploy. The team shipped it first, then improved the process by building better services and tooling. He recommends getting something over the line, then making it more reproducible, observable, and easier to operate.

## Notable quotes
- Neal Lathia: "In my mind at anytime you have a problem where like a human can kind of make a decision in like one or two seconds about it and in doing so they're generating data about the decision that they made, there's a good opportunity for machine learning." (09:23)
- Neal Lathia: "I don't want us to be training models and then like chucking them over a wall and at some other team to like put into production." (17:51)
- Neal Lathia: "The main reason behind that is that Python is not the like primary like mother tongue of our production infrastructure." (43:50)
- Neal Lathia: "Don't let that stop you from actually just shipping some machine learning." (1:00:48)
- Neal Lathia: "So don't fall into like analysis paralysis on the MLOps tooling." (1:01:28)

## Tools & references mentioned
- Monzo Bank
- TransferWise
- N26
- Revolut
- Spotify
- AWS
- Google Cloud
- Kubernetes
- Cassandra
- PyTorch
- Flask
- BigQuery
- dbt
- Airflow
- Looker
- BERT
- Transformers library
- Google AI Platform
- S3
- Demetrios Brinkmann
- Satish

## Who should watch
- You run a small machine learning team and need a practical way to move models into production without handing deployment to another group.
- Your organisation is deciding whether to build MLOps tooling or adopt external products, and you want to hear how Monzo weighs productivity against maintenance.
- You work with machine learning in a regulated setting and need to separate low-risk recommendations from models that require formal validation.

## Editor's note

Neal Lathia describes a small machine learning team that had to build reusable tooling and services rather than handle every model deployment by hand. ZenML lets teams define workflows as Python pipelines whose runs record steps, inputs, outputs, and code versions. The same pipeline can run on different infrastructure through configuration, so deployment work does not have to be rebuilt for each environment.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [MLOps vs LLMOps](https://mlopstalks.com/talks/mlops-vs-llmops) (Richa Sachdev, JPMorgan Chase & Willem Pienaar, Feast & Chris Van Pelt, Weights & Biases & Aparna Dhinakaran, Arize AI & Alex Ratner, Snorkel, 34:57)
- [Operationalize Machine Learning at Scale with MLOps](https://mlopstalks.com/talks/operationalize-machine-learning-at-scale-with-mlops) (Christopher Bergh, DataKitchen, 57:50)
- [Practical MLOps Part 2](https://mlopstalks.com/talks/practical-mlops-part-2) (Alfredo Deza, Author and Speaker, 1:01:38)
- [MLOps at Stripe](https://mlopstalks.com/talks/mlops-at-stripe) (Emmanuel Ameisen, Stripe, 44:36)
- [Doing MLOps](https://mlopstalks.com/talks/doing-mlops) (Noah Gift, Pragmatic AI Labs, 1:01:22)