# Federated Learning: Machine Learning on the Edge

Varun Kumar Khare, Nimble Edge | MLOps Reading Group | Episode 3 | 46:38

Source: https://www.youtube.com/watch?v=IWxBKGPHOBQ
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/federated-learning-machine-learning-on-the-edge
Published: 2022-01-18
Tags: deployment, edge, privacy, security

## TL;DR
- Federated learning trains models across phones, servers, and IoT devices while keeping raw training data with the users or organisations that hold it.
- Communication is usually the main cost, so edge deployments send compressed model updates and run more local computation on devices.
- Production systems must handle unreliable devices, biased client selection, malicious updates, and the lack of large-scale simulation tools.

## Summary
The reading group discusses the paper "Federated Learning: Strategies for Improving Communication Efficiency" and the earlier introduction of federated learning by Brendan McMahan and others. Varun Kumar Khare explains how clients download a shared model, train it on local data, and return updates for aggregation. The approach separates data collection from model training, which can reduce raw-data exposure and support personalised models. He distinguishes cross-silo, vertical, cross-device, and decentralised federated learning, then describes the practical limits of edge devices, including poor connectivity, uneven compute capacity, and non-identical data. The talk is honest about the hard parts of production use. Client selection can favour better-connected and more expensive devices, malicious clients can submit harmful updates, and existing libraries have struggled to simulate deployments at realistic scale. The discussion also covers debugging, data drift, hierarchical aggregation, and examples of federated learning in products such as Gboard and Google Assistant.

## Key ideas
### Federated learning separates model training from central data collection
[01:36](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=96s)
Varun Kumar Khare describes federated learning as machine learning across multiple entities, such as smartphones, servers, and IoT devices. A central entity sends out a model, clients train it on their own data, and the clients return updates. The raw data stays on the device or inside the organisation that owns it. This changes the usual cloud pattern, where teams copy data into a central store before training. The approach still allows a shared model to learn from many sources, even when the sources have different data distributions.

### A federated learning cycle moves models, local updates, and aggregate models
[03:54](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=234s)
The normal cycle begins with a host selecting a model and choosing clients. Selected clients download the model, train locally, and report an update. Because full models can be large, the system can send parameter differences instead, then compress them by retaining only important gradient coordinates. The cloud aggregates the incoming updates into a new global model, which starts another cycle. Varun says local training and cloud aggregation are relatively cheap compared with moving model data between devices and the cloud.

### Client selection is necessary because every device cannot train at once
[05:53](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=353s)
A recommendation system may have millions of eligible devices, but asking all of them to participate in every round would make the effective batch size too large and remove the stochastic behaviour useful for optimisation. In his example, a system selects a smaller group based on availability, participation frequency, training time, and network bandwidth. Devices then download the model, run local iterations, and send back updates. In production, the same device may participate only occasionally, while different groups of devices take part in successive rounds.

### Federated learning has several deployment shapes
[11:03](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=663s)
Varun distinguishes several ways to distribute data and coordination. In cross-silo learning, different servers hold data from different users. In vertical federated learning, the same users appear across servers, but different features are held by each server. Decentralised federated learning removes the central aggregator and lets participants exchange models with neighbours, similar to a gossip protocol. Cross-device learning applies the idea to phones and other edge devices, where connectivity is unreliable and compute, memory, and battery capacity vary widely.

### The approach can support personalisation while reducing raw-data exposure
[17:04](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=1024s)
Varun connects federated learning with hyper-personalisation. Each device can maintain a model tuned to its user's behaviour, while meta-learning methods treat each user or organisation as a separate task. He also argues that keeping raw data at the source reduces the amount of personally identifiable data held by the central service. Differential privacy and secure aggregation can add further protection. The original data remains with the user, and the central system receives model information for the intended application rather than a general-purpose copy of the user's data.

### Edge federated learning can reduce cloud compute costs, but its savings depend on the setup
[20:39](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=1239s)
In Varun's description, the cloud mainly performs aggregation while the more intensive training runs on user devices. He says this can make large recommendation systems much cheaper than running all training in the cloud. The same arrangement also lets a service use device-side computation without taking possession of the underlying personal data. The trade-off is that the system must deal with devices that disconnect, run slowly, have weak networks, or produce updates at different times.

### Production systems must control bias and malicious client updates
[22:37](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=1357s)
A round may select many devices but receive updates from only the faster and better-connected subset. Varun warns that this can bias the resulting model towards users with more expensive phones and stronger internet connections. A separate security problem occurs when malicious clients submit arbitrary updates to damage the aggregate model. He describes device-selection methods, threat modelling, and aggregation controls as ways to manage these risks, although the problems remain part of the engineering work rather than disappearing through federation.

### Federated learning needs realistic simulation and operational tooling
[25:50](https://www.youtube.com/watch?v=IWxBKGPHOBQ&t=1550s)
Research demonstrations often use tens or hundreds of workers, while a production recommendation service may need to reason about millions of devices. A useful simulator would model network delays, device constraints, participation rates, and other failures at much larger scale. Varun says available libraries at the time did not fully support this level of edge simulation. He describes Nimble Edge's work on simulators, new algorithms, deployment tooling, and privacy schemes so developers can connect federated learning to existing applications and cloud pipelines.

## Notable quotes
- "One of the biggest tenets of federated learning is that it decouples data collection and model training." (01:36)
- "What is costly is the data movement that happens, the movement of the models that is being done across the two devices, across the cloud." (09:26)
- "The only thing that you're doing on the cloud in this entire thing is aggregation." (21:19)
- "You have to deal with these kinds of model biases while you are doing the device selection and while you are doing the aggregation." (23:41)
- "The technology has been proven at scale, but the adoption needs to make it easier for everyone to adopt production pipeline." (40:27)

## Tools & references mentioned
- Federated Learning: Strategies for Improving Communication Efficiency
- Brendan McMahan
- Google
- GDPR
- CCPA
- OpenMined
- Nimble Edge
- Flower
- TensorFlow Federated
- Clara SDK
- NVIDIA
- FedML
- FetchSGD
- Johns Hopkins University
- Gboard
- Siri
- Google Assistant
- Samsung
- differential privacy
- secure aggregation

## Who should watch
- You are deciding whether a recommendation or personalisation workload should keep training data on user devices rather than copy it into the cloud.
- Your team needs to understand why federated learning experiments with a few hundred workers do not automatically translate to deployments across millions of unreliable devices.
- You are designing client selection, aggregation, debugging, or security controls for a cross-silo or cross-device system.

## Related talks

- [FLOps with Scaleout's Open-core Platform](https://mlopstalks.com/talks/flops-with-scaleouts-open-core-platform) (Marco Capuccini, Scaleout Systems, 44:36)
- [The revolution of Federated Learning](https://mlopstalks.com/talks/the-revolution-of-federated-learning) (Fabiana Clemente, MLOps Community & Ramen Dutta, TensoAI, 29:22)
- [Unleashing Sensitive Datasets with Distributed Data Science](https://mlopstalks.com/talks/unleashing-sensitive-datasets-with-distributed-data-science) (Blaise Thomson, Bitfount, 39:26)
- [MLOps + Machine Learning](https://mlopstalks.com/talks/mlops-machine-learning) (James Sutton, Algorithmia, 1:01:50)
- [Re-Platforming Your Tech Stack](https://mlopstalks.com/talks/re-platforming-your-tech-stack) (Michelle Marie Conway & Andrew Baker, Lloyds Banking Group, 51:15)