# FLOps with Scaleout's Open-core Platform

Marco Capuccini, Scaleout Systems | MLOps Meetup | Episode 104 | 44:36
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=z8IHMr-Z1M8
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/flops-with-scaleouts-open-core-platform
Published: 2022-06-28
Tags: data-engineering, model-serving, open-source, privacy

## TL;DR
- Federated machine learning keeps training data at each site and sends model weights to a central aggregator.
- Federated deployments face problems with heterogeneous data, slow clients, weak network connections, and possible malicious updates.
- Scaleout Systems designed its open-source platform with combiners that aggregate updates near clients before sending smaller payloads to a reducer.

## Summary
Marco Capuccini explains why some machine learning systems should move computation to data instead of collecting all data in a central store. Federated learning lets local clients train on private data and send model updates to an aggregator. He covers applications in medical imaging, air traffic management, and self-driving vehicles, where regulation, competition, or network costs limit data sharing. The talk focuses on the engineering problems around federated learning. Data differs across sites, clients have different speeds, and some networks are unreliable. Scaleout Systems addresses the scaling problem with a map-reduce-inspired architecture. Combiners aggregate updates near groups of clients before sending them to a reducer, which avoids making one central reducer handle every client directly. Marco also describes the planned FLOps Studio managed service, including alliance management, monitoring, malicious-update detection, and authentication. He is clear that federated learning still needs an initial phase for data anonymization and standardization, and that it is unsuitable when even a small central sample cannot be shared.

## Key ideas
### Data gravity makes centralizing every dataset increasingly difficult
[06:31](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=391s)
Marco says data has expanded across industry and academia, causing applications to gravitate toward where the data already lives. He expects future enterprise infrastructure to have fewer servers at many locations rather than thousands of servers in one data center. Scaleout Systems focuses on decentralized machine learning because data may sit in different enterprises, regions, or edge environments. The company began in 2018 in Uppsala and grew from life-science work into collaborations involving automotive and airline use cases. Its federated approach starts from the location of the data rather than assuming every dataset can be moved to one central store.

### Federated learning avoids moving private or expensive-to-transfer data
[10:26](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=626s)
The usual setup collects data from a mesh of edge, cloud, and enterprise systems, moves it into a central store, trains a model there, and exposes the result through APIs. Marco challenges this pattern when organizations want to collaborate without sharing proprietary data, when regulations such as GDPR prevent sharing customer or patient data, or when edge data is too large and costly to transfer. Federated learning keeps data with local clients. The clients train locally, send model weights to a central aggregator, receive the latest global model, and repeat the process. This preserves input privacy, although the weights can still reveal information about the training distribution.

### Federated averaging depends on local data being comparable
[13:28](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=808s)
Marco describes federated averaging as a common industrial approach. A central aggregator initializes the weights, sends them to clients, and coordinates parallel local training rounds. Each client runs steps of stochastic gradient descent and returns a weight update. The aggregator averages those updates and sends the result back for the next round. In a medical imaging example for acute myeloid leukemia detection, the federated model initially shows some instability but eventually converges as well as the centralized global model. It also performs better than training on a single site. The result depends on what each site can observe. In his CIFAR-10 simulation, clients restricted to only a few classes produce much lower accuracy than clients with more homogeneous data.

### Federated systems have coordination problems that ordinary distributed training does not solve
[16:50](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=1010s)
Marco distinguishes federated learning from general distributed machine learning. The data is heterogeneous across sites, and system conditions also vary. Some clients may take longer to return updates. Waiting for every client leaves compute capacity idle, while asynchronous training can let the global model move ahead and make late updates harmful. Some sites also have weak connections, so network protocols designed for more uniform environments may perform poorly. These conditions make scalability, scheduling, and communication part of the learning system itself. The platform work at Scaleout Systems focuses heavily on this system-side difficulty rather than only on the training algorithm.

### Medical, airline, and vehicle use cases create different reasons to federate
[19:25](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=1165s)
In medical imaging, doctors must annotate data, and each organization may have only a small number of specialists. Multiple labs could improve a model together, but patient data cannot freely move across the internet. In air traffic management, airlines could collaborate on trajectory prediction, yet customer and competitive business data make sharing difficult. Self-driving vehicles create a different problem. A single organization may want to learn from a large fleet, but moving all raw sensor data is inefficient and cumbersome even when regulation is not the blocker. Across these examples, federated learning keeps computation close to the data while allowing model information to be combined.

### Scaleout's architecture uses combiners to reduce the central bottleneck
[23:23](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=1403s)
Scaleout's open-source platform has clients, a middle tier, and supporting services. Clients are the only components that access local data. The middle tier uses stateless load balancers and aggregators called combiners. Inspired by map-reduce, a combiner performs an initial aggregation close to a group of clients, then sends fewer and smaller updates to the reducer. This design is intended to support very large client populations without forcing one reducer to process every update directly. Combiners can be placed inside a high-bandwidth network, such as one organization's internal network, before updates cross a slower link. The deployment can also be simplified to one combiner and one reducer when all components share a fast network.

### The platform separates language-independent services from client code
[27:32](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=1652s)
The implementation uses Python 3.8 and 3.9 for its platform components, while users can write their client logic in another language through the platform's interfaces. Monitoring and service discovery use a REST API. Communication among clients, combiners, and reducers uses gRPC streams, which Marco says worked well over public networks. This language-agnostic design matters because future clients may run on constrained devices and could be implemented in Rust, C, or even assembly. The platform is available under an Apache 2.0 license, includes examples and helpers, has a user interface, and has a publicly available paper describing the underlying work.

### FLOps Studio adds management and safety controls around federated projects
[29:07](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=1747s)
Marco describes FLOps Studio as a planned managed platform built on the open-source federated learning system. It is intended to manage multiple federated alliances and projects through one API and user interface. Planned features include monitoring system and model performance across all three tiers, detecting malicious model updates, and scanning user code. A malicious client could send random weights to a combiner or reducer and damage the model. The service also includes authentication for its user interfaces and token-based authentication for clients and combiners. Marco says the federated part should fit into existing machine learning workflows instead of replacing every tool. He names MLflow, Kubeflow, and SageMaker as platforms it could interface with.

### Federated learning still needs a central research phase before automation
[33:31](https://www.youtube.com/watch?v=z8IHMr-Z1M8&t=2011s)
In response to Demetrios Brinkmann, Marco explains that the beginning of a federated workflow remains different from centralized machine learning. Data scientists need enough anonymized data to study the problem centrally, understand the data, and standardize it. This is especially difficult when microscopes or other equipment produce data with different characteristics. If one microscope changes the image hue, a model may fail to converge unless the difference is understood and addressed. Marco mentions methods such as CycleGAN to translate distributions into a common one. If an organization cannot share even a small anonymized sample, or if the problem cannot be standardized, federated learning may not be usable.

## Notable quotes
- Marco Capuccini: "Instead of having the data centrally and having a machine learning algorithm to look at it all at once, you can have several machine learning algorithms that can access data locally." (11:43)
- Marco Capuccini: "Federated machine learning is not distributed machine learning." (16:50)
- Marco Capuccini: "The idea is that we can aggregate the weights before we send it over the slow network to the reducer." (26:37)
- Marco Capuccini: "If you don't have any of the challenges that I have been talking about, of course don't bother." (37:51)

## Tools & references mentioned
- Scaleout Systems
- FLOps Studio
- Apache 2.0
- GDPR
- Federated Averaging
- Intel SGX
- MLflow
- Kubeflow
- SageMaker
- CycleGAN
- GitHub
- arXiv
- AstraZeneca

## Who should watch
- You are evaluating federated learning for patient, sensor, or customer data that cannot move into one central store.
- Your training system has many clients with different speeds, unreliable links, or large communication costs.
- You want to understand what platform work is needed around federated training before adopting a managed service.

## Related talks

- [Federated Learning: Machine Learning on the Edge](https://mlopstalks.com/talks/federated-learning-machine-learning-on-the-edge) (Varun Kumar Khare, Nimble Edge, 46:38)
- [ML Scalability Challenges](https://mlopstalks.com/talks/ml-scalability-challenges) (Waleed Kadous, Anyscale, 1:00:03)
- [MLOps + Machine Learning](https://mlopstalks.com/talks/mlops-machine-learning) (James Sutton, Algorithmia, 1:01:50)
- [Considerations and Optimizations for Deploying Open Source LLMs at Your Company](https://mlopstalks.com/talks/considerations-and-optimizations-for-deploying-open-source-llms-at-your-company) (Oscar Rovira, Mystic AI, 11:31)
- [Unleashing Sensitive Datasets with Distributed Data Science](https://mlopstalks.com/talks/unleashing-sensitive-datasets-with-distributed-data-science) (Blaise Thomson, Bitfount, 39:26)