# Private data, Data Science friendly

Jean-François Rajotte, University of British Columbia & Sumit Mukherjee, Microsoft AI for Good Research Lab | MLOps Community | Episode 9 | 41:39

Source: https://www.youtube.com/watch?v=dr9qvHc7YmU
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/private-data-data-science-friendly
Published: 2021-01-21
Tags: healthcare, privacy, synthetic-data

## TL;DR
- Synthetic data creates machine-generated samples with the same kinds of features as real data, without mapping each generated sample to one original record.
- PrivGAN partitions training data and uses a shared adversary to reduce overfitting while preserving more utility than some differentially private GAN approaches.
- Federated synthetic data sharing can help hospitals with limited or biased data, although healthcare use remains at the proof-of-concept stage and needs input from clinicians, regulators, lawyers, and data scientists.

## Summary
Fabiana, Jean-François Rajotte, and Sumit Mukherjee discuss synthetic data for healthcare and other settings where real data cannot be shared. Rajotte describes the practical barriers around protected health information, including approvals, agreements, and restricted on-premise access. Mukherjee explains that synthetic data is generated rather than collected and describes PrivGAN, which partitions data and trains generators with a centralized adversary. The method aims to reduce the influence of individual samples without the large utility loss associated with some differentially private GAN methods. The guests also cover membership inference attacks, mode collapse, privacy risks for outliers, and the different trade-offs between federated learning and synthetic data. A federated approach may help combine data from hospitals without moving the original records, while synthetic data can be reused for different tasks. They are careful about the limits: healthcare data is messy, current work is still based partly on toy data, and adoption depends on proof-of-concept studies and regulatory understanding.

## Key ideas
### Healthcare data access is blocked by practical and privacy constraints
[02:01](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=121s)
Jean-François Rajotte describes a need for data exchange in healthcare, followed by approvals, non-disclosure agreements, and restricted devices connected to on-premise systems. These controls can make collaboration difficult or impossible. Synthetic data offers a way to share information more easily, but it does not remove privacy concerns. A generator can still leak information or produce copies that resemble training data. Sumit Mukherjee describes a similar problem with medical images, which are often protected health information and may be inaccessible even when researchers work on site.

### Synthetic data resembles real data without copying individual records
[08:49](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=529s)
Mukherjee defines synthetic data as a machine-generated dataset whose samples have the same types of features as a real dataset, while no generated sample maps to one unique original sample. Rajotte gives a simpler model-based example. A fitted curve can generate new points by sampling along the curve and adding noise. The result looks realistic within the limits of the model, without collecting more real-world data. The useful test is whether the data supports a task, not only whether it looks convincing.

### PrivGAN trades some formal privacy strength for higher utility
[11:57](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=717s)
Mukherjee explains that earlier differentially private GAN approaches add noise during training so that any one sample has little effect on the model. That gives a strong guarantee, but he says the utility loss can make generated models unusable. PrivGAN instead randomly partitions the dataset into non-overlapping subsets, trains a generator and discriminator for each part, and adds a centralized adversary. The adversary tries to identify which generator produced synthetic data. The generators must fool both their own discriminator and the shared adversary, which pushes their learned distributions to be similar and reduces overfitting to a subset.

### Synthetic models can leak membership in the training data
[15:21](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=921s)
A membership inference attack asks whether a particular sample was used to train a model. Mukherjee says this can affect GANs as well as other models. An adversary may inspect a GAN discriminator's output because it can assign different probabilities to real images that appeared in training and real images that did not. Synthetic images can also be used in attacks. In healthcare, learning that someone was part of a clinical trial could reveal information about their health condition.

### Federated learning and synthetic data solve different parts of the problem
[26:01](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=1561s)
Rajotte describes adapting the PrivGAN architecture to a federated setting with a small number of data owners, such as hospitals. One owner may have too little data to generate useful synthetic samples, while another owner can help without sharing its original data. Connecting sites can also help address biased distributions, such as a children's hospital working with a general hospital. Mukherjee says federated learning may be better when there is one defined model to train. Synthetic data is more flexible because people can inspect it, train models for different labels, and use it for multiple tasks.

### Synthetic data has uses beyond privacy-preserving sharing
[27:55](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=1675s)
Mukherjee identifies sharing protected medical data and augmenting datasets as two main applications. Companies could share synthetic customer data between teams or release it instead of anonymized real data. Rajotte adds that synthetic data can stress-test a system with large volumes of records and help debug data problems. A Google example used synthetic data to investigate why a model failed on a subset, revealing issues such as inverted image colors or incorrectly grouped stop words. These debugging tasks do not require the generated data to be highly realistic.

### GANs can hide minority cases and outliers
[32:11](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=1931s)
Mukherjee explains mode collapse, where a GAN begins generating only a subset of the variation present in the source data. Privacy methods can add another problem because unusual records have a higher risk of identification and may therefore be obscured more. Outliers may disappear from a synthetic dataset, or classification performance on them may become much worse. Rajotte says conditional GANs could generate more examples of an underrepresented class, but the result cannot be better than the information available in the original data.

### Healthcare adoption still needs real-world validation and domain expertise
[35:30](https://www.youtube.com/watch?v=dr9qvHc7YmU&t=2130s)
Mukherjee says the work is still at the research stage, with experiments on toy datasets and early work on more complex healthcare data and images. Healthcare images contain artifacts and labeling differences across sites, unlike highly curated public datasets. The researchers are pursuing proof-of-concept demonstrations with university, Microsoft, cancer, and other collaborators to address regulators and clinicians. Mukherjee's final lesson is that machine learning researchers need to work with healthcare experts, who understand the data and the limits of the methods.

## Notable quotes
- Sumit Mukherjee: "Synthetic data is basically a machine generated data set or sample which has the same type of features as a real data set or sample but each sample in the dataset does not map to a unique sample in the original data set." (09:08)
- Sumit Mukherjee: "What PrivGAN does is it borrows from the notion that you don't want any one sample to overwhelmingly dictate the model generation process, but instead of one sample we loosen the definition and make it a subset of samples." (12:43)
- Sumit Mukherjee: "The benefits for synthetic data is that it's much more polyvalent." (26:44)
- Jean-François Rajotte: "Data is like money, everybody's showing it off but nobody wants to share it, even within organization." (39:02)
- Sumit Mukherjee: "Data scientists are not a substitute for actual healthcare experts." (40:10)

## Tools & references mentioned
- MLOps Community
- YData
- University of British Columbia
- Microsoft AI for Good Research Lab
- PrivGAN
- GANs
- differential privacy
- federated learning
- membership inference attacks
- Wasserstein GAN
- PacGAN
- Netflix Prize
- Google
- Microsoft BC
- Fred Hutch
- Cascadia Alliance

## Who should watch
- You work with medical, customer, or other protected data that cannot be moved between teams or institutions.
- You are comparing synthetic data with federated learning and need to understand their different reuse and privacy trade-offs.
- You are building healthcare machine learning systems and want a candid account of bias, membership attacks, data quality, and regulatory barriers.

## Related talks

- [Unleashing Sensitive Datasets with Distributed Data Science](https://mlopstalks.com/talks/unleashing-sensitive-datasets-with-distributed-data-science) (Blaise Thomson, Bitfount, 39:26)
- [The revolution of Federated Learning](https://mlopstalks.com/talks/the-revolution-of-federated-learning) (Fabiana Clemente, MLOps Community & Ramen Dutta, TensoAI, 29:22)
- [From Expectations to Synthetic Data Generation](https://mlopstalks.com/talks/from-expectations-to-synthetic-data-generation) (Fabiana Clemente, YData, 55:39)
- [Are Privacy-Enhancing Technologies a Myth?](https://mlopstalks.com/talks/are-privacy-enhancing-technologies-a-myth) (Fabiana Clement, MLOps Community & Charles Radcliffe, 22:59)
- [When Machine Learning Meets Privacy, Episode 1](https://mlopstalks.com/talks/when-machine-learning-meets-privacy-episode-1) (Fabiana Clemente, YData, 18:44)