# What is the open source ML framework Hermoine

Neylson Crepalde, A3Data | MLOps Meetup | Episode 17 | 16:04
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=H6if5q9Uo3w
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/what-is-the-open-source-ml-framework-hermoine
Published: 2020-06-11
Tags: developer-experience, experiment-tracking, open-source, testing

## TL;DR
- Hermoine gives ML projects a structure that supports reusable code, unit testing, and a clearer path from experimentation to production.
- Hermoine uses MLflow for experiment tracking while keeping the framework flexible enough for teams to shape it around their own needs.
- The framework packages common data, preprocessing, visualization, training, model-wrapping, and project-organization patterns into an ML project structure.

## Summary
Neylson Crepalde presents Hermoine, an open-source ML project structure framework built on top of MLflow. He starts with the gap between casual data science work and professional projects, where teams face repeated code, inconsistent organization, manual work, and weak code quality. Hermoine provides a structure for reusable code and unit tests, while also keeping the project open to changes in implementation. Its model wrapper stores the trained model, preprocessing object, metrics, and input columns together. Because the wrapper works as an MLflow Python model, teams can use MLflow workflows or deploy the model in a Docker container to cloud platforms. The framework also includes reusable helpers for data analysis, normalization, text vectorization, visualization, and metrics. A live project layout shows where data, outputs, source code, configuration, tests, notebooks, and ML components belong. Crepalde is honest that failures will happen, so the project should make failures easier to detect and recover from.

## Key ideas
### Hermoine addresses the gap between experiments and production code
[01:00](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=60s)
Crepalde contrasts casual data science with professional ML work. Teams encounter repeated code, weak organization, code-quality problems, manual work, and a gap between experimentation and production. Hermoine gives the project a structure to build on. That structure encourages reusable code and unit tests, so the project does not depend on one large notebook or a collection of disconnected scripts. The aim is to make the work easier to organize as it moves toward deployment.

### Hermoine treats failure recovery as part of project design
[01:55](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=115s)
Crepalde rejects the idea that a project can be flawless. Things will fail, and the practical questions are when they will fail, whether the team is ready, and whether it can recover quickly. Hermoine's structure is intended to make those concerns visible during development. Unit testing helps prevent some errors, while an organized project gives the team clearer places to change code and inspect the source of a problem. The framework therefore addresses operational work through both tools and the way the project is built.

### MLflow was chosen because Hermoine can remain flexible
[03:12](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=192s)
Hermoine uses MLflow to track experiments and log metrics and artifacts. Crepalde says the choice was based on how reasonable and integrable MLflow felt for their needs, rather than on a partnership with MLflow or Databricks. The framework is deliberately smaller than a system that forces teams into one workflow. He compares this with tools such as Kubeflow and DVC, which he describes as useful but, in his experience, more likely to require teams to adjust to the tool. Hermoine is meant to adjust to the team's needs.

### The framework separates data access, preprocessing, visualization, and training
[06:47](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=407s)
Hermoine defines an abstract DataSource class with spreadsheet and database implementations for imports, connections, and queries. A preprocessing class holds preprocessing and feature-engineering methods, with existing normalization and text-vectorizer classes available for reuse. A visualization class contains helpers for regression analysis, plots, and residual-distribution checks, while leaving room for project-specific plots. The train module acts as the main function. It brings together the data source and preprocessing steps, then calls trainer implementations for particular algorithms or libraries.

### The model wrapper keeps prediction inputs and evaluation details together
[10:36](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=636s)
After training, Hermoine produces a model wrapper containing the model, the preprocessing object, metrics from test runs, and the columns used to build the model. Crepalde says storing the columns helps address past problems with missing columns or columns appearing in a different order. The wrapper exposes methods for prediction, saving and loading the model, and retrieving metrics. It also works as an MLflow Python model, so the result can be handled through MLflow and deployed through its APIs or inside a Docker container on a cloud platform.

### Hermoine supplies a repeatable project layout
[12:57](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=777s)
The example layout has directories for raw and preprocessed data, outputs such as models and figures, source code, configuration, tests, notebooks, and ML components. The source directory contains the main train script and helper functions. The configuration file can hold the project name, developer names, and environments, with room for additions. The ML directory contains the data-source, visualization, preprocessing, trainer, wrapper, and metrics components. Hermoine also creates project files such as requirements, a README, and a code structure.

### MLflow model registration still requires explicit code
[15:15](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=915s)
In response to a question about whether the MLflow wrapper also handles model logging, registration, and lifecycle changes, Crepalde says it does not do those things automatically. The wrapper supports MLflow, but the team still has to write the logging code. He says he will show how to do that in the demo. This sets a boundary around Hermoine's role: it provides compatible project and model structures, while MLflow operations still need to be configured by the user.

## Notable quotes
- Neylson Crepalde: "The main question is not if it's going to fail, it's when it's going to fail, and when it does, am I ready to deal with it? Can I recover fast enough?" (02:21)
- Neylson Crepalde: "We wanted to build a tool that can adjust to your needs, not the other way around." (06:25)
- Neylson Crepalde: "The interesting part is that this wrapper class is an MLflow Python class, so it's totally recognizable by MLflow." (11:58)
- Neylson Crepalde: "It does not do it automatically. You have to code your logs." (15:39)

## Tools & references mentioned
- Hermoine
- MLOps Community
- MLflow
- Databricks
- Kubeflow
- Metaflow
- DVC
- Airflow
- scikit-learn
- XGBoost
- PyTorch
- TensorFlow
- AWS
- Google Cloud

## Who should watch
- You are turning notebooks or experiments into shared ML projects and need a directory structure that separates data, code, tests, and model components.
- Your team wants experiment tracking with MLflow but does not want an orchestration framework to dictate how the whole project must be organized.
- You need a model artifact that keeps preprocessing, metrics, input columns, and prediction methods together before deployment.

## Editor's note

Neylson Crepalde describes Hermoine as a way to move from experimentation to production with reusable code, unit tests, and a repeatable project structure. ZenML records each pipeline run's steps, inputs, outputs, and code version, so teams can trace a model or artifact back to the data and code that produced it. That tracking is built into the workflow rather than left to separate logging code.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [MLflow Open Source Framework Hermione Demo](https://mlopstalks.com/talks/mlflow-open-source-framework-hermione-demo) (Neylson Crepalde, A3Data, 17:39)
- [The Challenges of ML Operations & How Hermione Helps Along the Way](https://mlopstalks.com/talks/the-challenges-of-ml-operations-how-hermione-helps-along-the-way) (Neylson Crepalde, A3Data, 1:01:30)
- [Making MLflow](https://mlopstalks.com/talks/making-mlflow) (Corey Zumar, Databricks, 59:11)
- [mlctl and Hydrosphere Open Source MLOps Libraries Demo](https://mlopstalks.com/talks/mlctl-and-hydrosphere-open-source-mlops-libraries-demo) (Alex Chung, Intuit, 36:24)
- [How Hera is an Enabler of MLOps Integrations](https://mlopstalks.com/talks/how-hera-is-an-enabler-of-mlops-integrations) (Flaviu Vadan, Dyno Therapeutics, 41:33)