Deployment is where a machine learning model starts producing business value, whether predictions are used through requests, batches, or streams.
2
Reproducibility requires tracking the data, code, experiments, models, inputs, and outputs that produced a result.
3
Cubonacci uses a code-first, end-to-end approach with Git, data snapshots, experiments, deployment, and lineage connected in one platform.
Summary
Jan presents machine learning operations through three problems: deployment, reproducibility, and automation. Deployment turns a trained model into a request-based, batch, or streaming service that can inform users and business processes. Reproducibility requires more than saving model files. Teams may need to preserve data snapshots, source data configuration, code, random seeds, hyperparameters, evaluation results, model inputs, and predictions. Automation reduces manual work, speeds iteration, limits mistakes, and helps models respond to changes in the world. In the workshop section, Jan walks through a German credit dataset in Cubonacci. The platform connects a Git repository to a structured project, creates data snapshots, runs parallel experiments, records metadata and lineage, trains a final model, and deploys it as an API. Jan also explains how data schemas can support API contracts and how endpoints can be connected to new deployments without changing downstream systems. He is honest that perfect reproducibility and advanced deployment strategies were still incomplete.
Deployment is where a model starts creating business value
Jan describes deployment as the point where a trained model produces predictions that inform people or automate business processes. The prediction path can be request-based through an API, batch-based on a schedule or new data batch, or streaming through a service such as Kafka or RabbitMQ. He focuses on request-based serving, where a serialized model is placed in a standardized environment and exposed through a serving layer. Deployment can later include A/B testing, shadow runs, feedback from downstream systems, and automated decisions about replacing an older model.
Reproducibility is needed for trust, diagnosis, compliance, and handovers
Jan defines reproducibility as obtaining consistent results when an experiment is repeated. In machine learning, that includes the data, experiments, and predictions. A team needs to reproduce a past decision when a prediction led to a bad outcome, and financial or healthcare settings may require an exact record of how a decision was made. Reproducibility also helps colleagues transfer knowledge when people change jobs. Jan treats it as a sliding scale, since some systems can get close to identical results without reaching perfect repeatability.
Data snapshots require a policy because reconstruction and storage both have limits
Jan presents three approaches to training data. The ideal is to reconstruct a dataset from its configuration, parameters, and source data, although this requires strong controls over the source system and can conflict with GDPR deletion requirements. Saving the full snapshot removes dependence on the source system, but storage can become expensive. A middle policy can save snapshots only for models that reach production. If full storage is impossible, a team can store a hash and later check whether reconstruction produces the same dataset, although a mismatch means the dataset can no longer be reproduced.
Experiment and prediction lineage must include more than the model file
For experiments, Jan lists the training and evaluation code, random number generator seeds, hyperparameters, and results as important records. For predictions, the model and input data are normally enough to reproduce the output, unless stochastic behavior also needs to be controlled. He expects purpose-built storage and pipeline systems to focus more on immutability and lineage. Predictions can be stored with their inputs and outputs in an existing SQL or NoSQL system, or in a specialized store that records which model produced each output.
Automation makes iteration cheaper and helps models respond to change
Automation can begin with starting a process through one click and extend to APIs, data-triggered retraining, scheduled predictions, Git-based workflows, and monitoring alerts. Jan says manual deployment and maintenance make changes expensive and slow. Automation takes time to set up, but later reduces human intervention and mistakes. He also gives the example of a model that detected the major change caused by the coronavirus crisis through frequent retraining. Without automation, a model cannot adapt to changes in the relationship it is trying to model.
Cubonacci combines several machine learning operations through a code-first project
In the workshop, Jan links a Cubonacci project to a GitHub repository and uses the German credit example. The platform treats the codebase as the starting point for machine learning resources, alongside data. Its structure includes an algorithm interface with fit and predict methods, a data loader, configuration for hyperparameters, metrics, and requirements. Jan says this standardized structure lets Cubonacci inspect the project and connect experiments, infrastructure, deployment, and automation.
The workshop records experiment configuration and lineage through data snapshots and commits
Jan creates a data snapshot and an experiment with eight trials running two at a time. The experiment records its configuration, source snapshot, Git commit, algorithm, machine settings, optimization metric, and validation schema. Logs are captured with the automated metadata. After training, the individual trials show their hyperparameters and metrics. Jan explains that a team can return to an experiment, select the same snapshot, and rerun it with the same settings, although random number generator handling for perfect reproducibility was still on the roadmap.
Recommendation systems need live feedback because offline experiments are indirect
Answering a question about online recommendation systems, Jan says the model is often trained against a proxy task because the team does not know what would have happened under a different recommendation. The real evaluation begins after deployment. Teams need feedback such as click-through rate or the time someone watches a recommended movie. The surrounding practices remain similar, including lineage, model serialization, and fast iteration, but the organization must measure user response and connect it to the deployed model.
"That's the moment where you capture the value created by the initial time and money investment into this model."06:32
Who should watch
You are building a machine learning system that has to move from experiments into an API, scheduled job, or streaming process.
Your team needs to explain how a past model decision was produced, but data snapshots, code, and experiment records are scattered across systems.
You are evaluating a code-first or end-to-end approach to machine learning projects and want to see a worked example with experiments, deployment, and lineage.
