Luis Ceze argues that deployed machine learning models should be treated like other application code, while model creation and data preparation still need a specialised workflow.
2
Hardware dependence makes ML deployment slow and fragmented because teams must choose targets, libraries, compilers, and low-level optimisations for each model.
3
OctoML uses Apache TVM and automated benchmarking to produce portable model packages, compare hardware and price-performance options, and fit into existing DevOps flows.
Summary
Luis Ceze explains why he objects to using MLOps as a label for every part of machine learning delivery. Once a model is ready to deploy, he wants it to behave like ordinary application code and use DevOps practices. Model creation remains different because it depends on data, experimentation, and evaluation. The deployment problem is harder than it looks because models become high-performance linear algebra programs that need hardware-specific code. Apache TVM addresses this with compilation, graph optimisation, and machine learning that selects among many possible code-generation choices. OctoML builds a deployment platform around that work. It explores hardware targets, packaging options, batching, concurrency, latency, and throughput per dollar, then integrates with CI/CD systems. Ceze also connects efficient ML infrastructure to energy use, accessible AI, and his research on DNA storage and molecular computing. He is candid about academia, saying research has impact when it either addresses real industry needs or pursues fundamental questions without pretending to solve an immediate product problem.
Deployed models should become ordinary software components
Luis Ceze distinguishes model creation from model deployment. Models combine code and data to perform computations that were difficult to express with code alone, such as computer vision and natural language processing. That makes the path from a problem to a trained model specialised. Once the model is ready, however, Ceze argues that it should be embedded in the surrounding application and treated like other code. He uses MLOps for the model-creation flow, while the deployment and application-integration work should fit within DevOps. Mihail Eric restates this as a question of where traditional software operations end and specialised ML work begins.
Hardware-specific optimisation has made ML deployment a manual expert task
Ceze says machine learning models become high-performance linear algebra code, so they need to use processors, GPUs, or accelerators efficiently. Today that often means talented programmers write low-level loops with particular data types and instructions. Every hardware vendor also maintains its own system software, which creates fragmentation. He compares the situation with the mainframe industry, where software and hardware were tightly tied to one vendor. Teams must select a target, libraries, or a compiler, test performance, and repeat the work for other targets. Some teams respond by choosing a cloud instance and accepting suboptimal performance. Others cannot deploy to the edge because the model is too slow.
Apache TVM replaces hand-tuned library choices with compilation
Apache TVM, or Tensor Virtual Machine, provides an interface that abstracts hardware details from the person creating the model. Ceze describes its approach as compilation rather than interpreting a model through a fixed library. TVM can optimise a computation graph, fuse operations such as matrix multiplication and convolution, order computations to match the hardware, and select available processor or GPU instructions. There are too many possible code-generation choices to try them all, so TVM uses machine learning to predict which choices are worth testing. Ceze says the project was designed to extend across frameworks and hardware targets, and that it became an Apache Software Foundation project after production use and adoption by hyperscalers and hardware vendors.
TVM and TensorRT can be combined instead of treated as competing systems
Mihail Eric asks how TVM relates to ONNX Runtime and TensorRT. Ceze explains that ONNX Runtime mainly uses a library approach with pre-optimised operators, while TVM can generate fresh code for a whole model or parts of it. TensorRT is NVIDIA's machine learning compilation system and benefits from NVIDIA's knowledge of its own hardware. Ceze says TensorRT is also specialised around the model classes NVIDIA prioritises. TVM can route different parts of a model through different systems, using TensorRT where it performs well and TVM elsewhere. He presents this as a complementary approach that can help with emerging models and the long tail of models that have received less hand optimisation.
OctoML automates deployment across hardware and price-performance choices
OctoML is described as a machine learning deployment platform built to bring DevOps agility to ML deployments. It hides whether a model uses a library-based or compilation-based path and produces portable, runnable code for selected targets. The platform explores how a model behaves on different hardware, supports major cloud providers and some edge targets, and packages results for deployment through options such as a Python wheel or shared library. Ceze gives a latency example: a team may require inference below 100 milliseconds, then optimise throughput per dollar rather than chase every possible millisecond. OctoML automates tests across batching, concurrency, and related settings, then supports multiple model versions for comparison.
Different users value OctoML for different reasons
Ceze says mature ML teams care about performance and hardware independence because faster models can reduce cloud costs and portability makes scaling easier. Automation also cuts the time expensive engineers spend preparing models. Teams earlier in their ML journey care more about reducing deployment complexity. OctoML targets people who have model candidates ready for production, rather than teams still deciding whether ML can solve their problem. Ceze describes a flow where a model can move from a laptop to the cloud without code changes. For mature teams, OctoML also provides APIs and CI/CD integration so they can send in a model, receive a container, and deploy it without replacing their existing process.
Efficient inference matters because successful models run repeatedly
Ceze connects OctoML's goals to making AI sustainable and accessible. He says the energy cost of training receives attention, but successful models may run inference an enormous number of times, so inference efficiency matters as well. Lower compute costs can make open models usable by more people. He also argues that people should spend their time defining problems and improving models instead of writing hardware-specific code. In his view, reducing human effort is part of sustainability because it leaves more engineering capacity for useful work. Mihail Eric points out that the environmental cost of large language models is often missing from public discussion, and Ceze agrees that both training and repeated use need to be considered.
Molecular computing is a longer-term extension of the same systems work
Ceze connects his work on ML systems with research into biology and new computing substrates. He says silicon electronics is approaching fundamental limits, which motivates research into alternatives such as superconducting logic, quantum computing, and molecular approaches. His DNA storage work with the University of Washington and Microsoft began by examining the gap between data growth and storage capacity. DNA offers high density, durability, and continued relevance because it is part of living systems. Machine learning is used both to encode and decode data and to process noisy sequencing signals. Ceze says his group also demonstrated similarity search in molecular form, using DNA-encoded images and performing the search directly in DNA. He expects machine learning and wet-lab automation to help design such systems.
Academic research needs either real product contact or honest long-term questions
When Mihail Eric asks about moving from academia to industry, Ceze says the outputs differ. Academic work aims to change how people think through papers and students, while a company must build a product people will pay for. He warns academics not to let attractive technology distract them from product discipline. He also rejects the idea that all academic research must address an immediate industry problem. Long-horizon work can be valuable when it follows a fundamental technology trend. The problem is research that claims industry relevance while using an inaccurate picture of industry. Ceze also supports arrangements that let faculty work partly in academia and partly in industry, since companies still need future PhD graduates.
"Once you're at that point why not call it devops like if you do it right and you make a machine learning model once it's ready for deployment like an integral part of applications and use devops i think we're in a good place"Luis Ceze09:55
Who should watch
You are responsible for deploying models across cloud or edge hardware and spend too much time tuning targets, libraries, or inference settings.
Your team has production model candidates, but the handoff from data science or ML engineering to software operations is slow and manual.
You are interested in the systems behind efficient ML, or in how machine learning connects to DNA storage and molecular computing.
