Declarative MLOps defines the desired model-serving layout in Kubernetes files and lets the Kubernetes control plane schedule and maintain it.
2
A model can be packaged in one container for HTTP serving, message-queue processing, or scheduled batch prediction, with the startup command selecting the mode.
3
GitOps connects pull requests, container builds, image registries, and Kubernetes deployment through CI checks and tools such as Argo CD or Flux.
Summary
Rahul Parundekar explains how Kubernetes can provide a repeatable way to package and serve machine learning models. Declarative MLOps means describing the desired deployment, such as replica counts, container images, resources, and health checks, while Kubernetes handles scheduling and orchestration. He compares three serving patterns: synchronous HTTP endpoints, asynchronous message queues, and scheduled batch jobs. Rahul then walks through a container structure with a model class, health and prediction handlers, worker processes, tests, Docker instructions, and optional model-download stages. The same container can support all three modes through different commands. For delivery, he describes GitHub-based CI, pull-request tests, immutable image tags tied to Git commit hashes, vulnerability checks, and GitOps deployment with Argo CD or Flux. He also discusses environment overlays with Kustomize and Kubernetes observability. The talk is practical and frank about management complexity, debugging custom resource definitions, air-gapped environments, and the time teams need to learn Kubernetes.
Declarative MLOps describes the desired state and lets Kubernetes carry it out
Rahul defines the declarative approach as describing what needs to be accomplished without specifying how it should happen. An ML engineer can declare that a model server needs two replicas, a backend needs one replica, and particular container images should be used. Kubernetes stores those definitions, schedules pods on nodes, manages networking, and exposes the resulting services to users. Rahul presents this as a way to apply Kubernetes' existing orchestration model to model training and serving workloads.
Kubernetes is useful for model serving, although its management cost grows with scale
Rahul says Kubernetes has moved beyond the early stages of the technology hype cycle and is increasingly adopted for scalable workloads. He points to large companies using it for training and serving across thousands of nodes. He also gives several limits: operating complexity rises with scale, teams must balance infrastructure costs, custom resource definitions can be difficult to debug, and air-gapped deployments may lack mature support. He says teams should also consider whether they are ready to work with Kubernetes.
The serving pattern should match how predictions are requested
Rahul describes three ways to serve a model. An HTTP endpoint fits an interactive user action where someone waits for the prediction, with response times ideally in the hundreds or lower hundreds of milliseconds when possible. A message queue fits an event such as a file upload when no user interface needs an immediate result. Batch processing fits large scheduled workloads, such as nightly predictions, where no user is waiting. Kubernetes deployments, jobs, and CronJobs provide the basic primitives for these patterns.
One container can support HTTP, queue, and batch execution
Rahul recommends a model class with a single load method, model warm-up, and a predict method that accepts Python objects and returns predictions. The container then adds the files needed for each mode. HTTP serving needs health and prediction handlers, an application entry point, Nginx, a WSGI server, and a shutdown handler. Queue serving reads and writes serialized messages and can use a process manager. Batch serving reads input files, predicts in batches, and writes results to object storage. An entry-point command selects the mode.
Container checks should cover both model behavior and software behavior
Rahul uses unit tests as production-serving sanity checks rather than as a replacement for training evaluation. His example checks whether a zero-shot prediction is correct and verifies returned scores. He also recommends linting and type checks. Docker images should run as a non-root user, pin or update dependencies, and avoid embedding passwords or access tokens. Models can be downloaded while building the image, although very large models may need another approach. A two-stage Docker build can download a model first and copy it into a production image.
CI should connect reviewed code to an immutable model image
Rahul separates containerization from CI/CD and uses Docker Compose for local development and CI tests when the model depends on services such as a database or Redis. Pull requests run the container and its tests, with branch protection requiring review and a successful check before merging. After a merge, the pipeline builds and pushes the image to a registry. Rahul prefers tagging images with a short Git commit hash, because this connects the running container to the exact code and avoids accidentally reusing a version tag.
GitOps deploys the tested image through Kubernetes definitions
Rahul describes a separate repository containing the Kubernetes application definition. Argo CD or Flux watches that repository and applies changes to the cluster. HTTP serving uses a Deployment, resource settings, health checks, and a Service. Queue workers use a Deployment without a Service, while batch work uses a Job or CronJob. Kustomize can provide environment-specific overrides, such as different image versions for development, staging, and production. Rahul also recommends cluster observability tools such as New Relic.
AI Hero uses event-driven workers and model-backed spreadsheet columns
Rahul shows how A.I. Hero uses Kubernetes behind a spreadsheet-like data-cleaning interface. Models can validate email addresses, format phone numbers, clean locations, and check whether an image contains a face. Users can also label text directly in the spreadsheet and train or run predictions on a column. The system uses event-driven workers and a model library, which matches the queue-based serving pattern described earlier. He says the model features were behind a feature flag at the time of the talk.
"If the user action or a back-end process like let's say a file upload triggers the model prediction and there's no UI waiting for the response, message queue a model working behind a message queue would be a great option."Rahul Parundekar15:25
Who should watch
You are deciding whether to build model-serving containers yourself or adopt a Kubernetes-native serving tool.
Your team needs one deployment pattern for interactive predictions, event-driven workers, and scheduled batch jobs.
You want a concrete GitOps path from pull-request tests to immutable container images and Kubernetes deployments.
