Delivery Hero moved from Kubeflow pipelines and API calls to self-hosted asynchronous actors after rate limits, retries, and operational work made large-scale processing difficult.
2
Asya represents each AI or backend step as an independent actor that receives a message, adds information to its payload, and forwards it through a route.
3
The framework targets near-real-time workloads measured in seconds or minutes, with zero-to-infinity autoscaling and an HTTP gateway for synchronous clients.
Summary
Artem Yushkovskiy describes how Delivery Hero processes restaurant images with AI. The original system used Kubeflow pipelines that called external APIs, but scaling to hundreds and thousands of pipelines caused rate limits, random failures, long backoffs, and heavy engineering overhead. The team first moved models into a self-hosted cluster with SQS queues, then split the workflow into distributed asynchronous actors. Their open-source Asya framework has no central pipeline definition. A message carries a route, a pointer to its current step, and a payload that actors enrich as it moves through the system. Actors can scale independently, while routers can change future steps or create loops. Asya includes retries, error handling, fan-out, autoscaling, and a stateful HTTP gateway for clients that need synchronous calls. Yushkovskiy is clear about its limits: fan-in is unfinished, training is not its target, and Kubernetes remains part of the platform setup.
API-based pipelines broke down when image processing reached scale
Delivery Hero began with a simple Kubeflow pipeline that called AI APIs and connected them to backend logic. The design worked for an MVP, then failed as the team ran hundreds and thousands of pipelines. Requests were rate limited, errors appeared in random places, and pipelines waited through exponential backoff. Artem says the team spent 60 to 80 percent of its engineering effort keeping the system running. The image workflow also had a large cost constraint because it generated and assessed multiple enhancements for each restaurant image.
Self-hosting models and adding queues solved only part of the problem
The first major change was to self-host the AI models while keeping a request-response interface for the callers. A separate cluster ran the models, and SQS message queues connected them to the pipelines. This removed the external AI rate limits and lowered costs when the models were used enough to justify self-hosting. The pipelines still waited for work, accumulated failures, and remained difficult to scale. Artem says the team still spent at least half its time operating the pipelines at scale.
Independent asynchronous actors fit workflows with very different step speeds
The team then decoupled the workflow into actors. One actor may take milliseconds, while another takes seconds or minutes. Each actor receives a request, scales when needed, processes it, and sends it to the next actor. Queues and error processors handle retries when failures are recoverable. The actors run models on local GPUs, so capacity grows only when traffic creates work. Artem says this removed rate limits, avoided a central orchestrator, and reduced cost while keeping the system usable for near-real-time processing.
Asya puts the route in the message instead of in a pipeline definition
Asya, short for asynchronous actors, treats the message as the main unit of execution. The message contains a route, a pointer to the current step, and a payload. Actors use an enrichment pattern: they add information to the payload and pass it on rather than deleting earlier data. A router can rewrite the future route, select different paths, or add loops. The framework allows only future history to be changed, so a router cannot move the pointer backward and rewrite completed steps.
The framework separates Python work from platform operations
Data scientists write Python functions that take a dictionary and return a dictionary. They can return an error or multiple dictionaries for fan-out without defining the infrastructure or pipeline structure. Platform teams configure an asynchronous actor custom resource, autoscaling, and the workload. Asya injects the sidecar and supporting components. This gives the application author and platform team different interfaces to the same system. The framework also provides a stateful HTTP gateway, so synchronous clients can submit work and receive a result after the asynchronous route completes.
Asya is aimed at near-real-time workloads rather than millisecond APIs or long batch jobs
In the Q&A, Artem places Asya in the seconds-to-minutes range. It is not designed for millisecond response times because workloads may scale from zero and models can take time to load. It is also not aimed at jobs where latency does not matter for days. His own project processes image workloads with timeouts measured in minutes and is trying to reduce a five-minute target toward one minute. He says a typical 50-step workflow may contain only five or ten slow steps, with the rest made up of database calls and transformations.
The design still has unfinished parts and operational costs
Artem says fan-out is easy, while fan-in is hard because a stateful step must wait for a defined set of messages or a timeout. The framework does not directly solve GPU warm-up, so teams may need prebuilt machine images, persistent volumes, or some overprovisioning for predictable latency. Large payloads can grow through 25 or more steps, so images and videos should live in a persistent volume, S3, or another blob store. Kubernetes resource ownership is also difficult when Asya, an autoscaler, and another framework manage related deployments.
