# Vision Pipelines in Production: Serving & Optimisations

Biswaroop Bhattacharjee, Prem AI | AI in Production 2024 | 13:24

Source: https://www.youtube.com/watch?v=POL7gSRFXqE
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/vision-pipelines-in-production-serving-optimisations
Published: 2024-04-04
Tags: fine-tuning, inference, latency, model-serving

## TL;DR
- Fine-tuning a small, high-quality dataset can give image generation the control needed for specific layouts, attributes, and aspect ratios.
- A production vision pipeline can improve image quality with an upscaling and image-to-image loop that restores details lost during generation.
- Fusing LoRAs, compiling models, quantizing where possible, and batching or parallelizing pipeline stages can reduce latency and improve throughput.

## Summary
Biswaroop Bhattacharjee explains how to move a constrained image-generation system from experimentation toward production. His example generates realistic planet images with fixed composition, configurable visual attributes, and an ultrawide 21:9 aspect ratio. He recommends building an ordered prompt strategy and curating a small dataset with enough variation, then fine-tuning a suitable model. For Stable Diffusion XL, he chooses dimensions close to the aspect ratios used during training. Image quality can be improved with a custom upscaling pipeline: upscale the image, resize it, run an image-to-image step to restore detail, then upscale and resize again. For serving, he recommends fusing LoRAs, compiling models, using quantized variants where possible, and applying micro-batching and model multiplexing. He is also direct that custom upscaling remains an open, creative area with no single correct workflow.

## Key ideas
### Production generation needs explicit visual constraints
[01:51](https://www.youtube.com/watch?v=POL7gSRFXqE&t=111s)
Bhattacharjee frames the problem as generating millions of realistic planet images while enforcing composition and style rules. The planet must stay within a defined region, leave space at the sides and top, and use a 21:9 ultrawide aspect ratio. Other attributes need to remain controllable, including planet colour, moons, rings, nebulae, sky, and atmosphere. A plain text-to-image prompt does not provide enough consistency or control for this use case.

### A small, well-curated dataset can support fine-tuning
[03:31](https://www.youtube.com/watch?v=POL7gSRFXqE&t=211s)
He proposes an ordered prompt creation strategy rather than random descriptions. Prompts can begin with the planet colour, then describe atmospheric features, moons, and the surrounding sky. The team can create a small, high-quality image set with tools such as Photoshop or Midjourney, as long as it covers useful variation in the attributes that the prompts will control. The model choice depends partly on the available GPU resources.

### Model dimensions should stay close to the training aspect ratios
[06:05](https://www.youtube.com/watch?v=POL7gSRFXqE&t=365s)
The fine-tuned results follow the desired planet shape and composition more closely than the untuned outputs. For the 21:9 target, Bhattacharjee uses Stable Diffusion XL and chooses dimensions of 1536 by 640. He explains that this is close to an aspect ratio used in Stable Diffusion XL training, which he expects to produce higher-quality output than choosing an unrelated size.

### An upscaling loop can restore details after generation
[07:04](https://www.youtube.com/watch?v=POL7gSRFXqE&t=424s)
Bhattacharjee describes a custom upscaling pattern for adding missing detail. The pipeline first upscales the generated image, resizes it to the required size, and passes it through an image-to-image pipeline. It then upscales and resizes the result again. In his examples, this produces more defined magma rings and a more distinct nebula than a normal, smudged-looking upscale. He points to Comfy workflows as a public source of related approaches.

### Inference optimisations should target the whole pipeline
[09:18](https://www.youtube.com/watch?v=POL7gSRFXqE&t=558s)
For parameter-efficient fine-tuning, he recommends fusing LoRAs into the model when possible because this can improve latency. He also recommends using compilation and quantized models where they work for the chosen setup. Stable Fast packages several of these optimisations, especially for Hugging Face Diffusers, although Bhattacharjee notes that there are caveats and that other integrations may require additional work.

### Batching and multiplexing can improve service throughput
[10:55](https://www.youtube.com/watch?v=POL7gSRFXqE&t=655s)
Bhattacharjee describes Ray as a serving framework that can provide micro-batching and multiple-model multiplexing. These capabilities matter because a deployed vision service usually contains a pipeline rather than one isolated model. He argues that stages should be parallelized and batched wherever possible, since the pipeline has more opportunities for throughput and latency improvements than a single-model endpoint.

### Quality matters more than dataset size alone
[11:46](https://www.youtube.com/watch?v=POL7gSRFXqE&t=706s)
His closing advice is to focus on the quality of both images and prompts even when the dataset is small. He recommends taking the available low-effort performance improvements, then optimizing the complete pipeline rather than stopping at model inference. Custom upscaling deserves experimentation because it can materially change the visual result. Prem AI also announced a grant program offering free fine-tuning jobs, model deployments, and ad hoc machine learning support.

## Notable quotes
- "Fine tuning can be one of the solutions, but why fine tuning, why not just do any other approach?" (03:31)
- "You come up with a prompt creation strategy and you kind of curate a small high quality data set." (04:44)
- "There is no correct way currently. Everyone is still figuring it out." (07:35)
- "Since you're deploying whole pipelines and not just a single model, there are lots of opportunities to parallelize and batch." (12:13)

## Tools & references mentioned
- Prem AI
- Demetrios Brinkmann
- Photoshop
- Midjourney
- Stable Diffusion XL
- Comfy workflows
- Stable Fast
- Hugging Face Diffusers
- Ray

## Who should watch
- You are deploying text-to-image or other vision generation systems and need tighter control over composition, attributes, or output dimensions.
- Your generated images look acceptable at first glance but lose detail during resizing or upscaling.
- You are tuning inference cost and latency across a multi-stage vision pipeline rather than serving one model in isolation.

## Related talks

- [From MVP to Production](https://mlopstalks.com/talks/from-mvp-to-production) (Eric Peter, Databricks & Donné Stevenson & Phillip Carter, Honeycomb & Andrew Hoh, Last Mile AI, 32:53)
- [Productionizing AI: How to Think From the End](https://mlopstalks.com/talks/productionizing-ai-how-to-think-from-the-end) (Annie Condon, 11:11)
- [Scaling MLOps for Computer Vision](https://mlopstalks.com/talks/scaling-mlops-for-computer-vision) (David Espejo, Union & Fabio Grätz, Recogni & Arno Hollosi, Blackshark.ai, 58:54)
- [Scaling AI in Production](https://mlopstalks.com/talks/scaling-ai-in-production) (Srivatsan Srinivasan, AIEngineering, 51:56)
- [End-to-end Modern Machine Learning in Production](https://mlopstalks.com/talks/end-to-end-modern-machine-learning-in-production) (Omar Sanseviero, Hugging Face, 10:30)