Watch on YouTubeFine-tuning gives teams more control over model outputs and can reduce serving costs by allowing shorter prompts and smaller task-specific models.
Teams can run many fine-tuned model variants from a shared base model, then merge useful discoveries into later training runs.
Fireworks.ai serves model variants based on the same base model on shared GPUs, including different LoRA adapters, to improve utilization and reduce cost.
Dmytro Dzhulgakov describes how teams move from experimenting with many LLM variants to serving them in production. He explains why fine-tuning is useful for business tasks: it can make outputs more controllable, shorten prompts, and allow a smaller model to handle a narrow task. Larger organizations can let engineers create separate fine-tuned variants from one base model, test them in parallel, and later combine the best data or changes. The serving problem is that ordinary batching works well for one heavily used model, while dozens of lightly used variants leave GPUs underused. Fireworks.ai addresses this by placing variants built from the same base model on shared hardware and batching requests that differ only in their adapters. Dzhulgakov also mentions inference optimizations, OpenAI-compatible access, LangChain integration, and free developer credits. The talk is brief and focused on the serving pattern rather than on operational details of a complete production system.
Dzhulgakov says fine-tuning gives businesses more control over outputs than adding a few examples to a prompt. It can also reduce serving cost because shorter prompts are processed faster and more cheaply. A smaller model may handle a narrow task after fine-tuning, where a much larger model would otherwise be needed. He gives the example of using a 7-billion-parameter model for a specialized task instead of relying on a 70-billion-parameter model.
With foundation models, one base model may support many tasks. Dzhulgakov describes a pattern in which engineers branch from that model and create separate fine-tuned variants for production tasks or experiments. They can test these variants in parallel without changing one another's work. Teams can later combine their best discoveries, merge datasets, or perform a larger training run on the base model.
Full fine-tuning can be expensive and difficult to get right on small datasets. Dzhulgakov describes parameter-efficient fine-tuning, where the main model remains in inference mode and only a smaller set of parameters is learned. He names LoRA as one widely used method. LoRA adds a small adapter to each model layer, reducing the number of parameters trained and the GPU resources needed.
LLM inference is heavily constrained by moving data from memory. Serving one model efficiently therefore depends on sending many requests to it at the same time and batching them. Dzhulgakov says that parallel requests can improve efficiency substantially. The same approach breaks down when a team has dozens or hundreds of fine-tuned variants, because each variant may need separate GPU allocation while receiving little traffic.
Fireworks.ai groups variants that were fine-tuned from the same base model and deploys them on shared hardware. Requests for those variants can be processed together even though they use different LoRA adapters. This preserves batching efficiency while allowing teams to customize the model. A newly fine-tuned variant can also be uploaded and deployed on existing hardware instead of requiring a new allocation.
Dzhulgakov says Fireworks.ai combines cross-model batching with other LLM inference techniques. He mentions multi-attention, different forms of batching, disaggregating compute, and hardware configurations for particular use cases. The stated goal is to reduce the end-to-end cost of deploying and serving fine-tuned and base models compared with platforms that charge for separately allocated GPU hours.
The platform offers open-source models through its inference engine and is compatible with the OpenAI API format. Dzhulgakov says developers can use existing integrations or change the base URL in an existing SDK. He also mentions a LangChain integration and access through the Fireworks.ai REST API.
"We are OpenAI compatible, which means that you can use existing integrations or just change the base URL in an existing SDK."09:45