The model runs on your GPU, but concurrent requests exhaust its memory. A quiet endpoint takes too long to wake up. A faster model does little for users because requests wait elsewhere in the service. Start with the control that downloadable weights give you and the deployment work that comes with it. Then examine accelerator communication, memory pressure and the serving techniques that change what fits on the hardware. Application and service accounts show how narrow workloads and related model variants alter deployment choices. Autoscaling brings the discussion back to uneven real traffic. Finish with a team that deliberately hosts different models in different ways. These 2023 and 2024 examples explain operating tradeoffs; their prices, hardware comparisons and product capabilities belong to that period.
Why first: Kiely frames downloadable weights as control over deployment, data flow and model updates. He also lists the service machinery that an API used to hide. Start with that exchange: the reason to run an open model should be clear before choosing a GPU or an inference engine.
Why second: Rovira turns the deployment into concrete requirements: model-specific environments, enough GPU memory, streaming and monitoring. His comparison of infrastructure approaches makes the maintenance burden visible. This establishes what your service must supply before optimizing the computation inside it.
Why here: Splitting a large model across accelerators introduces communication costs as well as capacity. Jain connects that constraint to batching, quantization and horizontal scaling. A model that technically fits is only a starting point; the next explanation shows why the remaining memory can determine useful concurrency.
Why here: Lacroix explains the memory used by weights and the attention cache, then examines allocation and scheduling. His 2023 discussion of paged attention and chunked prompt processing connects memory waste to stalled decoding. Use these mechanisms to form a bottleneck hypothesis, rather than treating a larger accelerator as the only response.
Why here: Campos supplies an application-level example of changing the workload instead of only enlarging the hardware. His web-summarization account combines a smaller model and compression with a deployment sized around available instances. The historical result is useful because the serving design follows one concrete task, not a general model ranking.
Why here: Many quiet variants create a different problem from one busy model. Dzhulgakov describes batching requests for adapters that share a base model onto common hardware. This is the serving consequence of customization, rather than a lesson in training adapters: separate allocations can waste the capacity you just worked to recover.
Why here: Kadous brings the model fleet into contact with spiky traffic. His service needs per-model prompt configuration, warm replicas and startup behavior that can respond before a burst is over. The case complicates a simple scale-to-zero promise and shows why autoscaling settings belong in the deployment design.
Why last: Linkov describes choosing hosting separately for different models and finding latency in the queue rather than the model. That is a useful final check on all the preceding optimization: owning the weights does not require owning every endpoint, and faster inference is wasted if the surrounding request path still misses its target.