GPU data centers now need to plan for much higher rack densities, moving from sub-10 kW racks to 40 kW, 80 kW, and planned 130 to 135 kW racks.
2
Liquid cooling, modular construction, renewable power, and free-air cooling are becoming central to operating dense GPU infrastructure.
3
GPU cloud providers need orchestration that places workloads efficiently, reduces idle capacity, and hides hardware complexity from customers.
Summary
Craig Tavares describes the infrastructure behind Buzz HPC's GPU cloud, drawing on experience with telecom networks, data centers, power plants, and cloud platforms. He explains why AI changed data center design. Rack power has risen sharply, heat removal has moved from air to liquid cooling, and future facilities need flexible plumbing and electrical systems because standards keep changing. Buzz chooses locations with renewable power, including hydroelectric and geothermal sources in Canada, Sweden, and Paraguay. Tavares also explains the operational work behind a GPU cluster, from installing racks and firmware to testing networks, storage, drivers, and operating systems. On the software side, orchestration helps schedule workloads and reduce wasted GPU capacity. He argues that cloud, on-premises, and hybrid deployments each have a place, although hybrid systems are difficult to run because data movement is expensive. Sovereignty is becoming a major requirement as governments seek domestic control of infrastructure, operations, and data.
GPU cloud infrastructure starts with power, land, and local data requirements
Tavares says Buzz HPC operates GPU clouds from facilities in Sweden and Canada and serves customers with sovereign requirements. His earlier work connected telecom networks, data centers, and cloud platforms. Regional clouds remained useful because enterprises and governments wanted to know where their data was stored and needed domestic data residency. He says the first building blocks for a data center are power and land. AI has increased demand so much that location decisions now depend heavily on available energy, rather than only on proximity to large population centers.
Renewable energy can shape where GPU facilities are built
Buzz looks for locations with abundant renewable energy rather than producing power itself. Tavares describes Sweden as an anchor site using green energy, Quebec as another important base, and a large hydroelectric-powered acquisition in Paraguay through the parent company Hive Digital. Quebec attracted data centers because of cheap hydroelectric power originally developed for industrialization. Battery manufacturers also moved there because battery production requires substantial electricity and benefits from renewable sources. Tavares presents energy availability as a condition for scaling GPU facilities over the next several years.
AI workloads have pushed rack power from 10 kW toward 135 kW
Traditional data centers were designed around racks below 10 kW. The release of NVIDIA H-series GPUs pushed many systems toward 40 kW racks, then 80 kW racks within roughly three to six months. Buzz is planning for 130 to 135 kW racks with GB200 systems. Tavares says Jensen Huang has already encouraged data center operators to think about 400 kW racks. At those densities, the rack itself may no longer be the right unit of design. Power distribution, plumbing, and heat removal all need to accommodate further increases.
Liquid cooling and modular construction are responses to rising density
Air cannot remove heat from dense GPU systems as effectively as liquid, so data centers are shifting to liquid or water cooling. Tavares says the pipes needed to move enough liquid through the servers are already large. His engineers joke that they should not screw anything to the ground because the design may need to change the following year. He describes modular and adaptable construction as necessary because rack densities keep rising and no single design has solved every future requirement. Northern climates also allow free-air cooling for much of the year, reducing cooling overhead.
Cooling systems can reuse local resources and waste heat
Tavares describes several ways to reduce cooling costs. In northern Canada and Sweden, operators can use cold outside air for much of the year. He gives an average PUE of about 1.3 for these conditions, meaning cooling and facility systems add roughly 30 percent to server energy use. Hotter locations can use techniques such as evaporative cooling. Data centers can also send waste heat to nearby businesses through district energy systems. Another option is deep-lake or deep-water cooling, similar to the system used across much of downtown Toronto.
Standardized NVIDIA reference architectures help providers deliver reliable clusters
Buzz has chosen NVIDIA as its primary chip partner. Tavares says NVIDIA supports its cloud partners and provides reference architectures that help providers deliver a consistent customer experience. Standardization does not have to freeze a design. Providers can adopt process improvements in later versions. NVIDIA also gives Buzz early visibility into future hardware and software, which helps the company plan for higher density and changes in the software stack. Tavares connects this preparation to uptime, performance, and the large cost of the hardware.
Deploying a GPU cluster requires weeks of systems and network work
A data center can take 12 to 18 months to build, and GPU deployments require additional coordination with server vendors and OEMs. Tavares describes a cluster as highly complex, with many network connections that must work together. After installing racks, the operator connects the network, commissions servers, checks firmware and drivers, installs and patches the operating system, and pressure-tests the system. Even with a repeatable process, deployment can still take weeks. Storage, memory, networking, and GPU interconnects all need enough throughput for the cluster to behave like a unified system.
Orchestration reduces GPU waste and gives customers a simpler service
Buzz offers different consumption models, including bare metal and Kubernetes, and may also support Slurm. Tavares says bare metal can lead to waste because users treat servers as individual pets and hold onto scarce GPUs. Orchestration lets the provider schedule workloads in efficient locations and increase cluster utilization. Compression and other model optimizations can help, but the provider also needs to manage placement and usage. Tavares wants to show customers where they are wasting capacity and recommend a cheaper GPU when the workload does not require the most expensive option.
Hybrid AI deployments are useful but expensive to operate
Tavares sees cloud, on-premises, and hybrid infrastructure working together. A company might train on its own hardware, use cloud capacity for bursting, or run inference in the cloud while keeping another phase on-premises. The choice depends on utilization, data location, compliance, and cost. Hybrid is difficult because data often has to move between hyperscalers and specialized GPU platforms, and egress charges can change the economics. Buzz tries to provide compute, storage, security, resiliency, and application hosting in one platform so customers do not repeatedly move data between systems.
"Don't get used to one standard. That's why I'm saying don't screw anything down to the ground."Craig Tavares12:37
Who should watch
You are planning a GPU cluster and need to understand the power, cooling, network, storage, and commissioning work behind it.
You are deciding whether to use cloud GPUs, dedicated on-premises hardware, or a hybrid setup and need to account for data movement and utilization.
You work on infrastructure for a government or regulated organization with requirements around data residency, domestic control, and sovereign operations.
