GKE Optimization: Rightsizing Workloads, GPUs and Compute Costs

Google’s experience running Kubernetes at scale gives Google Kubernetes Engine (GKE) a strong infrastructure foundation. Yet infrastructure automation can’t determine whether an application’s four-CPU request reflects real production demand. That decision still rests with the teams operating the workloads—and the utilization figures suggest many requests leave substantial room for improvement.

The 2026 State of Kubernetes Optimization Report measured average GKE utilization at 8% for CPU, 20% for memory and 6% for GPUs. It found CPU overprovisioning in 69% of GKE clusters and memory overprovisioning in 79%. The report drew on direct measurements from more than 23,000 Kubernetes clusters and millions of compute resources across AWS, GCP and Azure, gathered before automated optimization was enabled.

Those results don’t point to a missing scheduler or an inability to provision nodes. Google created Kubernetes and has used it at enormous scale, experience reflected in GKE capabilities such as Autopilot. The harder gap is between the resources a workload asks Kubernetes to reserve and what that workload actually consumes.

Start with the workload requests

Low cluster utilization can look like a node capacity issue, but the source of waste may be a pod request that no longer matches the application. Consider a service configured for 2 CPUs and 2 GiB of memory whose normal use is closer to 500 millicores and 700 MiB. Changing the node pool won’t correct that mismatch. Kubernetes is responding to the resources the application declares, not guessing what it could get by with.

The operating model affects the cost of that mismatch. In GKE Autopilot, oversized requests directly raise the resources billed for a workload. In GKE Standard, they can also shape scheduling decisions and prompt the cluster to provision more node capacity than the applications consume. Rightsizing therefore has a direct cost effect in Autopilot and can affect both workload footprint and fleet size in Standard.

Requests should leave enough room for real variability, especially in production. The trouble starts when a cautious estimate becomes a permanent setting, long after the traffic pattern or workload has changed. Compare declared requests with observed CPU and memory use across a representative period. For services with weekly cycles, examining 7–14 days provides a more useful basis than a snapshot from a quiet hour.

Make the first changes on a small group of workloads. Check how the revised requests behave under production traffic, then broaden the rollout when the evidence supports it. A cluster-wide reduction can spread one mistaken assumption across hundreds of deployments; a measured rollout gives teams a chance to catch workload-specific behavior first.

Coordinate autoscaling with rightsizing

Vertical Pod Autoscaler (VPA) can adjust resource requests, while Horizontal Pod Autoscaler (HPA) can change replica counts. Running both without accounting for how they interact can create a feedback loop: VPA changes a request, HPA recalculates scaling based on that request, the replica count shifts, and the resulting utilization feeds back into VPA.

A cautious starting point is VPA recommendation-only mode, where teams can review suggestions before applying them. If a service needs both vertical and horizontal scaling, application signals such as request rate or queue depth can be more suitable HPA inputs than CPU or memory alone. The right configuration depends on the workload, but the control loops need to be considered together.

The report offers evidence that reducing excess capacity doesn’t have to mean accepting more failures: automated rightsizing cut provisioned CPU by roughly half while bringing out-of-memory (OOM) kills close to zero. That result isn’t a promise for every application. It does show why observed behavior can be a better guide than treating every existing request as untouchable.

Give GKE Standard a more accurate fleet baseline

After pod requests reflect production behavior, GKE Standard operators can tune the Compute Engine fleet around a more accurate view of demand. Cluster Autoscaler adjusts node capacity as workloads need it. Node Auto Provisioning can add node pools when a workload requires machine types the cluster doesn’t already offer.

Neither mechanism can identify inflated requests on its own. If ten services each ask for twice the CPU they typically consume, an autoscaler sees those requests as requirements and provisions capacity to meet them. Correcting requests gives autoscaling and node consolidation a better basis for placing pods efficiently, choosing appropriate machine types and removing capacity the cluster no longer needs.

That flexibility comes with an operational trade-off: Standard operators control the nodes, and unused capacity remains part of what they manage. Autopilot abstracts more of that infrastructure work, while Standard offers more direct control over fleet configuration. In either model, accurate workload requirements are a useful starting point.

Match capacity types to workload tolerance

Once the baseline is right-sized, some workloads may not need on-demand capacity. GKE Spot VMs can lower compute costs for jobs that tolerate interruption, with roughly 30 seconds of preemption notice. That makes Spot a poor default for every service, but a possible fit for batch processing, CI/CD runners, sufficiently replicated stateless services and machine learning training jobs that checkpoint their state.

Using Spot safely takes more than selecting a cheaper machine type. Workloads need suitable disruption budgets and termination handling, and clusters need an on-demand fallback for periods when Spot capacity isn’t available. Keeping interruption-tolerant work on Spot where possible, while reserving predictable capacity for services that need it, can make Spot part of the wider capacity plan.

Improve use of GPUs and alternative CPUs

Not every inefficiency comes from excessive CPU or memory requests. GPU utilization on GKE averaged 6% in the report, varied substantially during the day, and was accompanied by relatively uncommon GPU sharing. A workload that uses only part of a GPU much of the time may be a candidate for sharing or partitioning, depending on its performance requirements and the hardware involved.

One production example in the report moved seven models to GPU time-slicing, alongside improved bin-packing and Spot usage. The change reduced costs by more than 70% compared with the previous platform while maintaining latency. The useful takeaway is that teams can sometimes serve the same work with less dedicated GPU capacity by changing how models share hardware.

CPU architecture is another option. ARM accounts for about 9% of the Kubernetes CPU fleet and has grown 3.5 times faster than x86 since the second quarter of 2024. GKE offers ARM choices including T2A and Google’s Axion-based C4A and N4A families. For workloads that support linux/arm64, a limited trial can show whether the shift improves cost and performance before a wider migration.

Architecture changes don’t correct oversized requests. Moving an incorrectly sized service to a cheaper processor may lower the cost of its excess capacity, but it doesn’t remove the excess. Rightsizing first gives teams a clearer comparison when they evaluate different machine types.

Commit after establishing the baseline

Pricing commitments make more sense once teams understand what capacity they reliably need. GCP Sustained Use Discounts apply automatically to eligible Standard node usage, while Committed Use Discounts require a deliberate commitment against a baseline. If that baseline includes today’s overprovisioning, the commitment can preserve the cost of a configuration that should have been changed.

A practical sequence follows from the data: align workload requests with observed production use, make the node fleet fit those requirements, place interruption-tolerant work on Spot where appropriate, and improve GPU or ARM utilization for workloads that suit those options. Then evaluate commitments against the remaining steady demand, keeping burst capacity in view.

GKE can automate difficult infrastructure tasks, but workload owners still have to decide what their services need and how much interruption they can handle. Small, controlled changes backed by production observations give teams a way to improve utilization without making a risky fleet-wide adjustment. The clearest opportunity is to make each capacity decision reflect actual workload behavior before paying to reserve more of it.