Kubernetes v1.37: Advancing Workload-Aware Scheduling

[kasenur]

Kubernetes v1.37: Advancing Workload-Aware SchedulingBy Antoni Zawodny (Google), Bartosz Rejman (Google), Heba Elayoty (Microsoft), Jon Huhn (Microsoft), Maciej Skoczeń (Google), Maciej Wyrzuc (Google), Matt Matejczyk (Google) |
Tuesday, September 08, 2026AI/ML and complex batch workloads continue to push the boundaries of Kubernetes scheduling. Following the foundational workload-centric enhancements introduced in previous releases, Kubernetes v1.37 delivers the next major milestone in the Workload-Aware Scheduling (WAS) journey. In this release, the core Workload and PodGroup APIs—enabling gang scheduling—along with Workload-Aware Preemption (WAP) and shared DRA ResourceClaims for PodGroups, all graduate to Beta, solidifying their role in the Kubernetes ecosystem.To address the hierarchical scheduling requirements of modern high-performance distributed workloads, v1.37 introduces the new CompositePodGroup API.
This new API allows expressing multi-level topology constraints, gang scheduling, and preemption policies for complex, heterogeneous groups of Pods. Crucially, this architectural expansion unlocks native scheduling support for advanced workload structures commonly managed by higher-order extension APIs such as JobSet and LeaderWorkerSet (LWS).Alongside these API additions, v1.37 focuses on streamlining adoption by introducing a new set of controller integration APIs and the workloadbuilder Go library. These provide standardized building blocks that significantly simplify how out-of-tree controllers can integrate with WAS capabilities.
Utilizing these new tools, the native Job controller integration has been upgraded to fully consume the expanded WAS APIs—enabling advanced scheduling policies, flexible disruption modes, and topology-aware scheduling for standard batch workloads.Gang scheduling and Workload / PodGroup APIsKubernetes v1.37 delivers a major milestone: Workload / PodGroup APIs and gang scheduling
are officially graduating to Beta. This graduation signals that native, "all-or-nothing"
scheduling for workloads is solidifying for wider adoption.Key updates to the API and gang scheduling algorithm in this release include:Beta graduation and API versioning changesThe core Workload and PodGroup APIs have been promoted to v1beta1, meaning they are now one step away from
General Availability (GA). For early adopters who have been testing these features,
take note of the alpha versioning transition: v1alpha2 has been entirely replaced by v1alpha3.
This transition introduces breaking changes designed to clean up the API structure around disruptionMode.Native PodGroup queueingA significant under-the-hood improvement in v1.37 makes the PodGroup a first-class citizen in the scheduling queue.
Previously, even if belonging to a PodGroup, all member Pods were queued individually. Now, only the top-level PodGroup
object is queued. This ensures all Pods share the same queueing behavior and lays the groundwork
for more advanced PodGroup queueing strategies in the future.Dynamic elasticity with minCount mutabilityIn earlier iterations, the minCount field, which dictates the minimum number of Pods required
to successfully schedule a PodGroup, was strictly immutable. In v1.37, minCount is now mutable.
This API change unlocks flexibility for elastic workloads. Controllers can now dynamically
adjust the minimum required size of a gang on the fly, allowing workloads to gracefully degrade
or expand without interrupting already-scheduled Pods.Workload-aware preemptionIn Kubernetes v1.37 the separate
WorkloadAwarePreemption feature gate
for workload-aware preemption was merged into the GenericWorkload feature gate,
becoming a core part of the gang scheduling effort.While the core concepts of workload-aware preemption stay the same, there are some
differences between the v1.36 and v1.37 releases😛erformance and optimalityTo check whether a preemptor can fit in the cluster thanks to preemption, the scheduler
simulates the removal of all potential victims and re-runs the scheduling algorithm. After
that it tries to reprieve as many victims as possible. In the v1.36 release, the scheduling
algorithm was run for each victim reprieval, verifying whether with the victim reprieved, the algorithm
can still find a valid placement for the preemptor. In v1.37, the scheduling algorithm is run only once
and the preemptor Pods are assumed based on its output. Later, the reprieval checks
whether a victim can still run in its place with the preemptor assumed.PodGroup as a victimOne of the limitations of v1.36 was the fact that the default preemption for single Pods
was not aware of PodGroups and was not respecting their disruptionMode fields, allowing
for disruption of single Pods even when the PodGroup had disruptionMode: {all: {}} set.
Kubernetes v1.37 removes this limitation; the default preemption now respects the PodGroup
disruptionMode field.Rename of the disruptionMode fieldsDuring the promotion of the API to Beta, the disruptionMode field was changed to decouple
its naming from the PodGroup object, allowing consistent naming across PodGroups and CompositePodGroups.
The modes changed as follows: PodGroup became all, and Pod became single.Support for preemptionPolicyIn v1.36, the PodGroup does not have a preemptionPolicy field. The PodGroup can perform
preemption as long as none of the Pods forming it has preemptionPolicy: Never set. In v1.37,
when the PodGroupPreemptionPolicy
feature gate is enabled, a PodGroup also has a preemptionPolicy field. It serves as an authoritative field for whether
a PodGroup can perform preemption.CompositePodGroup APIIn Kubernetes v1.36, workload-aware scheduling established a clean separation between static workload templates (Workload) and runtime group state (PodGroup), but the supported scheduling policies were limited to a single, flat group. The CompositePodGroup API, introduced in Kubernetes v1.37, extends this model to support hierarchical scheduling requirements.This