Workload-Aware Scheduling in Kubernetes 1.37: The Building Blocks ArchitectureArticle · digest for October 19
Modular diagram of Kubernetes 1.37 scheduler building blocks with group pod placement
Article2 sections≈ 4 min

Workload-Aware Scheduling in Kubernetes 1.37: The Building Blocks Architecture

Anyone running Kubernetes for machine learning (ML/LLM), distributed analytics, or batch processing knows the default scheduler's biggest blind spot: it thinks in individual pods. It has no idea that your training job needs exactly 16 GPUs across four pods simultaneously—and that if only three pods' worth of capacity is free, launching them is pointless. They'll just sit idle, burning budget and blocking nodes for other queues.

This pattern is called gang scheduling (all-or-nothing group scheduling). Until recently, the ecosystem looked like a Tower of Babel: Kube-batch, Volcano, custom Kubeflow plugins, JobSet, and proprietary cloud-provider extensions. Every operator invented its own abstractions, wired custom hooks into kube-scheduler, and duplicated thousands of lines of code.

In Kubernetes 1.37, the Workload-Aware Scheduling (WAS) working group under SIG Scheduling—driven largely by research and architecture from Heba Elayoty—finally delivered a standardized integration mechanism. Instead of rewriting the scheduler into a monolith, Kubernetes introduced a Building Blocks paradigm.

Four Building Blocks of Gang Scheduling

The WAS architecture is split into two layers: the scheduler core (where Workload and PodGroup manifests graduate to Beta in scheduling.k8s.io/v1beta1) and typed Go structs for operator authors (under scheduling.k8s.io/v1alpha3).

Instead of manually assembling low-level core objects, a custom CRD author simply embeds ready-made leaf building blocks (WorkloadPodGroup...) into their spec:

  1. Scheduling Policy. Defines the core mode: either single-pod scheduling (basic) or group quorum (gang) with a minimum pod count minCount. Better yet, minCount is now mutable—you can resize a group dynamically without recreating the object.
  2. Scheduling Constraints. Topology co-location rules. You can strictly require that every pod in the group lands on the same InfiniBand network switch or within the same availability zone.
  3. Disruption Mode. Controls eviction behavior: evict pods individually (single) or tear down the entire group together (all). Naturally, the scheduler forbids all when using the basic policy.
  4. Resource Claims. Dynamic Resource Allocation (DRA) support, letting you request GPUs at the level of the entire task group.

For complex hierarchical systems—like the JobSet operator managing a collection of Jobs, or LeaderWorkerSet—the API provides composite WorkloadCompositePodGroup blocks with a minGroupCount parameter.

How a Controller Compiles the Manifests

Here is how this looks in practice when building your own ML operator. First, you embed the building blocks into your resource's Go spec:

package v1alpha1

import (
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    schedulingv1alpha3 "k8s.io/api/scheduling/v1alpha3"
)

// TrainJobSpec describes a distributed training job
type TrainJobSpec struct {
    Replicas    int32                                               `json:"replicas"`
    Scheduling  *schedulingv1alpha3.WorkloadPodGroupSchedulingPolicy `json:"scheduling,omitempty"`
    Constraints *schedulingv1alpha3.WorkloadPodGroupConstraints     `json:"constraints,omitempty"`
}

Then, inside the controller's reconcile loop, you bring in the official shared library k8s.io/component-helpers/scheduling/workloadbuilder. It validates the user's intent and automatically compiles the task tree into core scheduler objects:

package controller

import (
    "context"
    schedulingv1beta1 "k8s.io/api/scheduling/v1beta1"
    "k8s.io/component-helpers/scheduling/workloadbuilder"
)

func (r *TrainJobReconciler) buildWorkload(ctx context.Context, job *TrainJob) (*schedulingv1beta1.Workload, error) {
    // Build the leaf node of the pipeline
    item := workloadbuilder.NewLeafItem("worker").
        WithSchedulingPolicy(job.Spec.Scheduling).
        WithConstraints(job.Spec.Constraints).
        WithDefaultPolicy(schedulingv1alpha3.BasicPolicy)

    // Initialize the builder: it maps validation errors back to the source CRD fields
    builder := workloadbuilder.NewBuilder(item, job.Name, job.Namespace, job)
    if errList := builder.Validate(); len(errList) > 0 {
        return nil, errList.ToAggregate()
    }

    // Compile into the final core v1beta1 Workload manifest
    return builder.BuildWorkload()
}

Here lies a huge practical win: if a user makes a mistake (say, setting disruptionMode: all on a basic policy), the builder returns an error pointing to the exact path in the user's manifest (spec.scheduling.disruptionMode) instead of complaining about an internal compiled Workload. Combined with how Server-Side Apply in Kubernetes handles declarative field management and ownership, keeping CRD fields and internal scheduler objects in sync becomes crystal clear.

Drawing the Line with Kueue

Don't confuse WAS with job queueing: Kueue is still responsible for global quotas, team budgets, and cluster admission, while Workload-Aware Scheduling handles atomic node allocation once a group is admitted.

All in all, Kubernetes 1.37 gives platform engineers a clean foundation: no more reinventing the wheel just to teach a cluster how to schedule large AI/ML workloads properly.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 3