Kubernetes 1.34: DRA Goes GA and the cgroup-driver Clock Starts

Kubernetes 1.34 is a consolidation release. The headline is that Dynamic Resource Allocation (DRA), the new way to request GPUs and other devices, finally has a stable API. Around it sits a batch of authentication features going GA, real memory relief for the API server on large reads, and a set of deprecations that start a countdown you need to plan for. This post covers what changed, what to adopt now, what to stop using, and how to upgrade without surprises.

Previous release: Kubernetes 1.33. Next: 1.35 (coming soon).

Scheduling: dynamic resources and shared pod budgets

Device scheduling is where 1.34 makes its biggest move. The built-in resources.requests model handles CPU and memory well, but GPUs, TPUs (Tensor Processing Units), and smart NICs (Network Interface Cards) need richer selection: pick a device by attributes, share it, or accept an alternative. The old device-plugin API could not express that. DRA has been maturing since 1.30, and 1.34 makes its core API stable.

DRA core API is GA GA

DRA (Dynamic Resource Allocation) is a way to request devices by describing what you need, rather than asking for a fixed plugin resource. In 1.34 the resource.k8s.io/v1 API graduated to stable and is available by default (release blog, KEP #4381).

Four API kinds carry the model, plus one Pod field:

• DeviceClass describes a category of device a cluster offers.
• ResourceSlice is how a driver advertises the devices on a node.
• ResourceClaim is a request for one or more devices.
• ResourceClaimTemplate stamps out a per-Pod claim.
• Pods reference claims through a new spec.resourceClaims field.

The flow, from a template to a running Pod, looks like this.

Copycopy code to clipboard
[ ResourceClaimTemplate ]        [ driver publishes ResourceSlices per node ]
          │                                     │
          ▼                                     │
 Pod created ──▶ controller makes a ResourceClaim
          │                                     │
          ▼                                     ▼
     scheduler matches claim against available devices in ResourceSlices
          │
          ▼
     claim Allocated ──▶ Pod bound to the node that has the device

Caption: how a DRA claim is allocated, from template to bound Pod.

The template below asks for one device from a class. Notice the doubly-nested spec.spec (the outer spec is the template, the inner spec is the claim it produces) and the exactly wrapper around deviceClassName. That exactly nesting is new in the v1 API and is a common thing to get wrong if you copy older v1beta1 examples.

Copycopy code to clipboard
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: example-resource-claim-template
spec:
  spec:
    devices:
      requests:
      - name: gpu-claim
        exactly:
          deviceClassName: example-device-class

The Pod references the template by name, and a container opts in through resources.claims. Only containers that list the claim can see the device.

Copycopy code to clipboard
spec:
  resourceClaims:
  - name: gpu
    resourceClaimTemplateName: example-resource-claim-template
  containers:
  - name: app
    image: ubuntu:24.04
    resources:
      claims:
      - name: gpu

The exact field paths (spec.spec.devices.requests[].exactly.deviceClassName, spec.resourceClaims[].resourceClaimTemplateName, resources.claims[].name) are from the allocate-devices-with-DRA task page.

Adopt now if you run GPU or device workloads and can install a DRA driver; standardize on v1. Wait if your device plugins work and you have no driver for DRA yet. One caveat: the concept docs describe the wider DRA feature as stable from 1.35, and several DRA sub-features (admin access, prioritized lists) are still beta in 1.34, so treat 1.34 as "core API stable, ecosystem still settling."

Pod-level resources Beta

You can now set requests and limits at the Pod level, sharing one budget across containers instead of dividing it by hand. It is beta in 1.34, and the Horizontal Pod Autoscaler (HPA) now understands pod-level resources (KEP #2837).

Copycopy code to clipboard
spec:
  resources:            # Pod-level budget, shared by all containers
    requests:
      cpu: "1"
      memory: 1Gi
    limits:
      cpu: "2"
      memory: 2Gi

Notice this sits at spec.resources, a sibling of containers, not inside a container.

Why DRA chose structured parameters

DRA's design choice worth understanding is "structured parameters." Earlier DRA drafts let a vendor driver make allocation decisions opaquely, which meant the scheduler could not reason about devices and the cluster autoscaler could not predict them. Structured parameters flipped that: drivers publish device inventory as ResourceSlice objects the scheduler understands, so allocation happens in the scheduler with full visibility (KEP #4381). The cost is that drivers must describe devices in the structured model rather than run arbitrary logic. That trade bought DRA its path to a stable, core API.

What is still unfinished: admin access and prioritized alternatives are beta, and device taints and partitionable devices remain alpha, so the DRA story continues past 1.34.

Authentication and authorization

Authentication and authorization got the cleanup operators have wanted for a while. Flag-driven auth config was hard to reason about, anonymous access was all-or-nothing, and authorizers could not see the selectors on a request. 1.34 graduates the fixes for all three.

Structured authentication configuration is GA GA

Configuring API server authentication through a pile of --oidc-* flags was brittle. The AuthenticationConfiguration file, added in 1.29, lets you define multiple JWT (JSON Web Token) authenticators, validate claims with CEL (Common Expression Language) expressions, and reload without a restart. It is GA in 1.34 (KEP #3331).

Point the API server at a config file instead of flags:

Copycopy code to clipboard
kube-apiserver --authentication-config=/etc/kubernetes/auth.yaml

Notice this replaces the --oidc-issuer-url family of flags and supports more than one identity provider at once, which the flags never did.

Adopt now if you run your own control plane and use OIDC. Managed platforms expose this differently.

Anonymous auth for named endpoints, and selector-based authorization GA

Two more auth features went stable. You can now allow anonymous access only on specific endpoints such as /healthz, /readyz, and /livez, instead of turning it on cluster-wide (KEP #4633). And authorizers can now decide based on the field and label selectors in a list, watch, or deletecollection request, which makes per-node least-privilege rules feasible (KEP #4601). Both selector authorization gates (AuthorizeWithSelectors, AuthorizeNodeWithSelectors) are locked on (changelog).

Adopt the anonymous-endpoint restriction now if any bootstrap or probe path relies on anonymous access; it removes a common RBAC (Role-Based Access Control) misconfiguration risk.

Projected ServiceAccount tokens for image pulls Beta

Pulling private images used to need long-lived pull Secrets on the node. The kubelet can now request short-lived, audience-bound ServiceAccount tokens and authorize the pull by the Pod's own identity (KEP #4412). It is beta in 1.34.

Adopt if your registry supports token-based auth; it removes a standing credential from your nodes.

Slimmer API server memory on large reads

The API server also gets real memory relief under load. A list of thousands of Pods used to be serialized into one large buffer and held until the whole response was sent. Several large reads at once could push the API server into an out-of-memory (OOM) kill. 1.34 makes the streaming path the default and adds a cache that can serve older reads without hitting etcd.

Streaming lists and a snapshottable cache GA Beta

This is the reliability win most large clusters will feel. Streaming encoding for list responses is GA: the API server encodes and sends items one at a time instead of buffering the whole collection, so a large read no longer pins gigabytes (KEP #5116).

Copycopy code to clipboard
Before: build whole list ──▶ [ one big buffer ] ──▶ send   (can OOM the API server)
After:  encode item ──▶ send ──▶ free ──▶ next item        (memory stays flat)

Caption: streaming list encoding keeps API server memory flat regardless of list size.

Alongside it, a snapshottable watch cache is beta and on by default (ListFromCacheSnapshot). The API server keeps snapshots of recent state and serves paginated or older-resourceVersion reads from them instead of hitting etcd, until etcd compacts or the cache fills with events older than 75 seconds (KEP #4988). Streaming informers (WatchList) are also on by default for the API server and controller manager (KEP #3157).

Adopt: nothing to do. These are on by default. If you run very large clusters, upgrade for this alone.

Linux node swap is GA GA

Kubernetes historically refused to use swap, so a node under memory pressure killed processes abruptly. Swap support is GA in 1.34 (KEP #2400). The useful mode is LimitedSwap, which lets Pods use swap within their existing memory limits. The kubelet default stays NoSwap, so nothing changes unless you opt in.

Copycopy code to clipboard
# kubelet configuration
memorySwap:
  swapBehavior: LimitedSwap

Notice the default is still NoSwap; you must set LimitedSwap and provision swap on the node to use it.

Adopt if you run memory-heavy workloads with large but cold footprints and can accept swap's latency cost. Wait for latency-sensitive services.

Admission and kubectl tooling

Two smaller changes round out the release, one for admission control and one for kubectl output.

Mutating admission policies Beta

MutatingAdmissionPolicy is a declarative, in-process alternative to mutating webhooks, using CEL and JSON Patch (KEP #3962). It is beta but, unlike most beta features, is off by default: enable the gate and serve the API with --runtime-config=admissionregistration.k8s.io/v1beta1=true (changelog).

Adopt if you maintain mutating webhooks for simple defaulting; moving them in-process removes a network hop and a failure mode.

KYAML, a safer kubectl output format Alpha

KYAML is a YAML subset designed to avoid YAML footguns like the "Norway bug" (NO parsed as false) and ambiguous quoting. Every KYAML file is valid YAML, so you can feed it to any kubectl. With 1.34's kubectl you can request it as output with KUBECTL_KYAML=true kubectl get -o kyaml ... (KEP #5295). Alpha, opt-in.

Try it on a test cluster

1. On a test cluster, install a DRA driver and run the ResourceClaimTemplate plus Pod example above; watch the claim move to Allocated.
2. Request KYAML output: KUBECTL_KYAML=true kubectl get deploy -o kyaml.
3. Turn on LimitedSwap on one node with swap provisioned and observe a memory-heavy Pod.
4. Set a pod-level spec.resources budget on a multi-container Pod and confirm scheduling.
5. Grep your Services for trafficDistribution: PreferClose and rename to PreferSameZone.

What breaks when you upgrade

This is the part to read carefully before you touch a production cluster. Some items were removed outright in 1.34, some are on a clock, and a few behaviours changed under you.

Removed in 1.34

Migration note for DRA: if any resourceclaims, resourceclaimtemplates, deviceclasses, or resourceslices were stored by a cluster older than 1.32, delete them before upgrading and recreate them afterward, per the changelog.

Deprecated now, gone later

To find nodes still on an at-risk container runtime, scrape the kubelet metric the release calls out:

Copycopy code to clipboard
kubelet_cri_losing_support

Notice a non-zero value flags nodes whose containerd version will lose support soon, which is your containerd-1.x migration list.

To find Services using the deprecated traffic distribution value:

Copycopy code to clipboard
kubectl get svc -A -o json \
  | jq -r '.items[] | select(.spec.trafficDistribution=="PreferClose") | "\(.metadata.namespace)/\(.metadata.name)"'

Notice this lists every Service you need to switch to PreferSameZone.

Behaviour changes to check first

These come from the changelog's Urgent Upgrade Notes and deprecation section. Read them before you roll a cluster.

• Metrics labels changed. Many API server and etcd metrics dropped or renamed labels: resource_prefix becomes group + resource on watch-cache list metrics; type becomes resource + group on etcd_request_*; watch-event metrics use resource instead of kind. Dashboards and alerts that group on the old labels will break. Update them before upgrading (changelog).
• Static pods that reference API objects are denied. The kubelet now rejects a static Pod that references API objects (Secrets, ConfigMaps, and so on), rather than letting it run after mirror-pod creation fails. Audit static Pod manifests on your nodes.
• Scheduler PreFilter signature changed. The scheduling framework now passes NodeInfo to PreFilter plugins. Out-of-tree scheduler plugins must be rebuilt against 1.34.
• DRA served at v1, v1beta1 still available. The v1 API is the default; older stored alpha types were removed (see migration note).
• Defaults now on: ListFromCacheSnapshot, WatchList, external ServiceAccount token signing, and DRAPrioritizedList. These are transparent but change internal behaviour; validate on a test cluster.

Version skew rules are unchanged: the kubelet may be up to three minor versions behind the API server, and you upgrade the control plane before nodes (release policy).

Doing the upgrade in order

Do these in order to move from 1.33 to 1.34.

Copycopy code to clipboard
1. scan for removed/deprecated APIs and flags   ──▶ fix
2. update dashboards/alerts for new metric labels ──▶ verify
3. upgrade control plane (API server, scheduler, controller-manager)
4. rebuild out-of-tree scheduler plugins for PreFilter change
5. upgrade nodes (kubelet, CRI); check cgroup-driver auto-detect
6. remove GA feature-gate overrides; validate; keep 1.33 control plane image for rollback

Caption: recommended 1.33 to 1.34 upgrade order.

Pre-flight specifics:

1. Remove --cloud-config and --register-schedulable from kubelet configs; they no longer parse.
2. Confirm your CRI reports its cgroup driver (containerd 1.7+ and CRI-O do); otherwise pin the kubelet cgroup driver for now and plan a runtime upgrade before 1.36.
3. Check any tool reading the changed metrics.
4. Rebuild custom scheduler plugins.
5. Rollback: keep the 1.33 control-plane images. Control-plane rollback within a minor is supported; node rollback follows the same skew rules.

Where 1.34 leaves you

Kubernetes 1.34 makes DRA's core API stable, hardens authentication, and cuts API server memory on large reads. The action items are small but real: drop two removed kubelet flags, fix metric-label dashboards, audit static Pods, and start planning your move off manual cgroup-driver config and containerd 1.x before 1.36. Read the changelog's Urgent Upgrade Notes in full before a production upgrade.

Next in this series: Kubernetes 1.35.

Further reading

Release and dates:

• Kubernetes v1.34: Of Wind & Will
• Releases: 1.34
• v1.34.0 GitHub release
• CHANGELOG-1.34

Features:

• DRA: KEP #4381, allocate devices with DRA
• Structured auth config: KEP #3331
• Selector authorization: KEP #4601; anonymous endpoints: KEP #4633
• Streaming lists: KEP #5116; snapshottable cache: KEP #4988; WatchList: KEP #3157
• Linux swap: KEP #2400
• Projected image-pull tokens: KEP #4412
• Pod-level resources: KEP #2837
• MutatingAdmissionPolicy: KEP #3962
• KYAML: KEP #5295

Deprecations:

• cgroup driver from CRI: KEP #4033
• Traffic distribution: KEP #3015
• kuberc: KEP #3104