DRA vs Device Plugins: When to Migrate and What You Gain

If you run GPUs on Kubernetes, you have two ways to expose them: the device plugin API, which has been around for years, and Dynamic Resource Allocation (DRA), which reached a stable core API in 1.34. This post compares them honestly, feature by feature, and helps you decide whether to migrate now, later, or not yet. It uses the same terminology as the companion post on writing a DRA driver.

The short version: device plugins expose a device as a counted integer (nvidia.com/gpu: 2), so the scheduler only counts and cannot reason about which GPU. DRA exposes devices with attributes, and a claim selects them with CEL, so the scheduler allocates specific devices with full visibility. That is what lets DRA do what device plugins never could: sharing between containers and pods, attribute and topology-aware selection, and network-attached devices. If interchangeable whole-GPU allocation is all you need, device plugins still work and are simpler, so migrate only when you need those extras. Several DRA extensions matured between 1.34 and 1.36, and the table further down gives each one's KEP and stage per version so you do not adopt something still alpha.

This post targets Kubernetes 1.36. DRA uses the resource.k8s.io/v1 API (GA since 1.34, gate locked on since 1.35); the device plugin API is v1beta1 and unchanged (DRA concepts, device plugins). Extensions that are still moving toward GA after 1.36 are noted as beta; check the version you run before relying on them.

Where the device plugin model runs out

The device plugin API models a device as an extended resource, which is an integer counted per node. A plugin registers a name like nvidia.com/gpu and reports "this node has 8." A pod asks for nvidia.com/gpu: 2, and the scheduler finds a node with 2 free. The docs are explicit about the limits: extended resources are integer-only and cannot be overcommitted, and devices cannot be shared between containers (device plugins).

That model cannot express the things modern accelerator scheduling needs. You cannot ask for "a GPU with at least 40 GB," or "two GPUs on the same NUMA node," or "a slice of a GPU shared with another pod." Topology exists only at the node level through the kubelet's Topology Manager; the cluster scheduler never sees it. DRA was built to close exactly this gap.

What changes when the scheduler can read attributes

The two APIs work differently at every layer. Device plugins register a socket with the kubelet and stream a device count; the scheduler counts integers. DRA drivers publish devices as ResourceSlices to the API server; the scheduler matches claims against device attributes.

The device plugin gRPC service is ListAndWatch plus Allocate (and optional GetPreferredAllocation, PreStartContainer), registered at /var/lib/kubelet/device-plugins/kubelet.sock. DRA drivers instead implement NodePrepareResources and publish ResourceSlices, as covered in the DRA driver post.

Do you actually need to move

Copycopy code to clipboard
Do you need attribute selection, sharing, topology, or network devices?
        │
   no ──┴── yes
   │          │
   │          Is a DRA driver available for your hardware on 1.34+?
   │          │
   │     no ──┴── yes ──▶ MIGRATE NOW
   │     │
   │     └──▶ MIGRATE LATER (track the driver; pilot on a test cluster)
   │
   └──▶ STAY on device plugins (interchangeable whole-device counting is enough)

Caption: decide by whether you need DRA's capabilities and whether a driver exists yet.

The honest default: if whole-GPU, interchangeable allocation covers your workloads, device plugins are simpler and battle-tested, so stay. Move to DRA when a real requirement appears: fractional or shared GPUs, attribute-based selection, multi-device topology, or network-attached accelerators.

Switching without a big bang

You do not have to switch everything at once. Device plugins and a DRA driver can run on the same cluster, and even the same node, because they use different request mechanisms. To see the difference concretely, here is the same intent expressed both ways. First, the device-plugin request: an integer count of a named resource.

Copycopy code to clipboard
# Device plugin: ask for two interchangeable GPUs.
spec:
  containers:
    - name: trainer
      resources:
        limits:
          nvidia.com/gpu: 2

Notice there is no way to say which two GPUs; the scheduler only checks that a node reports two free.

The DRA equivalent expresses the same need through a ResourceClaimTemplate the pod references, which is where attribute selection would go if you needed it.

Copycopy code to clipboard
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata: { name: two-gpus }
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: gpu.example.com
            count: 2

Notice the request names a DeviceClass and a count; add a CEL selector here to require specific attributes, which the device-plugin form cannot express.

The migration shortcut: extended resources backed by DRA

Rewriting every workload to ResourceClaims is the slow path. The extended-resource bridge (KEP-5004, beta in 1.36) lets a DRA driver satisfy a classic resources.limits request, so existing manifests keep working while a DRA driver takes over the device underneath. A pod can use the implicit name derived from the DeviceClass:

Copycopy code to clipboard
spec:
  containers:
    - name: trainer
      resources:
        limits:
          deviceclass.resource.kubernetes.io/gpu.example.com: 1

Notice the resource name is deviceclass.resource.kubernetes.io/<DeviceClassName>; the scheduler allocates a matching device from the DRA driver, but the pod spec looks like a plain extended-resource request.

To keep the exact resource name your manifests already use, set extendedResourceName on the DeviceClass so the old name maps to DRA.

Copycopy code to clipboard
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata: { name: gpu.example.com }
spec:
  extendedResourceName: example.com/gpu   # pods can keep requesting example.com/gpu: 1

Notice this is the least-disruptive migration: workloads keep their resources.limits: example.com/gpu, and you swap the device plugin for a DRA driver behind that name. It is beta on 1.36, so validate before relying on it.

The safe path is still per workload. Move one non-critical workload, then watch two things: that the DRA allocation actually happens (a stuck request usually means no device matches), and that you are not double-counting a device across both systems. A node can advertise nvidia.com/gpu through a device plugin and publish ResourceSlices through a DRA driver at once, as long as they do not both own the same physical device.

Which extensions are ready on which version

DRA's core API is GA, but the interesting extensions matured across releases. Adopting one that is still alpha means it can be disabled or changed. This table gives each extension's KEP and its stage per version, so you can match adoption to the cluster you run.

Sources: 1.34 release, 1.36 release.

Three of these matter most in practice. Admin access (GA in 1.36) lets authorized users inspect devices already in use, for monitoring, which is how you build a GPU dashboard without holding a device away from workloads. Prioritized lists (GA in 1.36) let a claim accept alternatives in order, so a job that prefers one big GPU can fall back to two smaller ones instead of pending forever. Device taints and tolerations (beta in 1.36) let an operator mark a device unusable so the scheduler drains it, the same pattern as node taints; watch it graduate before you depend on it in production. The extended-resource-via-DRA bridge, which lets a DRA driver satisfy a classic resources.limits request, is also progressing and will smooth migration once it stabilizes.

Traps to avoid and edges still forming

• Double-owned devices. If both a device plugin and a DRA driver manage the same physical GPU, you can allocate it twice. Assign each device to exactly one system.
• Claim stuck pending. No device matches the claim's selector. Check kubectl get resourceslice -o yaml against the CEL expression.
• Adopting an immature extension. Partitionable devices and consumable capacity are beta on 1.36, and device taints only just reached beta; do not build production workflows on an extension that is alpha or freshly beta on your version. Use the table and confirm the stage for the exact release you run.
• Assuming topology works like device plugins. With device plugins, NUMA affinity is a node-level Topology Manager decision. With DRA, topology is a scheduler-level attribute; the mental model is different, and mixing them causes confusion.

The roadmap tells you which of these gotchas will fade. Partitionable devices (KEP-4815) and consumable capacity (KEP-5075) are the frontier: they are what make fine-grained GPU sharing and fractional allocation fully first-class, and both are beta as of 1.36. If your reason to migrate is sharing, pilot these on a test cluster and watch their graduation across the next few releases. The gang-scheduling companion post covers how a gang competes for DRA devices in multi-device AI jobs.

A hands-on checklist for a test cluster

1. On a DRA-enabled test cluster, run both a device plugin and a DRA driver, and confirm each owns different devices.
2. Move one workload from resources.limits: nvidia.com/gpu to a ResourceClaim and compare scheduling.
3. Taint a device with the device-taints feature and watch the scheduler avoid it.
4. Write a prioritized-list claim (firstAvailable) that prefers one large GPU and falls back to two small ones.
5. Check your cluster version against the extension table before enabling anything beyond the core.

The verdict

The choice stops being hard once you name your requirement. Device plugins count interchangeable devices; DRA allocates specific devices by attribute, with sharing and topology the older API never had. Stay on device plugins when interchangeable whole-GPU counting is enough. Move to DRA when you need those capabilities and a driver exists, workload by workload, and track the extension table so you do not lean on something still alpha. Companion posts cover JobSet for distributed training and gang scheduling for all-or-nothing placement.

Further reading

• Device plugins
• DRA concepts
• DRA API objects
• Allocate devices with DRA
• Extension KEPs: admin access 5018, prioritized list 4816, device taints 5055, partitionable devices 4815, consumable capacity 5075, extended resource 5004
• Writing a DRA driver
• Release blogs: 1.34, 1.36