Kube-scheduler Internals: Writing a Filter and Score Plugin

18.02.2026 — kubernetes, scheduler, scheduling-framework, go, platform-engineering — 9 min read

The default scheduler is good, but sometimes you need placement logic it does not have: prefer nodes with a warm cache, avoid nodes running a competing workload, or score by a custom metric. The scheduling framework is how you add that without forking the scheduler. This post explains how the scheduler works, then builds a real plugin with Filter and Score, wires it into a config profile, and runs it as a second scheduler. The code is source-verified against the framework interfaces and not executed here.

This is part of a scheduling series. Companion posts cover JobSet and Kueue for distributed training, and gang scheduling, which is itself built from scheduler plugins.

Placement is policy. The default scheduler encodes good general policy: spread pods, respect taints, balance resources. But your cluster may have a rule the defaults cannot express. A stateful service should prefer the node that already has its data cached. A latency-sensitive pod should avoid nodes running a batch job. A GPU workload should score nodes by a metric only your platform knows.

You could run a mutating webhook to force nodeName, but that throws away everything the scheduler does well. The scheduling framework is the supported extension mechanism: you add your logic at a specific point in the pipeline and inherit all the default behaviour around it (framework KEP-624).

This post targets Kubernetes 1.35. Plugin interfaces come from k8s.io/kube-scheduler/framework (package fwk), the out-of-tree registry from k8s.io/kubernetes/cmd/kube-scheduler/app, and config uses kubescheduler.config.k8s.io/v1 (scheduling framework, scheduler configuration). The framework interfaces are versioned with the scheduler, so build the plugin against the same minor version as your cluster; the signatures shown here are the current shape and carry across recent releases.

The rest of this post follows one pod through a single scheduling cycle, stopping at each extension point to show where Filter and Score fit, how scoring combines with the default plugins, how to test the plugin, and what breaks once it is running.

Two cycles per pod

The scheduler processes one pod at a time through two phases.

Scheduling cycle: choose a node for the pod. Runs synchronously, one pod at a time.
Binding cycle: bind the pod to the chosen node. Can run concurrently for different pods.
Extension point: a named stage where plugins run, such as Filter or Score.
Plugin: code registered at one or more extension points.
CycleState: per-pod scratch space plugins use to pass data between extension points.
Status: a plugin's result, with a code like Success, Unschedulable, or Error.

scheduling cycle:  PreFilter ─▶ Filter ─▶ PostFilter ─▶ PreScore ─▶ Score ─▶ Reserve ─▶ Permit
binding cycle:                                                            PreBind ─▶ Bind ─▶ PostBind

Caption: a pod flows through the scheduling cycle to pick a node, then the binding cycle to commit it.

The extension points in the order they fire

The extension points run in a fixed order. Understanding it tells you where your logic belongs.

PreFilter   compute once, or reject the pod early
   │
Filter      drop nodes that cannot run the pod (per node)
   │
PostFilter  runs only if no node survived (preemption lives here)
   │
PreScore    prep for scoring
   │
Score       rank each surviving node 0..100 (per node)
   │
Reserve     tentatively claim resources
   │
Permit      allow, deny, or wait (gang scheduling uses this)

Caption: the scheduling-cycle extension points, in order.

Filter and Score are where most custom logic goes. Filter answers "can this node run the pod at all," and any non-Success status excludes the node. Score answers "how good is this node," returning an integer that the framework normalizes and weights. PostFilter runs only when Filter left no feasible node, which is where preemption plugins try to make room.

Writing the plugin and wiring it in

The plugin below is a Go type implementing the Name, Filter, and Score methods from the framework. It is illustrative and source-verified against the interfaces, not executed here.

Start with the type and name. A plugin is any type the framework can register; Name returns the string used in config.

package cachelocality

import (
    "context"
    v1 "k8s.io/api/core/v1"
    fwk "k8s.io/kube-scheduler/framework"
)

const Name = "CacheLocality"

type CacheLocality struct{ handle fwk.Handle }

func (p *CacheLocality) Name() string { return Name }

Notice the plugin holds a fwk.Handle, the framework's accessor for shared state like the informer lister. You use it to read cluster state during Filter and Score.

Filter excludes nodes. Here it rejects nodes labelled as drained. Returning a non-success Status removes the node from consideration.

func (p *CacheLocality) Filter(ctx context.Context, state fwk.CycleState,
    pod *v1.Pod, nodeInfo fwk.NodeInfo) *fwk.Status {
    if nodeInfo.Node().Labels["cache/drained"] == "true" {
        return fwk.NewStatus(fwk.Unschedulable, "node is draining its cache")
    }
    return nil // nil means Success
}

Notice nil means the node passes. A Status with code Unschedulable drops the node but lets the pod retry later; UnschedulableAndUnresolvable means do not bother retrying on cluster changes.

Score ranks the survivors. Here a node with the pod's data cached scores higher. Score returns an int64; the framework normalizes it to the 0 to 100 range.

func (p *CacheLocality) Score(ctx context.Context, state fwk.CycleState,
    pod *v1.Pod, nodeInfo fwk.NodeInfo) (int64, *fwk.Status) {
    if nodeInfo.Node().Labels["cache/warm-for"] == pod.Labels["dataset"] {
        return 100, nil
    }
    return 0, nil
}

Notice the raw score is arbitrary; you do not have to return 0 to 100. If your range differs, implement ScoreExtensions().NormalizeScore to rescale before the framework applies the plugin's weight.

Registering and running the plugin

An out-of-tree scheduler is a small main that registers your plugin with the standard scheduler command. app.WithPlugin adds your factory to the registry; everything else is the normal scheduler.

package main

import (
    "os"
    "k8s.io/kubernetes/cmd/kube-scheduler/app"
    "example.com/sched/cachelocality"
)

func main() {
    command := app.NewSchedulerCommand(
        app.WithPlugin(cachelocality.Name, cachelocality.New),
    )
    if err := command.Execute(); err != nil {
        os.Exit(1)
    }
}

Notice you reuse app.NewSchedulerCommand, so your binary is a full scheduler with your plugin added, not a reimplementation. New is a factory function the framework calls to construct the plugin with its handle.

Enabling it in a profile

A KubeSchedulerConfiguration profile turns the plugin on and names the scheduler. Only pods that ask for this schedulerName use this profile.

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: cache-aware-scheduler
    plugins:
      filter:
        enabled: [{ name: CacheLocality }]
      score:
        enabled: [{ name: CacheLocality, weight: 5 }]

Notice weight multiplies this plugin's normalized score before it is summed with the other Score plugins. A higher weight makes cache locality matter more relative to the defaults, which still run.

Running it as a second scheduler

Deploy the binary as a second scheduler alongside the default one. It needs the scheduler RBAC and its own leader-election lease so it does not fight the default scheduler.

spec:
  containers:
    - name: scheduler
      image: my-scheduler:1.0
      args: ["--config=/etc/kubernetes/scheduler-config.yaml"]

Notice the second scheduler runs independently; the default scheduler keeps scheduling every pod that does not opt in. Grant it the system:kube-scheduler ClusterRole and a distinct leader-election lease name.

Finally, a pod opts in by name.

spec:
  schedulerName: cache-aware-scheduler

Notice only pods with this schedulerName are handled by your scheduler. Everything else stays on the default, so a plugin bug cannot take down cluster-wide scheduling.

How your score competes with the defaults

Your Score plugin does not run alone. The default Score plugins (resource balance, spread, affinity) run too, each normalized to 0 to 100 and multiplied by its weight, then summed. The node with the highest total wins. So your plugin does not decide placement outright; it nudges the ranking by its weight. Set the weight to reflect how much your signal should matter against the defaults. If your Score returns values outside 0 to 100, implement NormalizeScore so the weighting is fair.

Testing filter and score

Test Filter and Score as plain functions where you can, passing a constructed NodeInfo and pod and asserting the returned Status and score. For integration, the scheduler framework provides test harnesses that run a real framework with your plugin registered against a fake client, so you can assert end-to-end placement. Keep unit tests on the scoring logic and reserve the heavier harness for the wiring.

How a plugin can break scheduling

A plugin panic takes down the cycle. An unrecovered panic in Filter or Score crashes the scheduler. Guard against nil fields on NodeInfo and pod, and test edge cases.
A slow Filter blocks everything. The scheduling cycle is synchronous and one pod at a time. A Filter that makes a network call per node stalls all scheduling. Do slow work in PreFilter once, cache it in CycleState.
Pods stuck Pending with no explanation. If Filter excludes every node and no PostFilter plugin can preempt, the pod stays Pending. Surface a clear Unschedulable message so kubectl describe pod explains why.
Profile misconfiguration. A typo in the plugin name or a wrong apiVersion makes the scheduler ignore your plugin silently or fail to start. Check the scheduler logs on startup.
Version skew. The framework interfaces change across releases. Build your plugin against the same Kubernetes version as the cluster, or the plugin factory signature will not match.

The cost of owning a plugin

The framework itself is stable, but its Go interfaces moved to the k8s.io/kube-scheduler/framework module recently, so older tutorials import the wrong package. The main practical friction remains version skew: an out-of-tree plugin is tied to a Kubernetes version and you rebuild it on each upgrade, because the framework interfaces and the plugin factory signature can shift between minor releases. This is the real operational cost of a custom plugin: it is a scheduler you now own and must re-qualify on every cluster upgrade. For common needs like gang scheduling, prefer an existing plugin (scheduler-plugins coscheduling) over writing your own; a companion post covers those.

Run it yourself in kind

Build the plugin above against your cluster's Kubernetes version and run it as a second scheduler in kind.
Label some nodes cache/drained=true and confirm your Filter excludes them.
Label a node cache/warm-for=<dataset> and confirm scored placement prefers it.
Change the Score weight and watch placement shift relative to the default plugins.
Add a deliberately slow call in Filter and observe scheduling throughput drop, then move it to PreFilter.

From one plugin to gang scheduling

The scheduling framework lets you add placement logic at a precise point without forking the scheduler. Implement Filter to exclude nodes and Score to rank them, register with app.WithPlugin, enable it in a profile, and run it as a second scheduler that only opts-in pods use. Keep Filter fast and messages clear. A companion post applies all of this to gang scheduling, which is a set of scheduler plugins built on the same extension points.