Karpenter (Kubernetes Autoscaling) Cheat Sheet
Karpenter node autoprovisioning on Kubernetes covering NodePools, NodeClasses, consolidation, and disruption settings.
NodePool Definition
A NodePool defines constraints Karpenter uses when provisioning nodes.
apiVersion: karpenter.sh/v1kind: NodePoolmetadata: name: defaultspec: template: spec: requirements: - key: kubernetes.io/arch operator: In values: ["amd64"] - key: karpenter.sh/capacity-type operator: In values: ["spot", "on-demand"] - key: karpenter.k8s.aws/instance-category operator: In values: ["c", "m", "r"] nodeClassRef: group: karpenter.k8s.aws kind: EC2NodeClass name: default expireAfter: 720h limits: cpu: 1000 memory: 1000Gi disruption: consolidationPolicy: WhenEmptyOrUnderutilized consolidateAfter: 1m
EC2NodeClass (AWS)
Cloud-provider-specific node configuration referenced by a NodePool.
apiVersion: karpenter.k8s.aws/v1kind: EC2NodeClassmetadata: name: defaultspec: amiFamily: AL2023 role: KarpenterNodeRole-my-cluster subnetSelectorTerms: - tags: karpenter.sh/discovery: my-cluster securityGroupSelectorTerms: - tags: karpenter.sh/discovery: my-cluster blockDeviceMappings: - deviceName: /dev/xvda ebs: volumeSize: 50Gi volumeType: gp3 encrypted: true
Debugging & Common kubectl Commands
Inspect Karpenter's decisions and node lifecycle.
# Watch Karpenter controller logskubectl logs -f -n kube-system -l app.kubernetes.io/name=karpenter# List nodes with Karpenter-managed labelskubectl get nodes -L karpenter.sh/nodepool,karpenter.sh/capacity-type# See NodeClaims (Karpenter's abstraction over provisioned instances)kubectl get nodeclaims# Force drain/consolidate a nodekubectl delete node <node-name># Check why a pod is unschedulable (feeds Karpenter's decision)kubectl describe pod <pod-name> | grep -A5 Events
Disruption & Consolidation Settings
Fields controlling when Karpenter removes or replaces nodes.
- consolidationPolicy: WhenEmptyOrUnderutilized- allows Karpenter to consolidate nodes that are empty or running below capacity
- consolidationPolicy: WhenEmpty- only removes fully empty nodes, safer but less cost-efficient
- consolidateAfter- cooldown before Karpenter acts on an underutilized node
- expireAfter- forces node replacement after a max age, useful for AMI rotation
- terminationGracePeriod- max time Karpenter waits for pods to drain before force-terminating a node
- budgets- limits how many nodes can be disrupted concurrently or on a schedule
Weighted NodePools with Taints
Multiple NodePools let Karpenter prefer cheaper capacity first and fall back, using taints to reserve specialized nodes.
apiVersion: karpenter.sh/v1kind: NodePoolmetadata: name: spot-preferredspec: weight: 50 template: spec: requirements: - key: karpenter.sh/capacity-type operator: In values: ["spot"] nodeClassRef: { group: karpenter.k8s.aws, kind: EC2NodeClass, name: default }---apiVersion: karpenter.sh/v1kind: NodePoolmetadata: name: gpu-dedicatedspec: weight: 10 template: spec: taints: - key: nvidia.com/gpu value: "true" effect: NoSchedule requirements: - key: karpenter.k8s.aws/instance-family operator: In values: ["g5"] nodeClassRef: { group: karpenter.k8s.aws, kind: EC2NodeClass, name: gpu }
Spot Interruption Handling (SQS)
Karpenter drains spot nodes proactively by watching an SQS queue fed by EventBridge rules for interruption/rebalance events.
# Terraform sketch: EventBridge -> SQS -> Karpenter interruption-queueresource "aws_sqs_queue" "karpenter" { name = "karpenter-interruption" message_retention_seconds = 300}resource "aws_cloudwatch_event_rule" "spot_interruption" { event_pattern = jsonencode({ source = ["aws.ec2"] detail-type = ["EC2 Spot Instance Interruption Warning"] })}# Helm value: point the controller at the queue name# settings.interruptionQueue: karpenter-interruption
Drift Detection & Scheduled Disruption Budgets
Karpenter detects when live nodes no longer match their NodePool/NodeClass spec and replaces them within budget-limited windows.
spec: disruption: consolidationPolicy: WhenEmptyOrUnderutilized consolidateAfter: 5m budgets: # never disrupt more than 20% of nodes at once - nodes: "20%" # freeze all disruption during business-hours deploy window - nodes: "0" schedule: "0 9 * * mon-fri" duration: 8h# Drift is automatic: editing EC2NodeClass amiFamily or NodePool# requirements marks existing NodeClaims Drifted=True; Karpenter# cordons + replaces them respecting the budgets above.
Per-NodePool kubelet Configuration
Override kubelet behavior (pods-per-node, eviction thresholds) without touching the base AMI.
apiVersion: karpenter.sh/v1kind: NodePoolspec: template: spec: kubelet: maxPods: 110 podsPerCore: 2 systemReserved: cpu: 200m memory: 500Mi evictionHard: memory.available: 5% nodefs.available: 10%
Karpenter vs. Cluster Autoscaler
Key architectural differences to know when migrating from CA-managed node groups.
- No ASG/node group required- Karpenter launches EC2 instances directly via fleet API calls, bypassing Auto Scaling Groups entirely
- Just-in-time bin packing- picks the cheapest instance type/AZ combo that fits pending pods, rather than scaling a fixed-shape group
- NodeClaim abstraction- an internal CRD representing one provisioned instance, replaces CA's node group scaling decisions
- Consolidation vs scale-down- Karpenter can actively repack workloads onto fewer/cheaper nodes, not just remove idle ones
- Multi-instance-type flexibility- a single NodePool can span dozens of instance types/sizes for better spot availability
- Drift reconciliation- has no CA equivalent; CA never replaces nodes just because their spec changed
Set `karpenter.sh/do-not-disrupt: "true"` as a pod annotation on stateful or batch jobs you never want evicted mid-run — Karpenter's consolidation will otherwise happily terminate underutilized nodes running them.