Notes · August 2026 · Kubernetes, Node.js

half idle, fully busy.

A Node service stops coping somewhere around 300 requests a second. The slowest 1% of requests, p99, go from 50ms to anywhere between half a second and three. The process is doing every bit of work it will ever do. The autoscaler, watching CPU, reads 49% and does nothing.

The symptom

It looks like a broken metrics pipeline the first time you see it. Latency graphs climbing, replica count flat on its floor, and a CPU dashboard sitting calmly at just under half. Every component is working exactly as designed. The autoscaler is reading a number that has almost nothing to do with whether the service is busy.

One thread, two cores

Node runs your JavaScript on a single thread. Whatever else a pod is given, the part that executes application code is one core wide, and it cannot borrow a second one.

Kubernetes measures CPU utilization as a percentage of the pod’s request, not of a core. Ask for 2 CPUs, ordinary for a service that also compresses responses and collects telemetry, and the arithmetic is fixed before any traffic arrives: a completely saturated JavaScript thread burns one core out of two and reports 50%. A Horizontal Pod Autoscaler (the controller that adds and removes replicas, HPA from here) targeting 70% CPU will not fire at 49%. Not late. Never.

Here is an ordinary service measured across its saturation point: about 3.6ms of JSON work per request on the event loop, plus a 40ms downstream call, with load offered at a fixed arrival rate whether or not the server keeps up.

That is one run. Across six, the ELU and CPU columns repeat within a point; the last row’s tail does not, landing anywhere from 478ms to 3.1 seconds. Past the knee, latency stops being a number and starts being a range, which is its own argument for not scaling on it.

ELU is event-loop utilization: the fraction of time the loop spent doing work rather than sitting idle. It tracks the truth all the way up, 41% to 98%, and by the last row the service has fallen over. The CPU column tracks the same truth, compressed into the bottom half of its range, where the target lives that would have saved it. The ceiling is arithmetic: one core divided by 3.6ms is about 277 requests a second, and past that the excess turns into queue instead of into replicas.

This is the operational face of something I wrote about from inside the process: work that never shows up as a hot function still shows up as event-loop utilization. The number nobody graphs is also the number nobody scales on.

The other direction

The tempting conclusion is that CPU is a cautious, lagging version of the right signal. It isn’t. It can just as easily read high while the loop is nearly empty.

Node hands some work to libuv’s thread pool, a small set of background threads (four by default) used for file I/O, DNS, some crypto, and compression. That work burns CPU on threads that are not the event loop. Here is identical gzip work, run once asynchronously on the pool and once forced onto the loop:

Read those two rows against each other. The pod using nearly four cores is the healthy one, four times faster, with an event loop that is 85% idle; a CPU-targeting HPA sees 193% of a 2-core request and scales out hard for no reason. The pod using exactly one core is the dead one, and CPU reports it as half idle. Same work, opposite readings, and CPU is wrong in both directions. It is not a conservative proxy for saturation. It is a different quantity that happens to correlate when your runtime uses every core it is given, which Node does not.

Why not event-loop lag

perf_hooks has exposed the right number since Node 14.10. Call it with a previous reading and it returns the utilization for the interval between them, a ratio of active to total loop time that needs no calibration against instance size:

import { performance } from "node:perf_hooks";

let last = performance.eventLoopUtilization();

// Sample on the period your scraper collects on.
setInterval(() => {
  const next = performance.eventLoopUtilization();
  const { utilization } = performance.eventLoopUtilization(next, last);
  last = next;
  eluGauge.set(utilization); // 0..1, ready for the HPA
}, 15_000).unref();

The better-known metric is lag: schedule a timer for 100ms, measure how late it fires, call the difference the delay. It makes an excellent alert and a poor autoscaling input, for a reason that has nothing to do with accuracy. The HPA is a proportional controller, and its entire decision is one line:

desiredReplicas = ceil(currentReplicas * (currentMetric / targetMetric))

The formula assumes the metric scales with load: double the ratio, double the replicas. ELU satisfies that by construction, bounded on 0 to 1 and rising with traffic as the table shows.

Lag does not. It sits near zero while the loop has any headroom at all, because a queue that drains faster than it fills has nothing waiting, then goes vertical the moment arrival rate crosses service rate. That shape gives a proportional controller a ratio of 1.2 one moment and 60 the next, which is not a gradient it can steer on. The panel below shows what that does to a fleet.

Watch it

Two fleets, one traffic stream, different signals. The controller is the real thing: the HPA algorithm with its documented defaults. The service underneath it is a model, fitted to the table at the top of this page, running about sixteen times faster than real so the five-minute scale-down window is watchable.

The first preset is the case this essay opened with. Switch the left fleet to lag and wait: it pins to the replica ceiling, sits there while the stabilization window runs out, collapses back to the floor, immediately drowns, and climbs again. The ELU fleet finds its number and stays there. The third preset moves the work to the thread pool, where CPU reads high enough to scale out and nothing is saturated at all.

The target

Export the gauge, get it into Kubernetes as a custom metric with prometheus-adapter or KEDA, and point the HPA at it. Pod metric targets are Quantities rather than plain floats, so a target of 0.7 canonicalizes to 700m, which is what you will get back from kubectl whichever way you write it:

metrics:
  - type: Pods
    pods:
      metric:
        name: nodejs_event_loop_utilization
      target:
        type: AverageValue
        averageValue: 700m   # 0.7 utilization

Worth knowing: CPU is not a Kubernetes default in any deep sense. It is simply the only metric the original autoscaling API supported, and it is what every template you copy still ships with.

The table argues for a target near 0.7. Through 240 rps and 81% utilization, p99 stayed inside 100ms on every run; by 98% it is measured in seconds. The decision has to be made before that knee rather than on it, because new pods are not instant: image pull, process start, JIT warmup, and connection pools filling all happen after the HPA acts. Resist deriving the number from queueing theory, though. The classic formula wants latency to blow up as utilization climbs, and it does, but most of the latency here is a fixed 40ms downstream call that no amount of scaling touches. The measured knee is the honest evidence, and the shape transfers where the constants do not.

The tradeoff

ELU is a better primary signal, not a sufficient one. It is blind to memory pressure, to a saturated thread pool, and to a downstream dependency that is the real constraint, where scaling out only adds load to something already struggling. An HPA can carry several metrics and takes the highest recommendation among them, so ELU plus memory is a reasonable floor. It also cannot tell useful work from a runaway loop; both read 1.0.

There is a case for leaving CPU alone, too. If your service is genuinely CPU-bound in JavaScript, the two numbers converge and a metrics adapter buys nothing but a new thing to break. The gap between them is the whole justification for the extra machinery, so measure it first: if ELU and CPU-against-request track within a few points across your load range, keep the simple thing.

The principle

Autoscaling works when the metric you scale on is the resource that actually runs out. For a single-threaded runtime that resource is the event loop, and how full it is turns out to be a different number from how much CPU the pod is using. The default template scales on the one you can see rather than the one that is scarce.