Adaptive concurrency limiting

The part nothing else in npm has, and the reason this library exists.

import { pipeline, breaker, limiter } from "resilix";

const api = pipeline({
  key: (req) => req.model,
  policies: [
    breaker({ slowCallMs: 3_000 }),
    limiter(),              // infers the right concurrency from latency
  ],
});

A circuit breaker is binary: open or closed. A limiter is continuous — it works out how many concurrent calls your upstream can actually absorb, from latency alone, and sheds the excess. Latency rises before errors do, which is why this catches degradation a failure-rate breaker cannot see at all.


Algorithm	Vegas queue estimation by default (`gradient2`, `aimd` also available)
Signal	p90 of recent latency, via an O(1) P² estimator
Over the limit	queue to 2×, then shed proportionally to 3× — not a cliff
`429` / timeout	short-circuits the control loop; no waiting for the next interval
`4xx`	a latency sample, but no pressure — the upstream did real work

If you stream, call ctx.mark(). The limiter judges on time-to-first-token; feed it total duration and a healthy 45-second completion looks like saturation.

Two behaviours worth knowing. Growth is tethered to observed concurrency, so a limit of 200 is never invented while ten calls are in flight — the tether caps growth only and never shrinks the limit during a lull. And because the control loop runs on call settlement rather than a timer (resilix has no timers), staleAfterMs exists to stop a limiter clamped during an incident from staying clamped forever once traffic goes quiet.

Going deeper

The full design — Vegas queue estimation, the P² quantile estimator, the baseline problem, and the provenance of every default — is in the adaptive limiter spec.

Adaptive concurrency limiting ​