The 429s Kept Coming After Recovery: Measuring Retry Jitter Across 2,000 Clients

Exponential backoff alone was slower than a fixed interval

I started with the naive implementations: a flat 5 second interval, exponential backoff capped at 60 seconds, and three flavors of jitter.

Recovery is at 30.0s; capacity is 120 req/sec.

The second row is the one that stung. Exponential backoff without jitter reaches 99% completion at 507.5 seconds — nearly seven times slower than the completely unsophisticated fixed 5 second interval at 73.5 seconds.

That second row was what I had shipped. For a long time I assumed that having exponential backoff in place meant the problem was handled. The implementation I trusted because "it does exponential backoff" was losing to a flat timer.

Note that the request total is identical: 9,423 either way. The load on the server did not go down at all. Only the waiting got longer.

The mechanism is straightforward once you see it. When the population stays synchronized, growing the delay just grows the gap between collisions. Every doubling is pure dead time.

The cap rebuilds the synchronization you removed

Exponential backoff normally carries a cap. Sixty seconds is a common choice.

The cap turned out to be the problem.

A longer cap is worse, which is unsurprising. What I had missed sits one step earlier.

The instant clients hit the cap, every delay collapses to the same value. While delays are climbing through 1s, 2s, 4s, differing attempt counts keep the population somewhat spread. At the cap, that spread disappears entirely.

From then on, all 2,000 clients knock in unison every 60 seconds. The cap regenerates exactly the synchronization jitter is supposed to destroy.

I had added the cap as a safety valve so no user would wait too long. In practice it was a synchronization generator.

A ±10% jitter is not jitter

My jitter was a ±10% multiplier on the delay. Logs showed varied wait times, which was enough to make me believe it was doing its job.

I measured a range of widths.

±10% does help: peak QPS drops from 1,174 to 706.

But 706 against 120 req/sec of capacity is still 5.9x over. Plenty of clients keep getting rejected, and 99% completion lands at 129.4 seconds. Full jitter's 152 QPS is a different order of magnitude.

±10% was not jitter. It was the appearance of jitter. To break synchronization you need spread on the same order as the interval you are trying to break.

Full jitter is not a free win either. It has the lowest peak at 152 QPS, but 99% completion is 84.5 seconds — behind decorrelated jitter's 73.4 seconds.

Because full jitter can draw delays near zero, unlucky clients burn attempts quickly, get rejected repeatedly, and climb toward the cap. It flattens the peak by lengthening the tail.

If protecting the server is the priority, full jitter. If getting people back into the app is the priority, decorrelated jitter. That is how this dataset reads.

Honoring Retry-After made recovery slower

This is where the measurement diverged most sharply from my expectation.

A well-behaved server sends Retry-After with a 429, and clients are supposed to respect it. Mine did.

Here is the same scenario with the server returning Retry-After: 30.

The compliant implementation was the slowest. 283.5 seconds to 99%, against 73.4 seconds for the client that ignores the header entirely.

The reason is embarrassingly simple. The server hands the same number to every client, so every client returns at the same moment. The synchronization I was fighting so hard to break was being reconstructed by my own server, on every rejection.

And the more faithfully the client honors the header, the more completely it overwrites its own randomness. Correctness, in this narrow sense, strengthened the herd.

Layering full jitter on top of Retry-After brought 99% completion down to 60.9 seconds — the fastest row in the table. The server's estimate of when capacity frees up is genuinely useful information. The only defect is that it is broadcast identically.

The 1,208 peak QPS in that row is the initial burst at the 30 second mark. Those requests were already scheduled during the outage, so no client-side strategy removes them. Shaving that spike requires throttling on the server side.

Measuring again, with capacity that degrades

There is a lie buried in every table above. The simulator treats server capacity as a flat 120 req/sec.

Real servers do not behave that way. Under load the queue grows, GC kicks in, the connection pool drains. The more traffic arrives, the less of it the server can actually clear.

I had been carrying that caveat around without checking it. Adding the effect turned out to be a single expression, so I stopped putting it off.

// Discount effective capacity by how far arrivals exceed nominal capacity
const overload = arrivalsQps / CAPACITY_PER_SEC;
const factor = overload <= 1
  ? 1
  : Math.max(FLOOR, 1 / Math.pow(overload, EXP));  // EXP=0.5, FLOOR=0.35
capTick = CAP_TICK * factor;

EXP controls how sharply capacity falls away, FLOOR sets the floor below which it never drops. Starting with the gentler setting, here is the comparison against the flat-capacity runs.

What stood out is how little the peak-suppressing strategies cared about the change in assumptions.

Full jitter, peaking at 150 QPS, moved by 0.4%. Decorrelated jitter, peaking at 298, moved by 4.0%. Fixed 5s, peaking at 1,168, moved by 82.7% — and its request count swelled from 9,433 to 22,827.

Rejected, retried, capacity drops further, rejected harder. Once that loop starts turning, peak QPS stops being a momentary number and becomes the reason the tail keeps stretching.

Peak QPS protects your server, and it also protects you when your assumptions turn out to be wrong.

Does a harsher collapse widen the gap? I ran EXP=1.0 with FLOOR=0.2 as well.

Fixed 5s lands at 241.6 seconds and 45,961 requests — 4.9x the flat-capacity run. An implementation without jitter only gets worse as the assumptions get harder.

The reversal I expected never arrived. I had assumed that once capacity degraded, full jitter's lower peak would pull it ahead of decorrelated jitter. It came out 85.0s against 80.8s, in the same order as before.

The gap did close, though. Ten and a half seconds under flat capacity, four point two under the harsh collapse. With a steeper curve, or a lower floor, the order may well flip somewhere — I may be missing a regime where it does.

I am not confident enough to claim where that point is. What I can hand you is the finding that the order held while the margin shrank, plus the two constants to turn if your own setup disagrees.

Putting it into the app

Here is what the measurements turned into on the Expo / React Native side. Three decisions:

1. Decorrelated jitter by default, favoring time-to-recovery
2. Retry-After treated as a floor, always with randomness layered on top
3. A per-install fixed offset added to every delay, to further break same-second clustering

const BASE_DELAY_MS = 1000;
const MAX_DELAY_MS = 60000;
const MAX_ATTEMPTS = 6;
 
// A stable 0-1 offset derived from the install id, unchanged across launches
function installOffset(installId: string): number {
  let h = 2166136261;
  for (let i = 0; i < installId.length; i++) {
    h ^= installId.charCodeAt(i);
    h = Math.imul(h, 16777619);
  }
  return ((h >>> 0) % 100000) / 100000;
}
 
// Uniform draw over [BASE, prev * 3], anchored to the previous delay
function decorrelatedJitter(prevDelay: number, rand: () => number): number {
  const upper = Math.max(BASE_DELAY_MS * 2, prevDelay * 3);
  return Math.min(MAX_DELAY_MS, BASE_DELAY_MS + rand() * (upper - BASE_DELAY_MS));
}
 
// Accepts both the seconds form and the HTTP-date form
function parseRetryAfter(header: string | null): number | null {
  if (!header) return null;
  const sec = Number(header);
  if (Number.isFinite(sec)) return Math.max(0, sec * 1000);
  const at = Date.parse(header);
  return Number.isFinite(at) ? Math.max(0, at - Date.now()) : null;
}
 
const RETRYABLE = new Set([408, 425, 429, 500, 502, 503, 504]);
 
const sleep = (ms: number, signal?: AbortSignal) =>
  new Promise<void>((resolve, reject) => {
    const id = setTimeout(resolve, ms);
    signal?.addEventListener(
      'abort',
      () => { clearTimeout(id); reject(new DOMException('Aborted', 'AbortError')); },
      { once: true },
    );
  });
 
type RetryOpts = {
  installId?: string;
  rand?: () => number;
  signal?: AbortSignal;
  onRetry?: (info: { attempt: number; waitMs: number; reason: string }) => void;
};
 
export async function fetchWithRetry(
  url: string,
  init: RequestInit = {},
  opts: RetryOpts = {},
): Promise<Response> {
  const { installId = 'anonymous', rand = Math.random, signal, onRetry } = opts;
  const offset = installOffset(installId);
  let prevDelay = BASE_DELAY_MS;
  let lastErr: unknown;
 
  for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
    try {
      const res = await fetch(url, { ...init, signal });
      if (!RETRYABLE.has(res.status)) return res; // most 4xx return immediately
      lastErr = new Error(`HTTP ${res.status}`);
 
      const ra = parseRetryAfter(res.headers.get('retry-after'));
      // Retry-After is a floor; always layer randomness so clients do not realign
      prevDelay = ra !== null
        ? ra + rand() * Math.max(ra, BASE_DELAY_MS)
        : decorrelatedJitter(prevDelay, rand);
    } catch (e) {
      if ((e as Error).name === 'AbortError') throw e; // never retry a cancelled screen
      lastErr = e;
      prevDelay = decorrelatedJitter(prevDelay, rand);
    }
 
    if (attempt === MAX_ATTEMPTS) break;
    const wait = Math.min(MAX_DELAY_MS, prevDelay + offset * BASE_DELAY_MS);
    onRetry?.({ attempt, waitMs: Math.round(wait), reason: String((lastErr as Error)?.message ?? lastErr) });
    await sleep(wait, signal);
  }
  throw lastErr;
}

Running it against a local server that returns 429 three times and then 200:

status: 200 body: ok
server hits: 4 elapsed: 6272 ms
retry log: [
  {"attempt":1,"waitMs":1889,"reason":"HTTP 429"},
  {"attempt":2,"waitMs":1704,"reason":"HTTP 429"},
  {"attempt":3,"waitMs":2640,"reason":"HTTP 429"}
]

Against Retry-After: 1, the actual waits landed between 1.7 and 2.6 seconds. The floor is respected; the values do not collapse onto each other.

rand is injectable specifically for tests. Pass a constant and the entire wait sequence becomes reproducible, which keeps retry tests from flaking.

Checking that the per-install offset actually spreads

installOffset is a trimmed FNV-1a and nothing more. A skewed distribution would defeat its purpose, so I ran 2,000 UUIDs through it.

Histogram over 10 buckets (ideal = 200): 200 212 193 193 192 189 180 213 217 211
max/min: 217 / 180   skew: 18.5%

Highest bucket 217, lowest 180, against an ideal of 200 — an 18.5% skew.

That would be poor for anything cryptographic. For the goal here, which is breaking same-second clustering, it is fine.

The more interesting choice is why this is not Math.random(). A value that changes on every launch means that clients which restart the app realign with each other. During an outage people reopen the app constantly, so that realignment is not hypothetical. Deriving the offset from the install id keeps each client in its own slot no matter how many times it is reopened.

What I settled on

For my apps, the configuration ended up as:

1. Decorrelated jitter by default, 60 second cap, 6 attempts maximum
2. Retry-After used only as a floor, always multiplied by a 1x-2x random factor
3. A per-install fixed offset added to every wait
4. The server keeps sending Retry-After on 429, because the value itself is useful
5. AbortController to reliably stop retries when a screen is dismissed

One trap caught me before this reached production. I had AbortError in the retryable path, so a request fired from a screen the user had already dismissed kept going for up to six attempts in the background.

Nothing shows up in the logs. The screen is gone, so a failure has no observer. During post-recovery congestion, those unwanted requests were competing for the same capacity as the ones people were actually waiting on.

The fix is the single line if ((e as Error).name === 'AbortError') throw e; above. Obvious in isolation, easy to miss when retry logic is bolted on afterwards.

Which one you pick depends on what you are protecting.

The open question now is the initial burst at the 30 second mark. Those requests were already scheduled while the service was down, so no client-side strategy removes them. Throttling on the server side is the next thing worth measuring.

That the order survived degrading capacity genuinely surprised me. I had expected the peak-suppressing strategy to come out ahead, and only the margin moved. Being wrong is usually where the measurement earns its keep.

My own retry code was blocking my own server. That was an uncomfortable thing to sit with. But once the numbers were on the table, the fix was obvious.

If you are staring at the same problem, I hope some of this saves you an afternoon.