# Hot-path patterns

Verified 2026-09 against <https://nodejs.org/api/worker_threads.html>,
<https://nodejs.org/api/stream.html>, <https://github.com/piscinajs/piscina>,
<https://github.com/nodejs/undici> and <https://fastify.dev/docs/latest/Reference/Validation-and-Serialization/>.

## 1. Move CPU work off the loop

```ts
// src/thumbnails/pool.ts
import Piscina from 'piscina';

export const pool = new Piscina({
  filename: new URL('./resize.worker.js', import.meta.url).href,
  maxThreads: Math.max(1, Number(process.env.WORKER_THREADS ?? 2)),
  maxQueue: 100,                 // reject rather than queue forever
});
```

```ts
// src/thumbnails/resize.worker.ts  → compiled to resize.worker.js
export default async function resize({ key }: { key: string }): Promise<string> {
  // CPU-heavy work runs on a worker thread; the main loop keeps serving requests
  return `${key}-thumb`;
}
```

```ts
app.post('/thumbnails', async (request) => pool.run({ key: request.body.key }));
```

`new URL(..., import.meta.url).href` is the ESM-correct way to point at the worker file; a relative
string resolves against the process working directory and breaks the moment the service is started
from somewhere else.

`maxQueue` is the part people omit. Without it a burst queues unboundedly, memory grows and every
queued request times out anyway — failing fast at the edge is better behaviour.

Threads share nothing by default; arguments are structured-cloned. For large binary payloads, pass a
`SharedArrayBuffer` or transfer an `ArrayBuffer` rather than cloning megabytes per call.

## 2. Streams and backpressure

```ts
import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';

await pipeline(
  createReadStream(source),
  createGzip(),
  createWriteStream(target),
);
```

`pipeline` propagates errors, destroys every stream on failure and respects backpressure. Hand-rolled
`a.pipe(b)` chains leak file descriptors on error and silently ignore a full destination.

When writing manually, honour the return value:

```ts
if (!response.write(chunk)) {
  await once(response, 'drain');
}
```

Ignoring `false` from `write()` buffers the whole source in memory — the classic cause of "RSS is
huge but the heap looks fine". With `fetch`, consume the body as a stream
(`response.body.pipeThrough(...)`) rather than `await response.text()` when forwarding large payloads.

## 3. Serialisation

Fastify compiles a serialiser from a response schema, which is both faster than `JSON.stringify` and
safer, because unlisted fields are dropped:

```ts
app.get('/orders/:id', {
  schema: {
    response: {
      200: {
        type: 'object',
        properties: {
          id: { type: 'string' },
          status: { type: 'string' },
          total: { type: 'number' },
        },
        required: ['id', 'status', 'total'],
      },
    },
  },
}, handler);
```

An internal field added to the entity next year cannot leak through this endpoint — the schema is
both the performance win and the safety net.

Other wins in order of size: return fewer fields; paginate every list; stop deep-cloning with
`JSON.parse(JSON.stringify(x))` (use `structuredClone`, or do not clone); do compression at the proxy.

## 4. Caching with stampede protection

```ts
import { LRUCache } from 'lru-cache';

const inFlight = new Map<string, Promise<Product>>();

const cache = new LRUCache<string, Product>({
  max: 5_000,
  ttl: 60_000,
  ttlAutopurge: false,
});

export async function getProduct(id: string): Promise<Product> {
  const cached = cache.get(id);
  if (cached) return cached;

  const existing = inFlight.get(id);
  if (existing) return existing;          // coalesce concurrent misses

  const promise = loadProduct(id)
    .then((product) => {
      cache.set(id, product);
      return product;
    })
    .finally(() => inFlight.delete(id));

  inFlight.set(id, promise);
  return promise;
}
```

`max` and `ttl` together are the bound: `max` alone lets stale data live forever, `ttl` alone lets the
cache grow without limit. The `inFlight` map is the stampede protection — without it, the first
expiry under load sends every concurrent request to the origin at the same moment, which is how a
cache turns into an outage amplifier. Add jitter to TTLs when many keys are populated together, so
they do not all expire in the same second.

## 5. One dispatcher per upstream

```ts
import { Agent, setGlobalDispatcher, request } from 'undici';

setGlobalDispatcher(new Agent({
  keepAliveTimeout: 60_000,
  keepAliveMaxTimeout: 120_000,
  connections: 128,            // per origin
  headersTimeout: 5_000,
  bodyTimeout: 10_000,
}));
```

This is what makes global `fetch` reuse connections. A client constructed per request pays a TCP and
TLS handshake per request — often more time than the call itself — and leaves sockets in `TIME_WAIT`.

`connections` caps concurrency per origin, which is a feature: it stops your service from being the
reason an upstream falls over. `headersTimeout` and `bodyTimeout` are the safety net under the
per-call `AbortSignal.timeout` from `nodejs-development`.

## 6. Connection pool sizing, worked

```
pods × processes-per-pod × pool-size  ≤  db max_connections − reserved
```

A concrete case: Postgres `max_connections = 200`, 20 reserved for migrations, admin and monitoring,
6 pods, 1 process each.

```
(200 − 20) / 6 = 30 connections per pod
```

So `pool max = 30`, and probably less: pool size should be near the *concurrent query* count, not the
concurrent request count. A service where each request makes one short query rarely needs more than
10–20, and a bigger pool just moves the queue from your process into the database, where it is more
expensive.

Under serverless, multiply by the platform's concurrency instead of pod count, and put PgBouncer or
RDS Proxy in front rather than raising `max_connections`.

Symptom of getting it wrong: requests wait in the pool, ELU is low, loop lag is fine, and the database
looks idle. That is the pool, not the database.

## 7. `UV_THREADPOOL_SIZE` and container CPU

The libuv thread pool (default **4** threads) serves `fs`, `crypto` (`pbkdf2`, `bcrypt` via its
binding), `zlib` and `dns.lookup`. Ten concurrent bcrypt hashes with four threads means six are
queued, and the queue is invisible in ELU — the loop is idle while requests wait.

Raise it to match the CPU the container is actually allowed:

```bash
UV_THREADPOOL_SIZE=8 node dist/server.js
```

Raising it far beyond the CPU limit does not help: the threads exist but the cgroup quota still caps
the work, and with a Kubernetes CPU *limit* the excess shows up as **CFS throttling** —
`container_cpu_cfs_throttled_seconds_total` climbing while average CPU looks low. That metric is
worth an alert; it explains latency that no Node-level measurement accounts for.

Set CPU requests close to the real steady usage, and be cautious with tight limits on a bursty
service: the throttle is applied per 100 ms period, so a short burst is stopped even when the average
is far below the limit.
