Org Skills

Hot-path patterns

Supporting material for nodejs-performance. Agents load it on demand; it ships inside the skill folder.

RawSource

Verified 2026-09 against https://nodejs.org/api/worker_threads.html, https://nodejs.org/api/stream.html, https://github.com/piscinajs/piscina, https://github.com/nodejs/undici and https://fastify.dev/docs/latest/Reference/Validation-and-Serialization/.

1. Move CPU work off the loop

// src/thumbnails/pool.ts
import Piscina from 'piscina';

export const pool = new Piscina({
  filename: new URL('./resize.worker.js', import.meta.url).href,
  maxThreads: Math.max(1, Number(process.env.WORKER_THREADS ?? 2)),
  maxQueue: 100,                 // reject rather than queue forever
});
// src/thumbnails/resize.worker.ts  → compiled to resize.worker.js
export default async function resize({ key }: { key: string }): Promise<string> {
  // CPU-heavy work runs on a worker thread; the main loop keeps serving requests
  return `${key}-thumb`;
}
app.post('/thumbnails', async (request) => pool.run({ key: request.body.key }));

new URL(..., import.meta.url).href is the ESM-correct way to point at the worker file; a relative string resolves against the process working directory and breaks the moment the service is started from somewhere else.

maxQueue is the part people omit. Without it a burst queues unboundedly, memory grows and every queued request times out anyway — failing fast at the edge is better behaviour.

Threads share nothing by default; arguments are structured-cloned. For large binary payloads, pass a SharedArrayBuffer or transfer an ArrayBuffer rather than cloning megabytes per call.

2. Streams and backpressure

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';

await pipeline(
  createReadStream(source),
  createGzip(),
  createWriteStream(target),
);

pipeline propagates errors, destroys every stream on failure and respects backpressure. Hand-rolled a.pipe(b) chains leak file descriptors on error and silently ignore a full destination.

When writing manually, honour the return value:

if (!response.write(chunk)) {
  await once(response, 'drain');
}

Ignoring false from write() buffers the whole source in memory — the classic cause of "RSS is huge but the heap looks fine". With fetch, consume the body as a stream (response.body.pipeThrough(...)) rather than await response.text() when forwarding large payloads.

3. Serialisation

Fastify compiles a serialiser from a response schema, which is both faster than JSON.stringify and safer, because unlisted fields are dropped:

app.get('/orders/:id', {
  schema: {
    response: {
      200: {
        type: 'object',
        properties: {
          id: { type: 'string' },
          status: { type: 'string' },
          total: { type: 'number' },
        },
        required: ['id', 'status', 'total'],
      },
    },
  },
}, handler);

An internal field added to the entity next year cannot leak through this endpoint — the schema is both the performance win and the safety net.

Other wins in order of size: return fewer fields; paginate every list; stop deep-cloning with JSON.parse(JSON.stringify(x)) (use structuredClone, or do not clone); do compression at the proxy.

4. Caching with stampede protection

import { LRUCache } from 'lru-cache';

const inFlight = new Map<string, Promise<Product>>();

const cache = new LRUCache<string, Product>({
  max: 5_000,
  ttl: 60_000,
  ttlAutopurge: false,
});

export async function getProduct(id: string): Promise<Product> {
  const cached = cache.get(id);
  if (cached) return cached;

  const existing = inFlight.get(id);
  if (existing) return existing;          // coalesce concurrent misses

  const promise = loadProduct(id)
    .then((product) => {
      cache.set(id, product);
      return product;
    })
    .finally(() => inFlight.delete(id));

  inFlight.set(id, promise);
  return promise;
}

max and ttl together are the bound: max alone lets stale data live forever, ttl alone lets the cache grow without limit. The inFlight map is the stampede protection — without it, the first expiry under load sends every concurrent request to the origin at the same moment, which is how a cache turns into an outage amplifier. Add jitter to TTLs when many keys are populated together, so they do not all expire in the same second.

5. One dispatcher per upstream

import { Agent, setGlobalDispatcher, request } from 'undici';

setGlobalDispatcher(new Agent({
  keepAliveTimeout: 60_000,
  keepAliveMaxTimeout: 120_000,
  connections: 128,            // per origin
  headersTimeout: 5_000,
  bodyTimeout: 10_000,
}));

This is what makes global fetch reuse connections. A client constructed per request pays a TCP and TLS handshake per request — often more time than the call itself — and leaves sockets in TIME_WAIT.

connections caps concurrency per origin, which is a feature: it stops your service from being the reason an upstream falls over. headersTimeout and bodyTimeout are the safety net under the per-call AbortSignal.timeout from nodejs-development.

6. Connection pool sizing, worked

pods × processes-per-pod × pool-size  ≤  db max_connections − reserved

A concrete case: Postgres max_connections = 200, 20 reserved for migrations, admin and monitoring, 6 pods, 1 process each.

(200 − 20) / 6 = 30 connections per pod

So pool max = 30, and probably less: pool size should be near the concurrent query count, not the concurrent request count. A service where each request makes one short query rarely needs more than 10–20, and a bigger pool just moves the queue from your process into the database, where it is more expensive.

Under serverless, multiply by the platform's concurrency instead of pod count, and put PgBouncer or RDS Proxy in front rather than raising max_connections.

Symptom of getting it wrong: requests wait in the pool, ELU is low, loop lag is fine, and the database looks idle. That is the pool, not the database.

7. UV_THREADPOOL_SIZE and container CPU

The libuv thread pool (default 4 threads) serves fs, crypto (pbkdf2, bcrypt via its binding), zlib and dns.lookup. Ten concurrent bcrypt hashes with four threads means six are queued, and the queue is invisible in ELU — the loop is idle while requests wait.

Raise it to match the CPU the container is actually allowed:

UV_THREADPOOL_SIZE=8 node dist/server.js

Raising it far beyond the CPU limit does not help: the threads exist but the cgroup quota still caps the work, and with a Kubernetes CPU limit the excess shows up as CFS throttling — container_cpu_cfs_throttled_seconds_total climbing while average CPU looks low. That metric is worth an alert; it explains latency that no Node-level measurement accounts for.

Set CPU requests close to the real steady usage, and be cautious with tight limits on a bursty service: the throttle is applied per 100 ms period, so a short burst is stopped even when the average is far below the limit.