Sub-10ms WebSockets with Node.js & Bun: Kernel Tuning, epoll, and Memory Slicing

Our realtime gateway held 500,000 idle-but-live WebSocket connections across a fleet of Node.js servers. The fleet was sized by memory, not CPU: each connection cost 14.8 KB of resident memory, so a 16 GB node topped out around 1 million sockets in theory and 600,000 in practice.

Heap snapshots showed the cost was spread thin: per-socket buffers, an EventEmitter per connection, closure-captured state and a framing object allocated for every inbound message. None of it was a leak, which is exactly why it never showed up in a profiler.

“We were not out of CPU. We were paying a memory tax on every socket just to keep it politely idle.”

Default Linux TCP buffers are sized for throughput, not for a million mostly-silent sockets. Shrinking the per-socket read and write buffers returned hundreds of megabytes per node, and raising the file descriptor ceiling removed an entire class of EMFILE incidents.

1# fewer bytes pinned per idle connection2net.ipv4.tcp_rmem = 4096 8192 2621443net.ipv4.tcp_wmem = 4096 8192 2621444 5# room for a million sockets6fs.file-max = 20971527net.core.somaxconn = 655358net.ipv4.ip_local_port_range = 1024 65535

With tuned buffers the same workload fit in roughly 9 KB per socket on Node.js. The remaining gap was userland: the ws library and V8 itself.

Bun's native WebSocket server keeps connection state in native memory and hands your handler a single shared object, instead of allocating an emitter and closures per socket. Moving the gateway over took the per-connection cost from 14.8 KB to 2.1 KB, and removed the garbage-collector pauses that had been inflating our tail latency.

1Bun.serve({2  port: 8080,3  fetch(req, server) {4    return server.upgrade(req) ? undefined : new Response(null, { status: 426 });5  },6  websocket: {7    perMessageDeflate: false,8    message(ws, data) {9      ws.send(data);10    },11  },12});