nextWrite() throws an uncaughtException when a reserved connection's backend dies

Version

postgres 3.4.9 · Bun 1.4.0 · PostgreSQL 18.6 (also reproduces on 16)

What happens

A sql.reserve()d connection whose backend is terminated server-side kills the process on the next query:

TypeError: null is not an object (evaluating 'socket.write')
    at nextWrite (postgres/src/connection.js:255:15)

It surfaces as an uncaughtException, not a rejected promise — try/catch around the awaited query does not catch it, so there is no way for the application to defend against it.

Minimal reproduction

import postgres from 'postgres'

const sql = postgres(URL, { max: 2 })
const killer = postgres(URL, { max: 1 })

const reserved = await sql.reserve()
const [{ pid }] = await reserved`select pg_backend_pid() as pid`
await killer`select pg_terminate_backend(${pid})`

await reserved`select 1`.catch(() => {})   // rejects: CONNECTION_CLOSED  (correct)
await reserved`select 1`                   // <-- uncaughtException, process dies

Deterministic — first attempt, every time.

Why

closed() sets socket = null (connection.js#L448), and reconnect() only re-creates it on a later setTimeout (#L361) — so the connection sits with a null socket across ticks.

nextWrite() is the only consumer of socket in that file without a null guard. terminate() has if (socket) and end() has socket && …; this one dereferences directly.

It is also the one reached from the socket 'data' handler: the protocol dispatcher (L613–L847) and the SASL exchange (L688–L736) both call write(), and data is registered as socket.on('data', data) (L372). A throw there has no promise to reject, so it escapes to uncaughtException.

A pooled connection masks this, because the pool rotates a dead connection away. A reserved one is pinned and cannot be rotated, so every subsequent write on it is fatal. That is why this shows up as an unrecoverable crash loop specifically for reserve() users — in our case an advisory-lock helper, roughly one process exit every 8 minutes in production.

Worth noting how narrow the underlying race is without reserve(): ~2,800 synthetic connection kills (mid-handshake, blackholed handshakes, post-connect RSTs, >1024-byte statements to force the synchronous write branch) never reproduced it. Reserving is what makes it deterministic, because the pool can no longer heal it.

Suggested fix

Guard nextWrite() — and settle rather than drop. Returning false alone is not enough: the write vanishes and the caller awaits forever (verified — the query hangs).

function nextWrite(fn) {
  if (!socket) {
    chunk = null
    nextWriteTimer !== null && clearImmediate(nextWriteTimer)
    nextWriteTimer = null
    error(Errors.connection('CONNECTION_CLOSED', options, socket))
    return false
  }
  const x = socket.write(chunk, fn)
  ...

With this, the second query rejects with CONNECTION_CLOSED — catchable — and the process survives. We are running it as a patch against 3.4.9.

Happy to open a PR if the approach looks right.

Checked for duplicates

#896 is a different defect (query.origin undefined in queryError); #43 and #179 are pool/ECONNRESET reports without this stack.

Follow-up: I re-ran the repro across runtimes, since the original report was Bun-only and that invites a "probably a Bun thing" read. It isn't. Same defect on Node, and on every earlier Bun release I tested.

Sandbox outside our own repo, clean postgres@3.4.9 from npm (confirmed nextWrite unguarded), PostgreSQL 17.11, the exact repro from the issue body:

Runtime	unpatched	with the guard from the issue body
Node 23.10.0	`TypeError: Cannot read properties of null (reading 'write')` → exit 1	rejects `CONNECTION_CLOSED`, exit 0
Bun 1.4.0	`TypeError: null is not an object (evaluating 'socket.write')` → exit 1	rejects `CONNECTION_CLOSED`, exit 0
Bun 1.3.14	same → exit 1	same, exit 0
Bun 1.2.23	same → exit 1	same, exit 0

Deterministic, first attempt, on all four.

Node stack — the processImmediate frame makes the path explicit:

node_modules/postgres/src/connection.js:255
    const x = socket.write(chunk, fn)
                     ^
TypeError: Cannot read properties of null (reading 'write')
    at Immediate.nextWrite (node_modules/postgres/src/connection.js:255:22)
    at process.processImmediate (node:internal/timers:508:21)

Node.js v23.10.0

I instrumented nextWrite to dump its call site on both engines: it is the setImmediate(nextWrite) scheduled by write(), firing after closed() has already set socket = null. Attaching a process.on('uncaughtException') listener confirms it arrives as an uncaughtException on both.

Correction to the snippet I filed

The repro in the issue body uses top-level await, and under Bun that hides the crash: while module evaluation is still suspended, Bun prints the uncaught exception but does not terminate, so you observe a query that never settles rather than a dead process. Node exits either way. Reduced away from postgres.js entirely:

setImmediate(() => { null.write('x') })
setTimeout(() => { console.log('ALIVE'); process.exit(0) }, 1500)
await new Promise(() => {})     // module evaluation still suspended

Bun 1.2.23 / 1.3.14 / 1.4.0 → prints the TypeError, then ALIVE, exit 0. Node 23 → exit 1.

So if you paste my original snippet into Bun it will look like a hang rather than a crash. Wrapping the same body in a function reproduces the process exit on all four runtimes — better to use this form:

import postgres from 'postgres'

async function main() {
  const sql = postgres(URL, { max: 2 })
  const killer = postgres(URL, { max: 1 })

  const reserved = await sql.reserve()
  const [{ pid }] = await reserved`select pg_backend_pid() as pid`
  await killer`select pg_terminate_backend(${pid})`

  await reserved`select 1`.catch(() => {})   // rejects CONNECTION_CLOSED (correct)
  await reserved`select 1`                   // <-- uncaughtException, process dies
}
main()

Worth noting the hang is real on Node too, independently of that runtime difference: with an uncaughtException listener installed so the process survives, query#2 never settles on either engine. That is why the guard has to error(...) rather than just return false.

The suggested guard was re-verified on all four runtimes — query#2 rejects with CONNECTION_CLOSED and the process exits 0. Still happy to send it as a PR if the approach looks right.

Confirming this on a second, independent trigger path, and adding one data point that differs from your testing.

Same crash, reached through `begin()` rather than `reserve()`

sql.begin() hits the identical nextWrite dereference, and there the application has no way to avoid it at all. When the in-flight query fails, the transaction implementation issues its own ROLLBACK from the catch block:

https://github.com/porsager/postgres/blob/master/src/index.js#L266

closed() has already set socket = null, so that internal rollback is the write that lands on the null socket. With reserve() the app at least controls the offending query; with begin() the fatal write is postgres.js's own, so no amount of caller-side care helps.

Minimal reproduction

import postgres from 'postgres'

const sql = postgres(URL, { max: 2 })
const killer = postgres(URL, { max: 1 })

let pid = null
const tx = sql.begin(async (t) => {
  const [row] = await t`select pg_backend_pid() as pid`
  pid = row.pid
  await t`select pg_sleep(2)`        // in flight when the backend dies
})
while (pid === null) await new Promise(r => setTimeout(r, 50))
await killer`select pg_terminate_backend(${pid})`

await tx.catch(() => {})   // rejects CONNECTION_CLOSED (correct)
// ...and separately: TypeError from a setImmediate callback, uncaughtException

Both conditions are load-bearing, which narrows it usefully:

shape	uncaught
in-flight query, no transaction	0
transaction + query in flight	1
transaction open, nothing in flight	0

5/5 deterministic. Node 26.3, PostgreSQL 17, postgres 3.4.8. Also reproduced against 3.4.9, where nextWrite is byte-identical.

One difference from your results

We are running your suggested guard, with the error(Errors.connection('CONNECTION_CLOSED', options, socket)) call rather than a bare return false. Your point about the bare return is right and we measured it: it converts the crash into a hang.

On the transaction path the guard behaves exactly as you describe. The transaction rejects with CONNECTION_CLOSED and the process survives.

On the reserve() path, though, a follow-up query still did not settle within 5s in our environment, with or without the error(...) call. That differs from your "re-verified on all four runtimes", so it is probably something about our setup rather than a contradiction, but it seemed worth flagging in case the queued-query path settles differently on Node 26 or PostgreSQL 17. We have no reserve() call sites, so we did not chase it further.

Happy to test a PR against the transaction path if that is useful.

[Fix] Retry the Docker migration runner when Postgres drops the conne…

One more trigger for the begin() path described above, without pg_terminate_backend: the transaction's connection is cut by the network (a TCP proxy destroying both sockets) while a query of the transaction is in flight. sql.begin() rejects with CONNECTION_CLOSED, and begin()'s own ROLLBACK then throws from nextWrite (connection.js:255) outside any promise.

Deterministic on postgres 3.4.9 with Bun 1.4.2 (TypeError: null is not an object (evaluating 'socket.write')) and Node 24.21.0 (TypeError: Cannot read properties of null (reading 'write')), PostgreSQL 17.

Standalone repro (no dependency but postgres; DATABASE_URL=postgres://… node repro.mjs):

// The connection of a sql.begin() transaction is cut (as a network failure would) while its query runs.
import net from 'node:net'
import postgres from 'postgres'

process.on('uncaughtException', (e) => {
  console.log('UNCAUGHT:', e && e.stack || e)
  process.exit(1)
})

const target = new URL(process.env.DATABASE_URL ?? 'postgres://postgres@127.0.0.1:5432/postgres')

// A plain TCP proxy, so the test can cut the connection the way a network failure or a restart would.
const sockets = new Set()
const proxy = net.createServer((client) => {
  const server = net.connect(Number(target.port || 5432), target.hostname)
  client.pipe(server).pipe(client)
  sockets.add(client).add(server)
  const close = () => { client.destroy(); server.destroy() }
  client.on('error', close); server.on('error', close)
  client.on('close', close); server.on('close', close)
})
await new Promise((r) => proxy.listen(0, '127.0.0.1', r))
const url = new URL(target)
url.hostname = '127.0.0.1'
url.port = String(proxy.address().port)

const sql = postgres(url.toString(), { max: 1 })
await sql`select 1` // connect

// Cut every connection while the transaction's query is running.
setTimeout(() => { for (const s of sockets) s.destroy() }, 200)

try {
  await sql.begin(async (tx) => {
    await tx`select pg_sleep(2)`
  })
  console.log('begin resolved (unexpected)')
} catch (e) {
  console.log('begin rejected as expected:', e.code ?? e.message)
}

await new Promise((r) => setTimeout(r, 500))
console.log('no uncaught exception: OK')
await sql.end({ timeout: 1 })
proxy.close()

We hit it in a database layer that retries transactions after a lost connection; a guard in nextWrite such as #1209 / #1168 would fix it for us.

Confirming the diagnosis from an independent production case, and adding two things: a trigger that does not need reserve(), and a reproduction that needs no PostgreSQL at all.

Independent production case: the trigger is process shutdown

postgres 3.4.9 · Node 24.21.0 · self-hosted service, pooled connections only (no reserve() anywhere in the codebase).

The same line, same signature:

TypeError: Cannot read properties of null (reading 'write')
    at Immediate.nextWrite (postgres/src/connection.js:255:22)
    at process.processImmediate (node:internal/timers:574:21)

What makes it deterministic here is not reserve() but SIGTERM. The database is a child process of the Node process, in the same cgroup, and the service is stopped with systemd's default KillMode=control-group. So the DB receives SIGTERM in the same instant as the server. The server's shutdown path survives long enough to run its drain (we bundle async-exit-hook, which defers exit), issues one more query on a pooled connection whose socket has just been nulled by closed(), and dies instead of stopping.

Measured over the retained journal: 2 null-write TypeErrors, both immediately followed by Main process exited, code=exited, status=1/FAILURE, out of 23 service stops. Every stop-attributable crash is this one; zero occur outside a shutdown.

I read your "without reserve() the race is very narrow" note as consistent with this — we are not racing anything, we are being told to write after the socket is already gone. The pool gets no chance to rotate the dead connection because the process is on its way out.

Your settle-vs-drop warning is correct, and we measured it

You are right that return false alone is not enough. I had shipped exactly that locally first; it silences the crash and leaves the caller awaiting a reply that can never arrive. I changed the guard to the shape you proposed:

function nextWrite(fn) {
  if (socket === null) {
    chunk = null
    nextWriteTimer !== null && clearImmediate(nextWriteTimer)
    nextWriteTimer = null
    error(Errors.connection('CONNECTION_CLOSED', options, socket))
    return false
  }
  const x = socket.write(chunk, fn)
  ...

A/B against two copies of the installed package, one stock and one guarded, driven through the real Connection:

arm	result
stock 3.4.9	fatal `TypeError` at `Immediate.nextWrite`, process exits 1
guarded	exits 0, and the in-flight query settles with `CONNECTION_CLOSED`

So the guard does not trade a crash for a hang.

Reproduction without a PostgreSQL server

Worth posting separately if useful — the failure needs no database, no network and no reserve(), which makes it a much cheaper regression test than a live kill:

Connection accepts an options.socket factory, and when options.socket is set, connect() short-circuits straight to connected(). So a plain EventEmitter with write/end/destroy/setKeepAlive is enough:

build Connection with sql.options from the real postgres() factory (so nothing is hand-configured) and a fake socket;
connection.connect(query), let connected() flush the startup message;
feed AuthenticationOk + BackendKeyData + ReadyForQuery so the connection is established — this matters, because while initial is set closed() reconnects and the socket is not left null, so the crash will not reproduce;
emit 'close' → real closed() → socket = null;
connection.execute(otherQuery) → coalesced buffer, setImmediate(nextWrite);
the immediate fires. Stock: fatal. Guarded: settles.

One gotcha for anyone writing this: data() reads the frame length at offset 1 (incoming.readUInt32BE(1)) and dispatches on xs[0], so the buffer it wants is [type][int32 length][payload] — not the [int32 length][type][payload] of the PostgreSQL wire format. Feeding standard framing is silently treated as an incomplete message and the handshake never parses, which looks exactly like "the bug did not reproduce".

Happy to open a PR for the nextWrite guard if that would help — it is 8 lines and needs no behaviour change for the healthy path.

A second defect on the same path, which the nextWrite guard from #1209 turns from a crash into a poisoned pool: release() of a reserved connection whose socket already closed puts it back in open with socket === null.

Sequence (3.4.7; the same code is in 3.4.9, src/index.js):

The backend dies → closed() → onclose(c): move(c, closed) and c.reserved = null.
The holder calls reserved.release() → c.reserved = null; onopen(c) → move(c, open). Nothing checks that the pool already took the connection back.
The next pool query routed to c enters execute(). closed() rejected the in-flight query but never cleared query (errored() doesn't null it), so the new query goes to sent, and with Guard nextWrite against a closed socket - fixes #1208 #1209 nextWrite rejects it with CONNECTION_CLOSED, decorated with the stale query (queryError(query, err) defines query/stack on the error before the new query receives it). Without Guard nextWrite against a closed socket - fixes #1208 #1209 it is the same TypeError.
c stays in open for good: every later query routed to it fails the same way.

Whether it happens depends on whether the socket close beats release(), so with a real pool it is intermittent (it was 3 of 5 CI runs for us, never locally). With max: 1 and waiting for onclose, it is deterministic:

import postgres from 'postgres'

let signalClosed
const closed = new Promise(r => signalClosed = r)
const sql = postgres(URL, { max: 1, onclose: () => signalClosed() })
const killer = postgres(URL, { max: 1 })

const reserved = await sql.reserve()
const [{ pid }] = await reserved`select pg_backend_pid() as pid`
await killer`select pg_terminate_backend(${pid})`
await reserved`select 'stale' as q`.catch(() => {})   // 57P01 — correct
await closed                                          // the socket closed first
reserved.release()

await sql`select 1`   // rejects CONNECTION_CLOSED (with #1209); err.query === "select 'stale' as q"

20 of 20 without a fix. Releasing before the close passes 5 of 5, which is the race.

What we apply as a patch on top of #1209 (also makes release() idempotent, and harmless once the connection was re-reserved by someone else):

-    c.reserved = () => queue.length
+    const reservation = c.reserved = () => queue.length
       ? c.execute(queue.shift())
       : move(c, reserved)
     c.reserved.release = true

     const sql = Sql(handler)
     sql.release = () => {
+      if (c.reserved !== reservation)
+        return
       c.reserved = null
       onopen(c)
     }

20 of 20 with it. Happy to open a PR if that helps.

Independent reproduction on Node 26.0.0, postgres 3.4.9, PostgreSQL 18.6, without reserve():

Inside sql.begin(...), if the backend is terminated while a statement is in flight (we inject database outages in a fault-tolerance test), the transaction's ROLLBACK is queued after closed() has set socket = null. nextWrite() then throws TypeError: Cannot read properties of null (reading 'write'), which escapes as an uncaughtException and takes the worker process down. So transaction connections hit it too, not only reserved ones.

We now carry a local patch equivalent to #1209 (guard nextWrite() and settle through error(Errors.connection('CONNECTION_CLOSED', options))). With it the rollback rejects with a catchable CONNECTION_CLOSED, our workers survive repeated outages, and an offline regression that queues a rollback after socket close passes. +1 for merging #1209.

Reproduction against a real PostgreSQL 17 with sql.begin (3.4.9)

We hit the nextWrite null-socket throw through sql.begin, not reserve(). With postgres(url, { max: 1, prepare: false }) and with the defaults, a transaction whose session is ended by pg_terminate_backend from another connection while select pg_sleep(10) is in flight throws TypeError: Cannot read properties of null (reading 'write') from Immediate.nextWrite in 20 of 20 runs (Linux, Node 26.8.1). The same happens when the server closes an idle-in-transaction session and the callback then issues its next statement, and when the socket is closed cleanly with no server message. It does not happen on ECONNRESET.

The begin promise itself rejects with CONNECTION_CLOSED. The rollback issued by scope() is queued behind the stale query and never settles. With max: 1, later queries on the client were then not answered for as long as we watched (about 95 s), so a host that swallows uncaught exceptions (Next.js does) is left with a client it cannot use.

A terminated backend takes the clean-close path because ErrorResponse stores the error while a query is active and only ReadyForQuery delivers it; a FATAL is never followed by ReadyForQuery.

A guard in execute() (if (socket === null) return (queryError(q, CONNECTION_CLOSED), true)) removed the throw and the hang in our runs. It does not stop a callback that outlives its closed transaction from writing on the reconnected connection (#1216); with max: 1 that write was committed in 20 of 20 runs. Adding the closedError handling from #1215 removed that. We can also confirm, against a protocol double, the stale chunk/nextWriteTimer reconnect failure described in a comment on #1215, and #1223's start-up case; resetting the same state before the if (initial) return reconnect() early return removed the latter in our runs, including against a real server behind a relay that injects the termination. Trial scripts and results available on request.

fix: replace connection lifecycle flags with explicit phases

fix: bind reserve() and begin() handles to a connection lease

We reproduced this on 3.4.9 under Bun 1.4.0. A patch is in #1248 (link below), covering this path and the same defect in sql.begin(). For reserved connections: onclose settles the reserved queue, later statements reject with CONNECTION_CLOSED instead of reaching nextWrite on the null socket, and release() does nothing after a close.

#1248 (comment)

fix(database): survive a connection lost mid-transaction; keep loopba…

refactor(database): move from postgres.js to node-postgres and drop i…

fix!: close the full-codebase review findings across security, billin…

fix(driver): send nothing more once a session's backend has ended (#209)

v4

Navigation Menu

Repository navigation

nextWrite() throws an uncaughtException when a reserved connection's backend dies #1208

Version

What happens

Minimal reproduction

Why

Suggested fix

Checked for duplicates

Correction to the snippet I filed

Same crash, reached through `begin()` rather than `reserve()`

Minimal reproduction

One difference from your results

Independent production case: the trigger is process shutdown

Your settle-vs-drop warning is correct, and we measured it

Reproduction without a PostgreSQL server

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

nextWrite() throws an uncaughtException when a reserved connection's backend dies #1208

Description

Version

What happens

Minimal reproduction

Why

Suggested fix

Checked for duplicates

Activity

XavierGeerinck commented on Sep 2, 2026

Correction to the snippet I filed

aeris-aios commented on Sep 2, 2026

Same crash, reached through begin() rather than reserve()

Minimal reproduction

One difference from your results

daniellmartins commented on Oct 1, 2026

xicopac commented on Oct 3, 2026

Independent production case: the trigger is process shutdown

Your settle-vs-drop warning is correct, and we measured it

Reproduction without a PostgreSQL server

oscarsovino commented on Oct 3, 2026

ckaraca commented on Oct 6, 2026

shiftynick commented on Oct 6, 2026

zakriyarahman commented on Oct 8, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Same crash, reached through `begin()` rather than `reserve()`