Skip to content

Validate idle sessions with buffered non-blocking reads - #14

Merged
quinnj merged 6 commits into
mainfrom
dead-peer-validation
Sep 22, 2026
Merged

quinnj merged 6 commits into
mainfrom
dead-peer-validation

Conversation

@quinnj

@quinnj quinnj commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Closes #13. Detects idle-terminated sessions before pool reuse, without sending a probe query or retrying any user operation.

isopen reports local socket state. A server FATAL and EOF can already be waiting in the receive queue while that state still says open. Checking for one pending byte does not solve this safely: the rest of the PostgreSQL message or encrypted record may not have arrived.

Postgres.isvalid(conn) now collects available input through Reseau's non-blocking tryread! APIs. Validation and all normal post-authentication reads share one buffered stream, including startup session settings and reconnects. Only complete PostgreSQL messages are parsed. Partial messages remain available to a later validation, query, or notification read. GSS-encrypted frames also preserve partial headers and bodies.

  • Fatal server messages, EOF, invalid framing, and transport failures make validation return false and close the transport.
  • Notices, notifications, and parameter changes are handled normally. An in-use connection is left untouched. A true result means no known failure was observed; it is not proof of server liveness.
  • Validation collects at most 128 KiB per call. Pool checkout takes exclusive ownership first, then validates outside the shared pool lock, so a waiting user callback cannot stall other checkouts. Exceptions return the pool permit.
  • Postgres.isvalid is qualified public API, not exported and not an extension of Base.isvalid. isopen keeps its existing local-state meaning.
  • The exact-read helper retained in this PR rejects truncated message bodies instead of parsing short input as valid data.

Validation:

  • Full local Docker suite passed 6,404 checks on macOS / Julia 1.12.6 with two threads and coverage, including plain TCP, TLS, GSS, dead-peer replacement, session setup, transactions, COPY, and reconnects.
  • Final focused tests passed another 151 assertions covering fragmented GSS input and EOF, partial PostgreSQL messages handed to normal reads, independent pool checkout during a waiting callback, and permit cleanup after a throwing callback.
  • Local documentation build succeeded. All 16 CI checks pass at 3013885, including cross-platform tests, PostgreSQL-version tests, documentation, and coverage checks.

Depends on JuliaServices/Reseau.jl#163. Version remains 2.2.0. The four temporary CI installs of Reseau's nonblocking-peek branch must be removed after Reseau 1.5.0 is registered.

Co-authored by Codex

quinnj and others added 3 commits September 21, 2026 12:27
`isopen(conn)` reflects only the local socket. When the server terminates a
session that is sitting idle (`pg_terminate_backend`, a restart, failover,
`idle_session_timeout`), it sends a FATAL ErrorResponse and closes, and
those bytes wait unread in the socket. The pool validated idle connections
with `isopen`, so it handed the dead connection to the next borrower, who
absorbed the failure (#13).

Add `isvalid(conn::Connection)`: read whatever the server has already sent
without blocking and without sending anything. Asynchronous notices,
notifications, and parameter changes are dispatched to the style callbacks;
a FATAL, an unexpected message, a read failure, or a closed stream closes the
socket and reports the connection dead; nothing pending means alive. The
connection lock is held, so a statement in flight is never disturbed. The
pool checks with `isvalid` on acquire and release, so a terminated idle
connection is replaced rather than reused. Nothing is ever retried.

The non-blocking peek behind this is `Reseau.TCP.pending_input` and
`Reseau.TLS.pending_input` (Reseau 1.5.0); the GSS transport gets the same
contract on top of its plaintext buffer.

Version 2.2.0. The CI steps that install the unreleased Reseau branch are
temporary until Reseau 1.5.0 is registered.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`read(socket, len)` returns short when the peer closes mid-message (the Base
contract), so a message whose body never fully arrived parsed as a valid one:
an ErrorResponse or NoticeResponse with missing fields, a DataRow decoded
from a partial buffer, a partial COPY chunk. Three parsers already guarded
this by hand; the others did not, and `isvalid` reading a truncated notice
dispatched an empty notice before noticing the EOF.

One helper, `readbody`, now does every body read: it closes the socket, whose
stream position is unknowable after a short read, and throws. The three
hand-written guards are replaced by it. The SASL authentication exchange
keeps its plain reads: a truncated token fails the handshake regardless.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A live PostgreSQL cannot be made to send, on demand, the message shapes
`isvalid` has branches for: a notice or parameter change while idle, a
warning followed by a bare close (what an immediate shutdown or a crash of
another backend produces), a message that cannot occur on an idle session,
a truncated message, or a close with no bytes at all. Drive them from the
fake server used by the GSSAPI tests, on plain TCP, and check that live
connections stay valid with their callbacks fed, and that every dead shape
reports false and closes the socket.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@quinnj quinnj changed the title Detect idle-terminated sessions with isvalid and use it in the pool Validate idle sessions with buffered non-blocking reads Sep 21, 2026
quinnj and others added 2 commits September 21, 2026 15:23
The four temporary CI steps that installed Reseau from the nonblocking-peek
branch are no longer needed: Reseau 1.5.0 carries the non-blocking tryread!
APIs and is registered, and the compat bound already requires it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@quinnj
quinnj merged commit 83fe9e6 into main Sep 22, 2026
16 of 30 checks passed
@quinnj-bot quinnj-bot Bot mentioned this pull request Oct 27, 2026
3 of 10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

isopen reports a terminated connection as healthy until the next query fails

1 participant