We run smartsync-tuning self-hosted against Postgres (RDS, PG15). After moving our full production traffic onto it, the Postgres connection count kept climbing until it hit max_connections and the DB started refusing new connections (sorry, too
many clients already and remaining connection slots are reserved...). The logs were also full of could not receive data from client: Connection reset by peer.
It traces back to the query Adapter opening a new connection for every operation and never closing it. In internal/pkg/query/datatube.go, InsertLog, runQuery, and GetNumOfRequests all do:
conn, err := qa.driver.Open(qa.dbName)
with no matching conn.Close() anywhere. Since the Adapter holds a raw driver.Driver rather than a *sql.DB, there's no pool reaping idle connections either, so the only thing that closes these sockets is the GC finalizer once the conn goes out of
scope.
I confirmed it by enabling log_disconnections. For single-row inserts the session times never drop below ~9s, average around 27s, and bunch up between 15 and 45s. That's GC timing, not query time. At our insert rate (~29/sec) that works out to
800+ connections open at once, which is what hits the ceiling. They're also closed with a TCP reset instead of a clean terminate, which is consistent with finalizer cleanup rather than an explicit close.
The fix should be simple: close the connection at each of those call sites (defer conn.Close() after Open), or better, build a single shared *sql.DB at startup and let database/sql pool for you. Either one drops connection lifetime to milliseconds
and keeps the open count low regardless of load.
Happy to test a patch against our setup.
We run smartsync-tuning self-hosted against Postgres (RDS, PG15). After moving our full production traffic onto it, the Postgres connection count kept climbing until it hit max_connections and the DB started refusing new connections (sorry, too
many clients already and remaining connection slots are reserved...). The logs were also full of could not receive data from client: Connection reset by peer.
It traces back to the query Adapter opening a new connection for every operation and never closing it. In internal/pkg/query/datatube.go, InsertLog, runQuery, and GetNumOfRequests all do:
conn, err := qa.driver.Open(qa.dbName)
with no matching conn.Close() anywhere. Since the Adapter holds a raw driver.Driver rather than a *sql.DB, there's no pool reaping idle connections either, so the only thing that closes these sockets is the GC finalizer once the conn goes out of
scope.
I confirmed it by enabling log_disconnections. For single-row inserts the session times never drop below ~9s, average around 27s, and bunch up between 15 and 45s. That's GC timing, not query time. At our insert rate (~29/sec) that works out to
800+ connections open at once, which is what hits the ceiling. They're also closed with a TCP reset instead of a clean terminate, which is consistent with finalizer cleanup rather than an explicit close.
The fix should be simple: close the connection at each of those call sites (defer conn.Close() after Open), or better, build a single shared *sql.DB at startup and let database/sql pool for you. Either one drops connection lifetime to milliseconds
and keeps the open count low regardless of load.
Happy to test a patch against our setup.