stat_replication: fix "slot_name does not exist" by joining pg_replication_slots#1313
Open
megative wants to merge 1 commit into
Open
stat_replication: fix "slot_name does not exist" by joining pg_replication_slots#1313megative wants to merge 1 commit into
megative wants to merge 1 commit into
Conversation
…ation_slots The stat_replication collector currently fails every scrape with: pq: column "slot_name" does not exist slot_name is not a column of pg_stat_replication on any supported PostgreSQL version (verified on PG 17.10.0 Docker, PG 18 Docker, and the official PG 17 and PG 18 docs). It lives on pg_replication_slots and has to be pulled in via a JOIN on active_pid = pid. Before prometheus-community#1306 the same collector ran via the legacy yaml-driven exporter, which used SELECT * and silently produced an empty slot_name label when the column was missing. prometheus-community#1306 promoted slot_name into an explicit SQL column list, turning that silent no-op into a hard parse-time error. CI did not catch it because the unit tests use sqlmock and do not validate against the real Postgres schema. stat_replication is defaultEnabled, so every user on main after prometheus-community#1306 sees the collector dead on every scrape, regardless of whether they run Aurora or vanilla Postgres. Fix: LEFT JOIN pg_replication_slots s ON s.active_pid = pg_stat_replication.pid and read s.slot_name. The label is populated when a named slot is in use, empty otherwise (matching the previous yaml behavior). Fixes prometheus-community#1310. Signed-off-by: Pavel K <megativ3@gmail.com>
4 tasks
ArthurSens
requested changes
Jul 4, 2026
ArthurSens
left a comment
Contributor
There was a problem hiding this comment.
Thanks a lot for the fix, and apologies for not seeing this PR before making the 0.20.0 release.
There's still some small work to be done to make sure this also works with older versions of PG.
Could you also rebase the PR?
| s.slot_name, | ||
| (case pg_is_in_recovery() when 't' then pg_xlog_location_diff(pg_last_xlog_receive_location(), replay_location)::float else pg_xlog_location_diff(pg_current_xlog_location(), replay_location)::float end) AS pg_xlog_location_diff | ||
| FROM pg_stat_replication | ||
| LEFT JOIN pg_replication_slots s ON s.active_pid = pg_stat_replication.pid |
Contributor
There was a problem hiding this comment.
Seems like this query only works for postgres 9.5 and above, because active_pid was only added in PG 9.5.x. See https://www.postgresql.org/docs/9.5/view-pg-replication-slots.html and https://www.postgresql.org/docs/9.4/catalog-pg-replication-slots.html
Could you please have a query for 9.4 and below that doesn't include slot_name?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1310. The
stat_replicationcollector currently fails every scrape withpq: column "slot_name" does not exist— affects every user onmainafter #1306, not just Aurora (despite the issue title).The fix is a
LEFT JOIN pg_replication_slots s ON s.active_pid = pg_stat_replication.pidso thatslot_nameis read from the view where it actually lives, with an empty label when no named slot is in use.Test plan
go build ./...,go vet ./...,go test ./...clean.pg_scrape_collector_success{collector="stat_replication"} = 1(was0).pg_scrape_collector_success{collector="stat_replication"} = 1(was0).Full timeline and root cause
Timeline
Before Refactor stat_replication into standalone collector #1306,
pg_stat_replicationwas scraped through the legacy yaml-driven exporter with aSELECT * FROM pg_stat_replicationquery. The yaml mapping listedslot_nameas a label, but the column does not actually exist onpg_stat_replication— so the yaml exporter silently produced an empty label. No error.slot_namehas never been a column ofpg_stat_replicationon any supported PG version. It lives onpg_replication_slots. To populate it for an active WAL sender you have to JOIN the two views. The legacy yaml never did. The label was effectively aspirational — described, but always empty. A latent bug sat inqueries.yamlfor years.Refactor stat_replication into standalone collector #1306 ("Refactor stat_replication into standalone collector") migrated this query into a Go-native collector. The new SQL was rewritten with an explicit column list, including
slot_name:That promoted
slot_namefrom "yaml label silently missing" to "explicit SQL column that must exist". It does not.CI on Refactor stat_replication into standalone collector #1306 went green because unit tests for
stat_replicationusesqlmock, which returns whatever columns the test declares — the real Postgres schema is never validated. Integration tests apparently do not exercise this collector. The bug shipped.After Refactor stat_replication into standalone collector #1306 merged,
stat_replicationfails on every scrape, on every PG version. Because it isdefaultEnabled, every user is affected automatically.Verification that
slot_nameis not onpg_stat_replicationThree independent sources:
\d pg_stat_replicationonpostgres:17.10.0Docker — 20 columns, noslot_name.\d pg_stat_replicationonpostgres:18Docker — 20 columns, noslot_name.slot_name. The PG 18 docs explicitly noteslot_namelives onpg_stat_replication_slots.Fix details
LEFT JOIN pg_replication_slots s ON s.active_pid = pg_stat_replication.pidand reads.slot_name. Behavior:slot_namepopulated.slot_nameempty (matches legacy yaml behavior).slot_nameempty.Applied to both the PG ≥ 10 and pre-10 query paths.
pg_replication_slotsexists since PG 9.4, so the pre-9.4 branch remains affected by the same root cause; that branch is not in CI scope and was already broken before this PR.