Skip to content

fix(alerts): inherit the queue's Redis connection in the alert store - #52

Open
omarzeineddine-ai wants to merge 3 commits into
pontusab:mainfrom
omarzeineddine-ai:fix/alert-store-redis-flood
Open

fix(alerts): inherit the queue's Redis connection in the alert store#52
omarzeineddine-ai wants to merge 3 commits into
pontusab:mainfrom
omarzeineddine-ai:fix/alert-store-redis-flood

Conversation

@omarzeineddine-ai

Copy link
Copy Markdown

Problem

Closes #33

Reported as an ioredis error flood in @getworkbench/fastify@0.9.1, but the adapter isn't at fault — and the regression window isn't 0.8.0→0.9.1 in git. The published fastify@0.9.1 pins core@0.8.0 while fastify@0.8.0 pins core@0.6.0, so downgrading the adapter actually rolls core back past 0.8.0 — the release that introduced Redis-backed alert persistence. That's where the bug lives.

RedisAlertStore builds its client from the first queue's connection by spreading it into new Redis({...}). BullMQ also accepts live ioredis Redis/Cluster instances as a connection. Spreading an instance produces a client with default options that dials localhost:6379 instead of the user's server, and no error listener was ever attached, so:

  • nothing on localhost:6379 → ioredis retries forever, every failure printed as [ioredis] Unhandled error event: AggregateError [ECONNREFUSED] — the exact flood in the issue screenshot. The dashboard keeps working because the queues use the real connection; only the alert store's derived client is broken.
  • an unrelated Redis on localhost:6379 → worse: alert contact points/rules are silently persisted to the wrong server.

Solution

  • Live Redis/Cluster instances are now duplicate()d, inheriting the exact connection config (same approach BullMQ uses internally). String URLs, options objects, and { url } objects behave as before.
  • The store's client always gets an error listener: it warns once ([workbench] Alert store Redis connection error (will keep retrying quietly): ...) and retries silently — alert persistence is non-critical and recovers on its own once Redis is reachable.

Verification

Reproduced end-to-end with the fastify adapter, a queue connected via an ioredis instance to Redis on 127.0.0.1:6390, and an unrelated Redis on 6379:

  • Before: creating a contact point through the dashboard API wrote to the 6379 instance; the queue's Redis on 6390 got nothing.
  • After: the same flow writes to 6390; 6379 is untouched.
  • After, with the queue's Redis unreachable: zero Unhandled error event lines, exactly one workbench warning, server keeps serving.

Unit tests (redis-alert-store.test.ts) cover: instance → duplicate() (not option-spreading), error listener present (unlistened error on an EventEmitter throws — the regression guard), warn-exactly-once on repeated failures, and host/port/url option shapes preserved. bun test (21 pass), tsc --noEmit, and biome all pass.

Note for the release

The stale inter-package pin is worth fixing in the pipeline too: fastify@0.9.1 shipped depending on core@0.8.0, so users on "0.9.1" never actually got 0.9.x core code. This fix needs a core release and adapter releases that pin it.

The Redis alert store built its client by spreading the first queue's
connection object into new Redis({...}). BullMQ also accepts live
ioredis Redis/Cluster instances as a connection — spreading one of
those produces a client with default options that dials localhost:6379
instead of the queue's server. With nothing listening there, ioredis
retries forever and every failure surfaces as an '[ioredis] Unhandled
error event: ECONNREFUSED' flood (no error listener was attached);
with an unrelated Redis on 6379, alert config was silently persisted
to the wrong server.

Live instances are now duplicate()d so the store inherits the exact
connection config, and the client gets an error listener that warns
once and keeps retrying quietly — alert persistence is non-critical
and recovers on its own when Redis becomes reachable.

Closes pontusab#33
Regression tests for pontusab#33: live ioredis instances are duplicated rather
than spread into options, an error listener is attached so connection
failures are never unhandled, repeated errors warn exactly once, and
host/port/url option shapes are preserved.
@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

@omarzeineddine-ai is attempting to deploy a commit to the Pontus Abrahamsson's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flood of ioredis error messages in @getworkbench/fastify@0.9.1

1 participant