fix(alerts): inherit the queue's Redis connection in the alert store - #52
Open
omarzeineddine-ai wants to merge 3 commits into
Open
fix(alerts): inherit the queue's Redis connection in the alert store#52omarzeineddine-ai wants to merge 3 commits into
omarzeineddine-ai wants to merge 3 commits into
Conversation
The Redis alert store built its client by spreading the first queue's
connection object into new Redis({...}). BullMQ also accepts live
ioredis Redis/Cluster instances as a connection — spreading one of
those produces a client with default options that dials localhost:6379
instead of the queue's server. With nothing listening there, ioredis
retries forever and every failure surfaces as an '[ioredis] Unhandled
error event: ECONNREFUSED' flood (no error listener was attached);
with an unrelated Redis on 6379, alert config was silently persisted
to the wrong server.
Live instances are now duplicate()d so the store inherits the exact
connection config, and the client gets an error listener that warns
once and keeps retrying quietly — alert persistence is non-critical
and recovers on its own when Redis becomes reachable.
Closes pontusab#33
Regression tests for pontusab#33: live ioredis instances are duplicated rather than spread into options, an error listener is attached so connection failures are never unhandled, repeated errors warn exactly once, and host/port/url option shapes are preserved.
|
@omarzeineddine-ai is attempting to deploy a commit to the Pontus Abrahamsson's projects Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Closes #33
Reported as an ioredis error flood in
@getworkbench/fastify@0.9.1, but the adapter isn't at fault — and the regression window isn't 0.8.0→0.9.1 in git. The publishedfastify@0.9.1pinscore@0.8.0whilefastify@0.8.0pinscore@0.6.0, so downgrading the adapter actually rolls core back past 0.8.0 — the release that introduced Redis-backed alert persistence. That's where the bug lives.RedisAlertStorebuilds its client from the first queue's connection by spreading it intonew Redis({...}). BullMQ also accepts live ioredisRedis/Clusterinstances as a connection. Spreading an instance produces a client with default options that dialslocalhost:6379instead of the user's server, and noerrorlistener was ever attached, so:localhost:6379→ ioredis retries forever, every failure printed as[ioredis] Unhandled error event: AggregateError [ECONNREFUSED]— the exact flood in the issue screenshot. The dashboard keeps working because the queues use the real connection; only the alert store's derived client is broken.localhost:6379→ worse: alert contact points/rules are silently persisted to the wrong server.Solution
Redis/Clusterinstances are nowduplicate()d, inheriting the exact connection config (same approach BullMQ uses internally). String URLs, options objects, and{ url }objects behave as before.errorlistener: it warns once ([workbench] Alert store Redis connection error (will keep retrying quietly): ...) and retries silently — alert persistence is non-critical and recovers on its own once Redis is reachable.Verification
Reproduced end-to-end with the fastify adapter, a queue connected via an ioredis instance to Redis on
127.0.0.1:6390, and an unrelated Redis on6379:Unhandled error eventlines, exactly one workbench warning, server keeps serving.Unit tests (
redis-alert-store.test.ts) cover: instance →duplicate()(not option-spreading), error listener present (unlistenederroron an EventEmitter throws — the regression guard), warn-exactly-once on repeated failures, and host/port/urloption shapes preserved.bun test(21 pass),tsc --noEmit, and biome all pass.Note for the release
The stale inter-package pin is worth fixing in the pipeline too:
fastify@0.9.1shipped depending oncore@0.8.0, so users on "0.9.1" never actually got 0.9.x core code. This fix needs a core release and adapter releases that pin it.