Repository navigation
perf(scheduler): publish an update only when addTask changes the first task or it is due - #67
Merged
Merged
Conversation
…t task or it is due
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every
addTaskpublishedupdateonSchedulerUtil, and every listener process answered each message with anupdateTimerround trip (ZRANGE). With L listeners, N adds cost N × L extra Redis round trips and listener CPU. Most of those messages changed nothing, because the new task was due after a first task that already had a timer leading to it.What changes
addTaskruns one Lua script,AddTaskScript:ZADD, then a read of the first member and its score. It publishesupdateonly when the new member is now first, or the first member is already past due. The!lockedrule is unchanged.addTaskstill wakes the listeners.PublishAfterWriteinAddTaskZaddandRescheduleZadd. A tie on score may publish or not. The model'sindex.ts:NNreferences are moved to the new lines.skills/redis-scheduler-interface/SKILL.mddescribe whenaddTaskpublishes.flowchart TD A["addTask"] --> S["AddTaskScript: ZADD, read first member and score"] S -->|"new member is first, or first is past due"| P{"locked?"} S -->|"otherwise"| N["no message: the first task's timer or batch ends in updateTimer, which finds the new task"] P -->|no| U["PUBLISH update: each listener runs updateTimer"] P -->|yes| N2["no message: this process's batch ends in updateTimer"]Measurement
Workload: one producer calls
addTask10,000 times, 100 at a time, against a local Redis 7 container. Each listener is a separate Node process that calledenableListener. Tasks are due an hour out, so no handler runs. The realistic shape isDate.now() + delay, so each new task is due after the queue head. Redis commands and CPU come fromINFO all, and process CPU fromprocess.cpuUsage(). There were 7 interleaved runs per side on a shared server, load average 3 to 5. Results are medians, with the range in brackets. Every run counted 10,000 queued members at the end, so there were 0 errors.Evidence level: confirmed through benchmarks.
The other orders, 4 listeners each:
PUBLISHis gone.Serial
addTasklatency did not change: p50 57–66 µs on both sides over 3,000 calls.Limiter. On main, listeners spend their CPU parsing one
ZRANGEreply per message per process. With this PR, adds are bound by the producer's round trip to Redis, and listeners sit idle.Cumulative impact, estimated: about 24 µs of listener CPU saved per add per listener, plus L + 1 Redis commands. At 1,000 adds/s with 4 listeners that is about 0.1 CPU-s/s across the listeners, and 5,000 fewer Redis commands/s.
Proof
New tests in
src/tests/index.test.ts, run withpnpm run build && pnpm exec ajs module test . -f dist/tests/index.test.js:should publish an update only when the task becomes the first in the queuefails on main'saddTask, which publishes every time:should publish an update for a later task while the first task is overduefails on a version that publishes only for a new first member. That was this branch's first draft, and review caught that it strands a due task after a rejected batch.After, at this head:
Formal check:
node .agents/skills/formal-scheduler-delivery/check.mjs, with and without--full, exits 0. Every rule holds in every configuration, and the record matches. On scratch copies of the model:PublishAfterWritealways{FALSE}givesBROKEN NoLostTaskin 8 states (CI) and 11 (full).pnpm run checksprintsChecks: 11 passed, 0 failed.Attention
updatemessage heals later than on main in one case. Suppose the first task is not yet due and its message is lost. Main healed at the nextaddTaskof any kind. This PR heals at the nextaddTaskafter that task falls due, or when a listener's timer next fires. The reviewer's probe gaveH@412on main andH@3004here, when the only later add came before H was due. Pub/sub was already at most once on main.EVAL. Listeners already did, throughTakeDueTasksScript. An ACL or proxy that blocks scripts for producer-only processes would breakaddTask.