Skip to content

fix(core): show flows live and keep finished flows listed - #53

Open
omarzeineddine-ai wants to merge 2 commits into
pontusab:mainfrom
omarzeineddine-ai:fix/live-flow-discovery
Open

fix(core): show flows live and keep finished flows listed#53
omarzeineddine-ai wants to merge 2 commits into
pontusab:mainfrom
omarzeineddine-ai:fix/live-flow-discovery

Conversation

@omarzeineddine-ai

Copy link
Copy Markdown

Problem

Closes #26

Two users report the flows page stays empty while flows run, showing them only after completion. Reproduced exactly: with a flow whose children take 20s, the parent is visible as waiting-children in the jobs API while /api/flows keeps returning [].

Two defects compound:

  1. Pinned cache. getFlows results are cached for 2 minutes on an LRU with updateAgeOnGet: true, while the UI polls /api/flows every 5 seconds. Each poll resets the entry's age, so the first snapshot — usually empty — never expires. The page only changes after a server restart or a >2-minute pause in polling (navigate away, come back), by which time the flow has completed. That's precisely both reporters' experience.
  2. Over-aggressive queue prefilter. The per-queue gate only counted waiting/waiting-children/active jobs, while the job scan below deliberately fetches completed/failed/delayed jobs too. Once a flow finishes and its queues go idle, every queue is skipped and the flow vanishes from the list. The pinned cache masked this; fixing (1) alone would have made completed flows visibly disappear seconds after finishing.

Solution

  • updateAgeOnGet: false — cache entries expire on their TTL instead of being pinned by reads.
  • Flows cache TTL 2m → 5s, matching the UI's poll cadence (queue counts underneath are already guarded by the 5s count cache).
  • The prefilter now skips only truly empty queues.

Note: the updateAgeOnGet line is the same one-line change #41 makes for the overview page — the two PRs merge cleanly in either order and each stays correct standalone.

Verification

Live flow with three 20-second children against docker Redis:

  • Before: /api/flows returns [] while the parent sits in waiting-children.
  • After: the flow appears within one 5s poll of creation (waiting-children, 0/4), updates, and remains listed as completed 4/4 after finishing.
  • bun test, tsc --noEmit, and biome pass.

No unit test added: the path is Redis-backed end to end and the repo has no BullMQ mocking infrastructure — the verification above is the regression check.

Two defects hid in-flight flows from the flows page (pontusab#26):

- The flows result was cached for 2 minutes on an LRU with
  updateAgeOnGet: true, while the UI polls /api/flows every 5s. Each
  poll reset the entry's age, so the first snapshot (usually empty)
  was pinned forever and the page only changed after a server restart
  or a >2min pause in polling — by which time flows had completed,
  matching both user reports. Stop resetting TTL on access and align
  the flows TTL with the UI's 5s poll.

- The per-queue prefilter only considered waiting/waiting-children/
  active counts, while the job scan below deliberately includes
  completed and failed jobs. Once a flow finished and its queues went
  idle, the filter skipped every queue and the flow vanished from the
  list. Skip only truly empty queues instead.

Verified against a live flow with 20s children: the flow now appears
within one 5s poll of creation (status waiting-children) and remains
listed as completed 4/4 after it finishes.

Closes pontusab#26
@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

@omarzeineddine-ai is attempting to deploy a commit to the Pontus Abrahamsson's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flows do not show up live

1 participant