Skip to content

fix(workflows): reload a saved workflow only after its commit - #581

Closed
Silthus wants to merge 4 commits into
masterfrom
email-v2/00-reload-after-commit
Closed

Silthus wants to merge 4 commits into
masterfrom
email-v2/00-reload-after-commit

Conversation

@Silthus

@Silthus Silthus commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Part of the code-first rebuild of the email send stack; replaces parts of PostHog#121867 (the workflow reload it needs, split out to land first).

Problem

Impact

Pays into Grow Workflows: an edit or a pause takes effect on the workers as soon as it commits. In scope: Workflows, every workflow save.

Merge Danger

Warning

🟡 Merge risk: medium · two-way door: a revert brings back the reload before commit

  • Not behind a flag. Every workflow save now reloads the CDP workers after its transaction commits, not before it.
  • A save inside an outer transaction reloads at the outer commit. A save in autocommit reloads at once, as before.
  • A bulk change under defer_hog_flow_reloads (UTM defaults, Customer.io import drafts) also reloads at the outer commit, not when its block ends.
  • A failed reload publish no longer fails the save, and no longer skips the commit callbacks after it.
  • Rebasing feat(workflows): show code-managed email templates read-only #575 to feat(messaging): mark a code send that leaves the sender to the project #578 onto this PR conflicts in transactional_email.py and in the tests that patch its reload. Those tests drop that patch.

Changes

Kind Files Lines
Production 6 +27 −61
Tests 5 +85 −26
  • A saved workflow reloads on the workers once the save commits. hog_flow_saved publishes the reload in transaction.on_commit, not from inside the save's transaction.
  • A bulk change reloads at the outer commit too. defer_hog_flow_reloads flushes its batch on commit, not when its block ends. It covers apply_utm_defaults and the Customer.io import drafts.
  • A failed reload publish never fails the save. Every reload goes through _reload_batch_on_workers: one retry, then a log line and a captured exception.
  • The extra after-commit reloads in sync_transactional_workflow, strip_placeholder_sender_overrides and rewrite_email_asset_url are gone, because the model covers them.
  • Mechanical: comments that named the old in-transaction publish now give the reason that still holds.

Saves that now reload at the outer commit, from transaction.atomic blocks on master:

  • services/hog_flow_writes.py: create, update, edit content, and publish a draft. A save that changes only draft and draft_updated_at never reloads.
  • services/hog_flow_revisions.py (restore) and services/workflow_proposals.py (apply).
  • Transactional email workflows and message template updates (sync_transactional_workflow).
  • The refresh_hog_flows, migrate_hog_flow_secret_inputs and strip_placeholder_sender_overrides commands.

The pause and resume writers and refresh_affected_hog_flows publish their own reload after commit, and do not change.

How did you test this code?

Test seams: the post_save and post_delete receivers (test_hog_flow.py), the transactional template PATCH endpoint (test_transactional_email.py), and the refresh_hog_flows command (test_refresh_hog_flows.py). Each runs the commit callbacks with captureOnCommitCallbacks(execute=True).

Test rationale, run red against master first:

  • An email pause save whose reload runs before the commit.
  • A transactional template update whose reload runs inside its transaction.
  • A failed reload publish, on save and on delete, that fails the save or skips the commit callbacks after it (test_a_failed_reload_publish_is_retried_reported_and_runs_the_callbacks_after_it).
  • A command that adds its own reload on top of the model's: one reload-hog-flows publish per affected row.

Proofs on this head 8551c2c (saved with fleet proof):

  • Live run: Django at this head, cdp-api and both CDP workers at the node stack f9abed0, maildev as the sink. 21 requests through the public API: template edits then sends by id, code sends, uploads. Each email carries the content just saved.
  • Shadow diff: the same 21 requests on master 43333f8 and on this head. No difference after ids and times are normalized.
  • Prod shape: a gap, accepted by the standing rule until read-only production access exists on 2026-11-02.
  • Lane gate on this head: ruff, the IDOR check, the migration checks, repo-wide mypy, and the messaging and workflows tests pass.
Earlier proofs, at f3d2122 and before the 2026-10-31 rebase
  • Live run at f3d2122, with cdp-api and both CDP workers at master 43333f8 (saved with fleet proof): PATCH a transactional template's subject, then send it by id at once, three rounds. Each email has the subject just saved.
  • Control: master gives the same 3 of 3 on this box, so the race did not reproduce. This run shows no regression, not the fix. The tests pin the order of reload and commit.
Earlier live run, before the 2026-10-31 rebase (same code)

Django at the pre-rebase head, cdp-api and the CDP workers at master, on data that master wrote first (saved with fleet proof). Each send runs about 3 ms after its save answers.

Case Save Send Delivered
Template update, three subjects in a row 200 202 Each email has the subject just saved
Workflow update, three subjects in a row 200 202 Each email has the subject just saved
Workflow set to draft, then archived 200 400 transactional_email_paused Nothing
Workflow set active again, after each 200 202 Yes
  • Worker logs: each save gives exactly one reload on each worker and on cdp-api, and it arrives before the save answers. On master the same save gives two: one before the commit and one after it.
  • Prod shape: a gap until read-only production access exists on 2026-11-02.

CI: the fork has no Depot runners, so those jobs never start, and no secret for the project-board app, so flags-project-board fails; the other GitHub-hosted checks pass. The lane check passes: the PR changes only the Python side.

Release status

  • No feature flag controls this change

Docs update

None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5)

Question Answer Who Where
Does the reload-after-commit fix land inside #578 or on its own? On its own, off master, landing before #580 and #578 👤 Human map #461, 2026-10-25
May the code carry comments? No; names carry the reasons 👤 Human map #461, 2026-10-25
What happens when a reload publish fails after commit? One retry, then a log line and a captured exception; the save still succeeds 🤖 Agent review round 2

Involvement: one decision from Michael; reviewed inside #578 in two rounds; QA swarm on 2026-11-01 with two panel rounds (GPT-6.1 Sol review, Opus 5.5 fresh eyes): 4 findings fixed, 2 rejected; live run and shadow diff captured.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

🤖 CI report

⚠️ Comment density — 6% of added code lines are comments (6 of 103)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File Comment lines Added lines
products/workflows/backend/services/workflow_email_health.py 4 4
products/workflows/backend/management/commands/rewrite_email_asset_url.py 2 3

This check does not block merging. It updates on every push and clears when the share drops.

Silthus and others added 3 commits October 31, 2026 08:18
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…loads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…that still holds

The post_save reload now publishes after commit, so the old reason no
longer matches the code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus force-pushed the email-v2/00-reload-after-commit branch from 7044941 to f3d2122 Compare October 31, 2026 09:08
A single save or delete now reloads through the same retry and error
capture as a deferred batch. The two management commands drop their own
after-commit workflow reload, because the model already publishes one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus

Silthus commented Nov 2, 2026

Copy link
Copy Markdown
Owner Author

Moved to PostHog#127170.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant