Skip to content

feat(messaging): send the workflow version a template send by key was checked against - #585

Closed
Silthus wants to merge 12 commits into
email-v2/11-first-code-send-eventfrom
email-v2/12-send-graded-template-version
Closed

Silthus wants to merge 12 commits into
email-v2/11-first-code-send-eventfrom
email-v2/12-send-graded-template-version

Conversation

@Silthus

@Silthus Silthus commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Part of the code-first rebuild of the email send stack; new in the rebuild, with no closed PR to replace: a template send delivers the content Django graded (decision D3 on map #461).

Problem

  • A template send on send/email/ right after an upload can deliver the previous content, while Django answered that the caller's template.hash is current.
  • Django grades the hash, but the email goes out from the workflow copy cdp-api and the worker cached, which can lag after a lost reload.
  • This is the Django half: it tells cdp-api which workflow version holds the graded content. The cdp-api and worker half is feat(cdp): send a template from the workflow version Django graded #584.

Impact

Pays into Grow Workflows through billed workflows_emails actions per paying org: a developer who uploads a template and sends it at once gets the new content. In scope: the send/email/ route.

Merge Danger

Note

🟢 Merge risk: low · two-way door: a revert stops sending the version, and delivery works as today

Changes

Kind Files Lines
Production 5 +56 −8
Tests 2 +88 −7
  • A template send posts the version of its workflow that holds the graded content, as send.hogFlowVersion. Django reads it after grading the hash, from the writer database, so it is never older than the graded content.
  • A workflow deleted between the grade and the version read answers as the cdp-api 404 does today (Unavailable to the caller), without a stack trace in the log.
  • A send that the email pipeline can't deliver yet gets a 503 that says so. When cdp-api still holds an older workflow version than the graded one, the caller gets master's 503 delivery_unavailable shape with its own detail:
{
  "type": "server_error",
  "code": "delivery_unavailable",
  "detail": "This template's newest version hasn't reached the email pipeline yet, so nothing was sent. Retry in a few seconds, with the same Idempotency-Key if you sent one.",
  "attr": null
}
  • Nothing was sent in that case, so a retry is safe with or without a key. The internal reason, workflow_version_behind, never reaches the caller.
  • Every other 503 keeps master's body, on both routes.
  • Every send by template.key posts the version, with or without from, reply_to or capture_engagement, so it goes to the accepted-send route. Its default captureEngagement: true changes nothing: the project's setting still decides.
  • A send by template.id keeps master's route, which grades no hash and posts no version. One that sets capture_engagement (feat(messaging): send a stored template by key on send/email #574), which takes workflows-email-from-code, goes to the accepted-send route and posts the version, like a send by template.key. Only a field the caller sent counts, so a form body that omits it keeps master's route.
  • No serializer or OpenAPI change, so no regenerated types.

How did you test this code?

Test seams: the DRF client on send/email/, asserting the body posted to cdp-api (test_transactional_email.py).

Test rationale, each run red first:

  • A send by template.key that posts no version, with or without the send options, or a version from before a save. A PATCH that saves new content is followed by a send that must post the new version.
  • A version read from a lagging read replica.
  • A workflow deleted before the version read that logs an exception.
  • The three rendered-content rows compare the whole send dict, which pins that the field is absent there.
  • A version-behind 503 from cdp-api that answers the generic 503, or that puts workflow_version_behind anywhere in the body.

Proofs (saved with fleet proof on ef44c9b):

Earlier live run, 2026-10-31

Django at 3a30f51c529, cdp-api and both CDP workers at #584 (f9abed0841c), mail caught in maildev.

Send by template.key Result
Current hash, right after an upload changed the content 202, the new content delivered
Superseded hash 202 with the template_superseded warning, the new content delivered
A hash PostHog doesn't hold 409 template_outdated, nothing sent
With from, reply_to and capture_engagement 202, sent from support@example.com
posthog-node local template (Silthus/posthog-js#7), new key 409, upload, then 202

CI: the fork has no Depot runners, so those jobs never start. Semgrep Checks Pass fails on every fork PR, because its path filter job runs only for the PostHog owner; semgrep with the repo's rules passes locally on the changed files. The lane check reports cross_lane=false.

Release status

  • This change is behind a feature flag and is not available to users

Docs update

None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5)

Question Answer Who Where
Does a template send deliver what Django graded? Yes, through #584 and this PR 👤 Human map #461, decision D3 of 2026-10-25
May the code carry comments? No; names carry the reasons 👤 Human map #461, 2026-10-25
What does a send get when the pipeline holds an older version? The contract's 503 delivery_unavailable with the version lag in detail; replaces a new public workflow_version_behind code 👤 Human approved (conductor, 2026-10-26)
Does a 503 carry why and fix? No; this stack reuses master's refusal shape, so the lag goes in detail 👤 Human map #461, 2026-10-28
Key the cache on the hash, or on a version floor? A version floor, read after grading from the writer; replaces D3's "key the cache on the hash" 👍 Accepted default approved (conductor, 2026-10-26)
Does a send by template.id with capture_engagement post the version? Yes, it takes the accepted-send route like a key send; other id sends keep master's route 🤖 Agent dj3 QA swarm, 2026-11-01
Must #585 deploy with #583? Yes; apart, a keyed retry of a key send answers 422. Kept as a stated condition, not code 🤖 Agent dj3 QA swarm, 2026-11-01
Must the retry reuse the key? Only if the caller sent one: nothing was sent, so a retry without a key is safe too 👍 Accepted default approved (conductor, 2026-10-26)

Open: a send by template.id posts no version, so one right after an upload can still use the cached copy. Recommendation: leave it; that send grades no hash.

Involvement: one decision from Michael and one from the map's conductor; review round 1 (GPT-6.1 Sol, Opus fresh eyes) fixed, then final reads by GPT-6.1 Sol; dj3 round 1 (GPT-6.1 Sol) fixed, its final read raised the cdp-api route order, answered in the merge risk; live run captured; cross-version proof in #584. dj3 QA swarm (GPT-6.1 Sol review, Opus fresh eyes): one merge fix, two rejections; live run, shadow diff and cross-version on this head.

🤖 Generated with Claude Code

@Silthus
Silthus force-pushed the email-v2/11-first-code-send-event branch from 42bfa91 to 1867b69 Compare October 31, 2026 14:21
@Silthus
Silthus force-pushed the email-v2/12-send-graded-template-version branch from 99d058f to 5117bfd Compare October 31, 2026 14:30
@Silthus
Silthus force-pushed the email-v2/11-first-code-send-event branch from 1867b69 to 5e72c0b Compare October 31, 2026 14:31
@Silthus
Silthus force-pushed the email-v2/12-send-graded-template-version branch from 5117bfd to 8e26418 Compare October 31, 2026 14:32
@Silthus Silthus changed the title feat(messaging): send the workflow version a template send graded feat(messaging): send the workflow version a template send by key was checked against Oct 31, 2026
@Silthus
Silthus force-pushed the email-v2/12-send-graded-template-version branch 2 times, most recently from 061e4b8 to 5b09e8f Compare October 31, 2026 15:50
Silthus and others added 2 commits October 31, 2026 15:54
… checked against

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 6 commits November 1, 2026 17:54
…d-graded-template-version

# Conflicts:
#	products/messaging/backend/facade/transactional_email.py
#	products/messaging/backend/presentation/views/message_templates.py
…s capture_engagement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d-graded-template-version

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d-graded-template-version

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus added a commit that referenced this pull request Nov 2, 2026
#585 reports a project's first code send through one operation; the merge keeps that operation and passes it this part's delivery_mode property.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 4 commits November 2, 2026 01:12
…d-graded-template-version

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d-graded-template-version

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…end sets capture_engagement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus

Silthus commented Nov 2, 2026

Copy link
Copy Markdown
Owner Author

Moved to PostHog#127188.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant