Skip to content

fix(cdp): record the failed run row for jobs read from kafka - #130999

Open
CShine612 wants to merge 1 commit into
masterfrom
claude/cdp-kafka-scheduled-at-6a94be
Open

CShine612 wants to merge 1 commit into
masterfrom
claude/cdp-kafka-scheduled-at-6a94be

Conversation

@CShine612

Copy link
Copy Markdown
Contributor

Problem

  • Destination runs on the Kafka queue lose their failed row when the function is disabled or deleted between fetch retries.
  • Building that row throws toJSDate is not a function (error tracking issue).

Changes

  • These runs now get their failed row, with the scheduled retry time.
  • A new deserializeInvocation revives queueScheduledAt from an ISO string into a UTC DateTime on Kafka reads.
  • The call-site change in consumeKafkaBatch is mechanical.

How did you test this code?

Test rationale: two rows in job-queue-kafka.test.ts round-trip a set and an unset queueScheduledAt, which no existing test covers.

  • On a local stack with real Kafka, the CDP worker and ClickHouse, a destination disabled before its retry now gets its failed row with the retry time.
  • Master's worker throws the toJSDate error there and writes no failed row.
  • Not checked: the run history UI, a deleted function, more than one retry.

Release status

  • No feature flag controls this change

Docs update

  • None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5

Skills: /writing-tests, /writing-code-comments, /simplify, /code-review, /qa-team, /writing-pr-descriptions, Review Hog lenses

  • No open PR changes these lines. The test data is invented.
More context for AI agents
  • Session: Claude Code desktop, local worktree; tools: jest, git, gh. Started from an error tracking sweep for self-serve bugs.
  • Decisions: revive at the Kafka parse boundary, not in the results service, so every reader gets a DateTime.
  • Decisions: the parse-boundary fix stays clear of draft feat(cdp): send a template from the workflow version Django graded #127130, which changes other parts of hog-invocation-results.service.ts.
  • Decisions: the bytes on the topic do not change, so old and new pods can run side by side.
  • Known limitations: a malformed ISO string becomes an invalid DateTime, but the producer cannot write one. An explicit null stays null.
  • Known limitations: the test calls deserializeInvocation, not consumeKafkaBatch.
  • Follow-ups: a typed wire format for serializeInvocation, so a new DateTime field cannot come back as a string without a type error.
  • Follow-ups: on Kafka, the worker cannot keep a job whose terminal row fails to produce, because dequeue is a no-op there.
  • Read closely: deserializeInvocation in nodejs/src/cdp/services/job-queue/job-queue-kafka.ts.

🤖 Generated with Claude Code

The Kafka job queue writes invocations with JSON.stringify, so a Luxon
DateTime in queueScheduledAt comes back as an ISO string. Recording a
terminal failure for such an invocation then threw "toJSDate is not a
function", and the failed lifecycle row was never written.

Add deserializeInvocation next to serializeInvocation and turn the
string back into a UTC DateTime there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CShine612 CShine612 self-assigned this Nov 6, 2026
@trunk-io

trunk-io Bot commented Nov 6, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

@CShine612
CShine612 marked this pull request as ready for review November 6, 2026 16:44
@stamphog
stamphog Bot requested a review from a team November 6, 2026 16:44
@parameterai

parameterai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Risk: No findings

This PR fixes a crash in the CDP Kafka job consumer by reviving the serialized queueScheduledAt ISO string into a luxon DateTime at the parse boundary, so terminal "failed" run rows can be written when a destination is disabled between retries. The data on this topic is produced exclusively by internal CDP services with server-computed timestamps, so no attacker-controlled input crosses a trust boundary and no security risk is introduced.

Sentinel reviewed 21bbcb9 · Review settings

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant