[Amperstrand]

What

repro-check has failed on every main push since #448 merged the audit step. Live failure signature (kind-9841, fd582973 / d622688b):

build-env: ERROR: SOURCE_DATE_EPOCH not set and no git history
available to derive it; export it explicitly
❌ Failure - Main Audit the SDK Go pins against the live OpenWrt feeds

The audit runs scripts/sdk-go-version.sh check → sources packaging/build-env.sh, which requires SOURCE_DATE_EPOCH. The lane already contains a derivation chain — but only inside the later repro-build step; act checkouts have no usable .git, so nothing before the audit can supply it. #448's branch never exercised this (#473: ngit never ran on it), so the audit shipped blind and broke the lane it guards.

Change

One new step before the audit: the same derivation chain the repro-build step uses (git → head_commit.timestamp → job clock), exported via GITHUB_ENV. The repro-build step's existing fallback chain is untouched — it now finds the variable already set.

Verification

• SOURCE_DATE_EPOCH=<t> bash scripts/sdk-go-version.sh check → "SDK Go pins match the live OpenWrt feeds" (the exact env this step creates)
• YAML parse clean; make go-battery in progress on the branch
• The GitHub twin is unaffected — its runners have git metadata, so build-env.sh derives the epoch there (its audit has been green)

Note for whoever merges: this and #477's node-pin half are the two repo-side reds on main today; both exist because their PRs merged without runner evidence — #473 remains the forcing function for that class.

[Amperstrand]

Verified at head bcc7f3b9 in a clean worktree (no rebase, no push). Diff is YAML-only; make go-battery on the branch: 16 modules green, exit 0 — branch soundness only, the fix evidence is below.

Failure mechanism — reproduced locally, confirmed live on the relay

ngit-ci checkouts carry no .git at all (.ngit/README.md: git rev-parse reports not a git repository — not a shallow clone, an absent one). The audit step #448 added runs scripts/sdk-go-version.sh check, which sources packaging/build-env.sh before doing anything else (sdk-go-version.sh:43-46). The epoch guard (build-env.sh:94-108) finds SOURCE_DATE_EPOCH unset — nothing earlier in the job sets it; the lane's only derivation chain lived inside the later repro-build step, which never executes because the audit dies first — and git -C "$TG_ROOT" rev-parse HEAD fails in a .git-less checkout, so line 103 fires the exact live error.

Live corroboration (kind 9841 on relay.ngit.dev, w=.ngit/act/workflows/repro-check.yml): the binaries job has conclusion=failure on 8 consecutive main pushes (f8a42fa2, 8e7a2e5c, fd582973, d622688b, dbd0777c, 22edca2e, 6b6c11e8, 596b9e98), each log tail ending:

build-env: ERROR: SOURCE_DATE_EPOCH not set and no git history available to derive it; export it explicitly
❌  Failure - Main Audit the SDK Go pins against the live OpenWrt feeds

The last success (fd3c0630) predates the audit step (750a2ec5, #448) reaching main pushes. Deterministic local reproduction: git archive export of the head with no .git, then

$ env -u SOURCE_DATE_EPOCH sh scripts/sdk-go-version.sh check
build-env: ERROR: SOURCE_DATE_EPOCH not set and no git history available to derive it; export it explicitly   # exit 1

— byte-for-byte the runner signature.

The fix, verified at head — resolution logic both ways, then the audit's own sub-path

I extracted the new step's run: block verbatim (modulo act's pre-substitution of ${{ github.event.head_commit.timestamp }}) and ran it against the no-.git copy:

• push shape (timestamp 2026-09-22T17:52:51+02:00, the PR head's own commit time): writes SOURCE_DATE_EPOCH=1790092371 to $GITHUB_ENV, exactly date -u -d <ts> +%s. ✔
• pull_request shape (expression renders empty — github.event.head_commit doesn't exist on PR events): the [ -n "" ] guard skips, the NOTE prints, the job clock pins it; output still a valid integer. ✔
• the previously-failing command, in the exact env this step creates — the full audit sub-path against the live feeds, not just the guard:

$ SOURCE_DATE_EPOCH=1790092371 sh scripts/sdk-go-version.sh check     # in the no-.git copy
SDK Go pins match the live OpenWrt feeds.                              # exit 0

Honest limit: the one thing local testing cannot prove is the runner's own GITHUB_ENV → next-step injection on this act deployment. That mechanism is already relied on and exercised by these same workflows (the shard jobs' SDK_REF/PACKAGE_FILENAME hand-offs, verify-publication.yml's scope pass-through), so I have no reason to doubt it — but the lane's next push after merge is the final arbiter.

• the guard's error branch, both ways: preset epoch + no git → passes with TG_STRICT_INPUTS=1, BUILD_TIME_UTC=2026-09-22 15:52:51 UTC (deterministic); garbage epoch → SOURCE_DATE_EPOCH must be a unix epoch integer, got 'not-an-epoch' (the same validation ci(ngit): token-emit the package-job SOURCE_DATE_EPOCH expression #491 smoke-tested). And with git present but the variable unset, build-env derives 1790092371 == git log -1 --format=%ct HEAD — the GitHub twin's path, which is why the twin is structurally unaffected (its actions/checkout leaves full git metadata).
• YAML parses (yaml.safe_load); the resolve step sits strictly between the toolchain check and the audit step.

Scope, composition, hygiene

• No other job shares the hazard. The workflow has a single job (binaries) and exactly two build-env consumers: the audit (now fed by the new step) and the repro-build step, whose own cascade is untouched (0 deletions) and now finds the variable already set — same value by construction (identical cascade), still one epoch for both clean roots, so the comparison's meaning is unchanged. No other .ngit workflow runs sdk-go-version.sh (grep: only this file and its .github twin); the workflows that do source build-env supply the epoch themselves (stage-1's determine-versioning/build-portal resolve steps; the shards via the ci(ngit): token-emit the package-job SOURCE_DATE_EPOCH expression #491-fixed needs.resolve-inputs.outputs expression). test.yml/go-test.yml never source it.
• Composes cleanly with ci(ngit): token-emit the package-job SOURCE_DATE_EPOCH expression #491 — disjoint. ci(ngit): token-emit the package-job SOURCE_DATE_EPOCH expression #491 fixed the shard generator (scripts/ngit-gen-shards.py) collapsing ${{ … }} → ${ … } inside the eleven generated build-package-*.yml package-job env lines; this PR touches hand-maintained repro-check.yml, which the generator never renders (grep: no match). Same guard, different files, no interaction.
• PR body accuracy: every claim checked out against the diff and the evidence above. Single commit, clean message, no AI attribution, MERGEABLE; main is 15 commits ahead (a99d983a) but git diff pr/519 upstream/main -- <the workflow> shows only this step — the file is untouched on main since the fork, so the staleness is cosmetic.
• One repo-rule gap: no CHANGELOG entry. CI fixes belong under [Unreleased] → Changed / Internal, and ci(ngit): token-emit the package-job SOURCE_DATE_EPOCH expression #491 — the sibling ngit-epoch fix — set the precedent by adding one. Suggested bullet: "The ngit repro-check lane resolves SOURCE_DATE_EPOCH before the SDK Go pins audit" — the act checkout has no git history for packaging/build-env.sh to derive, so the audit build(pin): follow the official OpenWrt SDK Go feed, one source of truth for CI Go #448 added failed the lane on every main push until this step fed it the commit timestamp (#519).
• Fragility note, not a blocker: the env → git → head_commit.timestamp → clock cascade now exists three times (this step, the repro-build step below it, stage-1's determine-versioning). Copying the proven chain was the right minimal move; if act's event payload ever changes shape, three places need the same edit — a shared helper could consolidate later.

Disposition

Land-with-followups. The fix is correct, minimal, and verified down the audit's real code path; the single follow-up is the one-line [Unreleased] Changed / Internal bullet above, folded in before merge. Note there is no ngit run for the PR head itself (relay query for kind-9841/9842 tagged c=bcc7f3b9…: zero events), so runner-side confirmation can only come from the lane's next run after merge — exactly the merged-without-runner-evidence gap the PR body names #473 as the forcing function for.

[Amperstrand]

Independent verification, both sides:

• Fix path replicated locally: SOURCE_DATE_EPOCH=1789971231 bash scripts/sdk-go-version.sh check → SDK Go pins match the live OpenWrt feeds. exit 0 — the exact environment this step creates for the audit.
• Bug is live on current main: repro-check.yml concluded failure at 17:44, 18:15 and 18:38 UTC today (the 18:38 run post-dates the chore(release): fold the post-#477 wave into v0.6.0-alpha4 #523 fold/tag) — the lane is red on every push, exactly as diagnosed.
• Diff reviewed: correct fallback order (git → head_commit.timestamp → job clock with a loud NOTE); no interaction with the shard generator's token mechanism (this workflow is hand-maintained YAML); the repro-build step's own chain is untouched and now finds the variable pre-set.

Separate observation while sweeping the lanes: test.yml's three reds today are not code — the captured signature is proxy.golang.org/golang.org/x/time/@v/v0.15.0.zip: net/http: TLS handshake timeout → FAIL [setup failed] in main-test (runner-side module fetch; the same tree is 16/16 green on make go-battery locally). Worth a re-run to clear, but nothing to fix in-tree.

Disposition: land — this and the release drive are the only things between main and a clean tag run.

[Amperstrand]

Changelog nit from my review resolved on the branch (885b9e3): the [Unreleased] → Changed / Internal bullet now records the lane fix, matching the sibling CI PRs. Ready to merge from my side.

[Amperstrand]

Rebased onto the current main tip a99d993a→a99d983a (c1913675, replacing 885b9e36): the changelog entry now sits under a fresh [Unreleased] → Changed / Internal (the old base predated the #523 fold, hence the conflict). YAML untouched, zero markers. The wave-3 'sole gap' (the CHANGELOG bullet) is closed by the entry itself — this PR is land-ready.

[Amperstrand]

Incident disclosure + correction: while pushing the rebase, a shell working-directory mistake briefly force-pushed an unrelated local branch (early wallet recover WIP commits) onto fix/repro-audit-epoch. It was detected and reverted within one minute — the branch now carries the intended rebased content (c1913675 → 2f257e8d on a99d983a, verified via ls-remote). No PR content was affected beyond that window; apologies for the noise.

[Amperstrand]

Verified the post-incident branch head c1913675 against origin/main: the delta is exactly the two intended pieces — the 17-line epoch-resolution step in repro-check.yml (9 SOURCE_DATE_EPOCH lines, the same derivation chain as the repro-build step) and the changelog entry. Nothing from the disclosed stray push survives (2f257e8d → c1913675 only; log inspected). Thanks for the verification and the changelog carry — and for the transparent incident report; the one-minute detection-and-revert is exactly how that should be handled.

Land-ready from my side too.

[Amperstrand]

Technical review (evidence-gathering pass, not an approval).

What it does. Unbreaks the ngit repro lane: since #448's SDK Go-pin audit landed, repro-check.yml failed on every push — the audit sources packaging/build-env.sh, which requires a SOURCE_DATE_EPOCH that act checkouts can't derive. The new step resolves the epoch before the audit and exports it via GITHUB_ENV. Workflow YAML verified parsing on this head.

Two notes for the rebase/merge: (1) The changelog block re-carries #491's already-merged entry (this branch predates that merge) — drop the duplicate on rebase. (2) The step uses the git → head_commit → job-clock cascade with an honest "within-run determinism only" note — acceptable for an audit lane (the epoch never reaches artifact bytes here, unlike stage 1), though pointing it at scripts/ngit-commit-epoch.sh — the established fail-closed resolver this repo standardized in #529 — would drop the job-clock branch and one moving part. Suggestion, not a blocker; no overlap with #529's stage-1 change (different lane, different file).

Disposition: land after a rebase (trivial: two files, the changelog duplicate is the only real resolution).

[Amperstrand]

Rebase done (b276ce9d): the duplicated #491 entry is gone (kept main's side of the conflict), a fresh [Unreleased] → Changed / Internal entry carries this fix, and the epoch step now uses scripts/ngit-commit-epoch.sh with the stage-1 fail-closed shape — your suggestion adopted, the job-clock branch and the head_commit expression are gone. YAML re-verified; sdk-go-version.sh check still passes with the epoch set.

[Amperstrand]

Merge-priority note: main's CI is now 3 of 4 lanes green on the newest commit (2226a04c) — go-test, build-package-binaries (first green since the node fix landed), and test.yml (the #520 drop did not recur). The only red lane is repro-check, failing with exactly this PR's signature:

build-env: ERROR: SOURCE_DATE_EPOCH not set and no git history available to derive it
❌ Failure - Main Audit the SDK Go pins against the live OpenWrt feeds

Merging this takes main's ngit CI to 4/4. It's rebased, both review notes actioned, land-ready.

[felixfelix-bot]

Review — head 8262f00f7e8a (card pinned c1913675d5bf; branch rebased twice since. b276ce9d → 8262f00f is a pure rebase onto 2796d96: the workflow delta is byte-identical, scripts untouched — reviewed at the live head.)

Root cause confirmed from source: scripts/sdk-go-version.sh sources packaging/build-env.sh, whose guard dies without SOURCE_DATE_EPOCH when no git history exists (build-env.sh:103) — i.e. every act checkout (.ngit/README: no usable .git). The new step resolves the epoch before the audit. Verified empirically in a no-.git tree of this exact head:

• audit without epoch → build-env: ERROR: SOURCE_DATE_EPOCH not set and no git history available to derive it (rc=1 — the exact lane signature); with the resolver's epoch → SDK Go pins match the live OpenWrt feeds., rc=0, against the live feeds.
• The step body runs verbatim (set -euo pipefail, numeric case-guard, >> "${GITHUB_ENV}"): rc=0, GITHUB_ENV receives SOURCE_DATE_EPOCH=1790282149 for this head; build-env re-rejects non-numeric input.
• It is the established stage-1 shape — line-for-line the twin in .ngit/act/workflows/build-package-binaries.yml:359-365 (only cosmetic diff: ${GITHUB_ENV:-/dev/null} there). No lane-local variant of the derivation.
• The downstream rebuild step adopts the pre-set epoch (its SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-…}" at :124 short-circuits), so the job-clock branch is dead on this lane — determinism gain, not a regression. TG_STRICT_INPUTS delta nil (repro-test.sh forces 1 inside each root regardless).
• YAML parses (9 steps, resolver directly before the audit); diff vs base is exactly +21/-0 (11 workflow lines + 10 changelog); changelog carries exactly one ci(repro): resolve SOURCE_DATE_EPOCH before the SDK Go audit #519 entry under [Unreleased] → Changed / Internal, and ci(ngit): token-emit the package-job SOURCE_DATE_EPOCH expression #491 exactly once (old release section).

[RISK] .ngit/act/workflows/repro-check.yml:101 — the resolver cannot serve pull_request-event runs: the ngit mirror holds no open-PR heads. Probed live: fetch-by-full-SHA of this head 8262f00f against relay.ngit.dev → not our ref (likewise the prior head b276ce9d and a sample of other open-PR heads); mirror main = 2796d96 = this PR's base, fetches fine. So a PR-triggered act run dies at the epoch step with the resolver's fail-closed message — no worse than today (the audit step already killed those runs), and the GitHub twin covers PRs (.github/workflows/repro-check.yml:46 — its runners have .git, build-env derives the epoch itself; that audit has been green). The failure this PR exists to fix — push: main — is served by construction: the pushed SHA is on the mirror. When PR-event greenness is wanted, git fetch https://github.com/OpenTollGate/tollgate-module-basic-go.git <sha> does serve open-PR heads (verified), so a GitHub-URL second fallback — or the established ci/* mirror-ref convention — closes it. Not blocking: strictly better than the status quo, no regression.

[NIT] PR body describes the superseded implementation — the "git → head_commit.timestamp → job clock" chain was the first commit; the current head replaced it with the fail-closed resolver. Body text only.

[INFO] repro-check.yml:116-119 — the "job-start clock as the last resort" comment in the rebuild step is now dead on this lane (the epoch step always supplies the var on the triggers that get past it). Optional cleanup, not required.

APPROVED — fixes main's only red lane with the repo-standard fail-closed resolver; verified end-to-end at this head. Cross-family gate-2.5 cold audit concurs (no additional findings).

[felixfelix-bot]

APPROVED at head 8262f00f7e8a — verified end-to-end (no-.git audit replication, resolver step body verbatim, mirror probes, changelog). Full review: issuecomment-5825512624.

[felixfelix-bot]

Correction addendum to my review above (head 8262f00f7e8a, comment 5825512624) — the cross-family audit pass caught three statements that need fixing. Verdict unchanged: APPROVED; no change requested in the PR itself.

(a) [RISK] mitigation retracted — "the GitHub twin covers PRs … its audit has been green" is false. Verified now: .github/workflows/repro-check.yml has zero runs ever (actions/workflows/repro-check.yml/runs → total_count 0; check-runs at this head = 0; the repo's Actions queue holds only stuck-queued "Graph Update" dynamic runs — no ordinary workflow has executed since 2026-08-27). The claim was inherited from the PR body instead of probed. Corrected consequence: on pull_request events PR-time CI coverage is nil on both lanes — the act lane dies at the epoch step (repro-check.yml:101) and the GitHub twin never runs. This is still not a regression (base died one step later at the audit, and the twin was equally dead), so it does not change the verdict — but the practical gap is bigger than stated: the ci/* mirror-ref convention or a GitHub-URL second fallback in the resolver would close it for both lanes, not just the act lane.

(b) "the ngit mirror holds no open-PR heads" — overstated. Re-probed with exact live head SHAs: of six sampled open-PR heads, 4b8838d3 (#579), 527be06d (#578), fe6ca663 (#573) are served by the mirror by full SHA; 5f2201e4 (#577), 19050bb9 (#572), 3884da64 (#571) are refused. The mirror serves some PR heads (likely those mirrored before CI last ran) and refuses others — the accurate statement is that this PR's head and its rebased-away predecessors are unserved, not that no PR heads exist on the mirror.

(c) Step-body evidence overstated — "the step body runs verbatim, rc=0" holds only with local git history or a served remote (NGIT_MIRROR_URL pointing at GitHub, which serves open-PR heads by full SHA). Under the exact lane condition (no .git, resolver's default mirror) that body exits rc=1 with the resolver's fail-closed "did not serve that commit" — which is by design (never a guessed epoch), and is precisely the [RISK] above. The verified facts stand: with the epoch supplied, sdk-go-version.sh check passes against the live feeds, and the numeric guard + GITHUB_ENV export behave as written.

Everything else in the review was independently reproduced by the auditor (diff scope +21/-0, build-env guard lines, twin step-body fidelity incl. the ${GITHUB_ENV} vs ${GITHUB_ENV:-/dev/null} note, changelog placement, YAML shape). The APPROVED verdict and the non-blocking [NIT]/[INFO] stand as posted.