Skip to content

Fail the required aggregate when any shard fails - #1460

Merged
julia-script merged 1 commit into
selfhostfrom
selfhost-aggregate-gate
Oct 12, 2026
Merged

julia-script merged 1 commit into
selfhostfrom
selfhost-aggregate-gate

Conversation

@julia-script

@julia-script julia-script commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

"Focused Linux Silk" ran test "$CHAIN" = success && test "$CORPUS" = success && test "$NATIVE_CASES" = success under bash -e. A failing command before the last && of a list does not end a -e script, so a red, skipped or cancelled chain, corpus or native-cases shard was ignored whenever the next line passed. The same pattern (test "$CORPUS" = success && test "$SEED" = success) has existed since 7ec4ab3 (2026-10-10).

The fix puts each shard's test on its own line. The cases line is unchanged: as the script's last command, its status is the step's status.

Checked locally with the step's exact script under bash -e:

chain, corpus, native-cases, cases, event old exit new exit
all success, push 0 0
corpus failure, push 0 1
cases skipped, pull_request 0 0
cases skipped, push 1 1

Workflow-only change. Not a guarded input, so it lands on selfhost directly.

Record: runs the hole let through

I scanned every selfhost.yml run created since 7ec4ab3 through the REST API, looking for a green aggregate while a gated job failed. These landed:

what head run failing job
selfhost push of #1449's merge 3ca78ea 38183198621 corpus (the four P6 programs, fixed by #1459)
selfhost push of #1412's merge 8dd394c 38091373410 corpus
#1412, final head (merged) c6ef907 38090594423 corpus
#1411, final head (merged) 7183113 38088890949 corpus
#1410, final head (merged) 06a9328 38083735397 corpus
#1409, final head (merged) 894a74a 38063248838 corpus

Still open, with a green aggregate over a red corpus: #1457 at c739f8d (run 38183231447).

A second form of the hole: a failed "Native N1 from pinned seed" skips the chain and corpus jobs, and the aggregate still passed. Only superseded heads of #1437, #1441, #1446 and #1452 were hit; none of those heads merged.

The 2026-10-10 corpus failures were fixed later: on 3ca78ea only the four P6 programs fail.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SJak9tcQcwEZgNXjANPTiA

Under bash -e a failing test before the last && of a list does not end the step,
so Focused Linux Silk passed with the native corpus red (push 3ca78ea). Test
each shard on its own line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SJak9tcQcwEZgNXjANPTiA
@julia-script
julia-script merged commit 42ca6ff into selfhost Oct 12, 2026
6 of 7 checks passed
julia-script added a commit that referenced this pull request Oct 18, 2026
Fail the required aggregate when any shard fails
@julia-script
julia-script deleted the selfhost-aggregate-gate branch October 18, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants