docs(skills): add the standing upstream-sync skill and two-tier validation cadence - #77
Merged
Conversation
…ation cadence Encode the captain's 2026-08-27 rulings for the weekly fork catch-up: keep-list decided by rule and reviewed at the PR, weekly lint plus colocated tests versus a monthly full suite (next monthly run 2026-10-01), unrelated breakage filed as follow-up, a fixed worker brief template, and the isolated Herdr lab setup documented once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
withally
force-pushed
the
fm/fm-upstream-sync-skill-f2
branch
from
August 27, 2026 13:48
46dd4f3 to
4f90bb8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Write the standing upstream-sync skill for withally/firstmate so weekly fork catch-up follows a durable checklist: keep-list decided by rule and reviewed at the PR; two-tier weekly/monthly validation with the next monthly full run on 2026-10-01; unrelated breakage filed as follow-up; docs/upstream-sync.md as the procedural spine; one documented Herdr lab setup; a fixed worker brief template; the AGENTS.md pointer; and the required skill/docs tests and lint. Preserve one sentence per Markdown line, use upstream only as the documented selective source, do not touch the m2 worktree, do not push to upstream, do not merge. Update the existing PR 77 through every no-mistakes gate and CI.
What Changed
docs/upstream-sync.mdas the procedural spine for the weekly fork catch-up — upstream remote guard with push disabled, branch-at-upstream/main, thechore: snapshot upstream mainaudit window with its earlier-boundary override, the keep rule, and the catch-up log carryingNext monthly full run: 2026-10-01..agents/skills/upstream-syncskill over that spine: dispatcher intake resolving the five values (UPSTREAM_BASE,SNAPSHOT,SETTLED,TIER,DATE), the worker checklist, the weekly/monthly tier decision, plusreferences/worker-brief.md(fixed brief template) andreferences/herdr-lab.md(one documented Herdr lab setup);AGENTS.mdgains the load-trigger pointer anddocs/documentation-audiences.jsonregisters all four new files.bin/fm-test-run.shchanged-path mapping so non-SKILL.mdfiles under.agents/skills/*resolve throughfamilies_for_test_referenceand otherwise fall back topure-contract-unit, with a matching case intests/fm-test-run.test.shthat commits a skill reference asset and asserts the pure-contract suite is selected.Risk Assessment
✅ Low: The change is almost entirely procedural documentation whose one executable delta — a single
.agents/skills/*)arm infamilies_for_changed_path— is narrow, correctly ordered, and covered by a behavioral test that fails closed without it; I traced the intake/worker boundary algorithm through a full next-sync cycle against live repo state and found no path that produces a wrong window, a lost keep-list, or a silently green validation.Testing
Ran the targeted suites the repo's own changed-file map selects for this diff (fm-test-run, fm-documentation-audiences, fm-ensure-agents-md, fm-herdr-lab, fm-lint) — all green — and proved the selector change is a real regression fix by running the new test against the base-commit selector, where it fails with
no changed-test mapping for source path: .agents/skills/upstream-sync/references/herdr-lab.md. Because the deliverable is a standing procedure rather than a library, I exercised the procedure itself instead of grepping its prose: the SKILL.md step-1 guard block was extracted verbatim and executed against a read-only clone of the real fork with the live upstream remote, producing every documentedblocked:refusal, passing on real values, and leavinggit push upstreamrefused via the DISABLED push URL; the doc's audit-window steps resolved the real snapshot marker to 0 PRs versus 69 from the forbidden range; the tier rule resolved weekly through September and monthly from 2026-10-01; the fixed worker brief rendered through bin/fm-brief.sh into the complete prompt a worker agent actually receives with all five values filled; and bin/fm-herdr-lab.sh's refusals matched the herdr-lab reference. Visual evidence is included as rendered HTML plus Chrome screenshots of the catch-up log table, the monthly-run line, and the skill head. Everything passed and the worktree is clean; no findings.Evidence: SKILL.md step-1 guard block executed against the real fork and live upstream
Source: SKILL.md step-1 guard block executed against the real fork and live upstream
### scenario: upstream remote points at some other fork blocked: upstream remote does not point at kunchenguid/firstmate -> exit=1 ### scenario: recorded base is not an ancestor of upstream/main blocked: recorded UPSTREAM_BASE is not an ancestor of upstream/main; upstream history was rewritten or the base is not upstream material -> exit=1 ### scenario: valid brief (real values) -> exit=0 === after the valid run, can the worker push to upstream? === fetch url = https://github.com/kunchenguid/firstmate.git push url = DISABLED fatal: 'DISABLED' does not appear to be a git repository -> git push upstream exit=128Evidence: fm-test-run.sh skill-reference routing, before vs after
Source: fm-test-run.sh skill-reference routing, before vs after
--- BEFORE (selector at base commit): a skill reference asset has no mapping fm-test-run: no changed-test mapping for source path: .agents/skills/upstream-sync/references/herdr-lab.md -> exit=2 --- AFTER (selector on this branch): the same paths select pure-contract coverage -> exit=0 (count=32)Evidence: Audit window on the real fork vs the forbidden range
Source: Audit window on the real fork vs the forbidden range
--- step 3: latest first-parent 'chore: snapshot upstream main' marker on origin/main 34d9081e491b98cb7800eb919dbd1bbf47395e72 2026-08-27 chore: snapshot upstream main for 2026-08-27 (#78) --- step 4: fork PRs to audit, i.e. $CATCHUP..origin/main first-parent --- step 5 negative control: the forbidden 'origin/main ^upstream/main' range commit count = 69 (vs 0 from the documented range)Evidence: Two-tier cadence resolution across dates
Source: Two-tier cadence resolution across dates
parsed Next monthly full run date : 2026-10-01 catch-up rows recording a full/monthly run on or after 2026-10-01 : none today=2026-08-27 -> TIER=weekly today=2026-09-05 -> TIER=weekly today=2026-09-30 -> TIER=weekly today=2026-10-01 -> TIER=monthly today=2026-10-03 -> TIER=monthlyEvidence: Emitted worker brief (the actual prompt a sync worker receives)
Source: Emitted worker brief (the actual prompt a sync worker receives)
Evidence: Herdr lab helper contract exercised against bin/fm-herdr-lab.sh
Source: Herdr lab helper contract exercised against bin/fm-herdr-lab.sh
name upstream-sync-weekly -> "fm-lab-upstream-sync-we-43292-18475" (label capped at 16 chars, fm-lab- prefix) refusal: session literally named default fm-herdr-lab: refusing session name 'default' -> exit=1 refusal: caller-supplied --session on run fm-herdr-lab: run forbids caller-supplied --session -> exit=1 refusal: server lifecycle subcommand through run fm-herdr-lab: run forbids server operations -> exit=1Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
⏭️ **Rebase** - skipped
Step was skipped.
docs/upstream-sync.md:1- This branch addsdocs/upstream-sync.mdas a new file whileorigin/mainnow carries a divergent copy that landed with the m2 cutover, so the branch and main conflict on that path. Verified live:git merge-base HEAD origin/mainisb0638f67e(pre-cutover),origin/mainis at34d9081(chore: snapshot upstream main for 2026-08-27 (#78)), andgit merge-tree --write-tree --name-only HEAD origin/mainreports exactly one conflict —CONFLICT (add/add): Merge conflict in docs/upstream-sync.md— whileAGENTS.md,bin/fm-test-run.sh, anddocs/documentation-audiences.jsonall auto-merge cleanly. The pipeline's rebase step owns resolving it, so this is not a blocker, but the resolution direction is load-bearing and worth stating: this branch's version must win wholesale.origin/main's copy is the older three-column log with noWindow end commitcolumn and noNext monthly full runline, and intake step 1 (SKILL.md:44-46) parses that column while intake step 2 (SKILL.md:59) reads that line — a resolution that kept or unioned main's copy would make the next sync stop atblocked: origin/main has a catch-up log whose newest row names no window end commitorblocked: origin/main carries no upstream-sync doc; the tier rule has no source. Nothing in main's copy is absent from this branch's (both catch-up rows are present here, with the m2 cutover already recorded as landed at34d9081), so taking this branch's file verbatim loses nothing..agents/skills/upstream-sync/SKILL.md:168- Step 5's recovery path says to read the excluded snapshot squash's catch-up log row (read its row with git show origin/main:docs/upstream-sync.md) but never states how to map a squash commit to a row, and step 12's required row fields cannot carry that mapping. Step 12 (SKILL.md:265) requires date, adopted base, window end commit, tier, and verdicts; the sync's own merge SHA is not among them and is unknowable when the row is appended, since the squash does not exist until merge. The mapping is unambiguous today only because this fix round hand-wrotelanded on origin/main as34d9081(chore: snapshot upstream main for 2026-08-27 #78)into the 2026-08-27 row's base column, which a future checklist-appended row will not carry. A derivable key does exist and is reliable: step 13 (SKILL.md:280) titles the PRchore: snapshot upstream main for <DATE>and step 12 records that same<DATE>as the row'sCatch-up date, so the squash's subject names its row exactly. Risk is low because rows are chronological and "the newest row at or before this squash" is a safe fallback, but an ambiguous lookup drops the accumulated keep-list, which is the silent loss steps 5 and 12 exist to prevent. Fix: state the key inline in step 5 — "the row whoseCatch-up dateequals the<DATE>in that squash's subject" — and mirror it at docs/upstream-sync.md:37.✅ **Test** - passed
✅ No issues found.
bin/fm-test-run.sh tests/fm-test-run.test.sh tests/fm-documentation-audiences.test.sh tests/fm-ensure-agents-md.test.sh— all greenbin/fm-test-run.sh tests/fm-herdr-lab.test.sh tests/fm-lint.test.sh— all greenRegression proof: replaced bin/fm-test-run.sh with the base-commit version and ranbash tests/fm-test-run.test.sh— new caseskill reference asset selects pure contract coveragefails; restored and it passesbin/fm-test-run.sh --list --changed --base b0638f6before/after the selector change on this change's own paths (exit 2 vs exit 0, 32 tests selected)Extracted the SKILL.md step-1 shell block verbatim and executed it in a scratch clone of withally/firstmate against the live kunchenguid/firstmate remote, across 5 scenarios (missing SNAPSHOT, wrong fork, unknown SNAPSHOT, non-ancestor base, valid brief)git push upstream HEAD:refs/heads/mainafter the valid step-1 run — refused, push url = DISABLEDdocs/upstream-sync.md step 3 marker search and step 4 audit range executed on the real fork, with the step-5 forbidden range as a negative controlApplied the skill's step-2 tier rule to the doc'sNext monthly full runline and catch-up log for 2026-08-27, 2026-09-05, 2026-09-30, 2026-10-01, 2026-10-03FM_HOME=<tmp> bin/fm-brief.sh fm-upstream-sync-2026-10-03 firstmate --mode no-mistakes --herdr-labwith references/worker-brief.md substituted for {TASK} and the five values filled — verified zero remaining placeholders in the emitted briefbin/fm-herdr-lab.sh name/provision/runrefusal paths: default session name, missing fm-lab- prefix, caller-supplied --session, server lifecycle subcommand, leading optionVerified everybin/fm-*.shand CLI flag referenced by the skill/docs exists (bin/fm-brief.sh --help,bin/fm-test-run.sh --help)Checked one-sentence-per-Markdown-line across all four new/changed Markdown surfaces and balanced code fencesRendered docs/upstream-sync.md and the three skill files to HTML with pandoc and captured Chrome screenshots of the catch-up log, the monthly-run line, and the skill headtests/fm-backend-herdr-focus-flash-e2e.test.sh- tests/fm-backend-herdr-focus-flash-e2e.test.sh requires the Herdr lab helper and a real herdr binary but sits in theunclassifiedfamily rather thanreal-herdr-gated, sobin/fm-test-run.sh --family real-herdr-gated --fail-on-gate-skip 'herdr not found'never covers it and only the monthly--allrun reaches it. Documented as-is in references/herdr-lab.md; the family assignment itself is executable test-map material and out of scope for the documentation phase.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.