Skip to content

Keep the batched cutover's resume marker when the target refuses the branch create - #117

Merged
Soph merged 7 commits into
mainfrom
soph/batched-cutover-drops-resume-marker
Aug 31, 2026
Merged

Keep the batched cutover's resume marker when the target refuses the branch create#117
Soph merged 7 commits into
mainfrom
soph/batched-cutover-drops-resume-marker

Conversation

@Soph

@Soph Soph commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Fixes item 3 of ENT-2060 — the batched instance of the mistake #116 fixed one-shot (653e9c5). Pre-existing, byte-identical on origin/main before that PR and untouched by its diff, so nobody should bisect it there.

Rebased onto main so it carries #116; the pinning commit is unchanged in content.

The bug

A batched bootstrap's temp ref refs/gitsync/bootstrap/heads/<branch> is the only record of how far an import got. The cutover push carries two commands in one request — advance the temp ref to the final checkpoint, and create the real branch there — and the temp ref was deleted immediately afterwards on the strength of that push returning no error.

Under --best-effort that inference is invalid. gitproto/push.go hands a per-ref ng to OnRejection and returns nil, so a target that refuses the branch create (protected branch, pre-receive policy) ended the run with neither the branch nor the marker, reporting success:

[server] request with 2 command(s), rejecting only refs/heads/master:
  refs/gitsync/bootstrap/heads/master c416d48f -> cbfbb940 : ok
  refs/heads/master                   <zero>   -> cbfbb940 : ng rejected by policy

Run() error : <nil>          ← exit 0
Pushed/Warned/BatchCount : 0 / 1 / 8
target after run: branch present=false, marker present=false

The nearby guard couldn't catch it either: it consults p.TargetRefs, the ref map captured at session start, not re-read after the push.

The rejection itself was surfaced honestly (Pushed: 0, Warned: 1, plan action=warn). What was surfaced nowhere is that the run also destroyed the checkpoint, and the process exits 0. With the marker gone and the branch never created, every object pushed so far is unreferenced on the target: nothing points at it, so the next run has no fetch have to negotiate against and re-transfers the entire history — and the target is free to GC it meanwhile. Batching is by definition the large-repo path (ENT-1948, >10 GiB), so this cost exactly the imports that can least afford it.

Reachable wherever best-effort is implied: --all-refs turns it on for bootstrap (cmd/git-sync/bootstrap.go) and sync (syncplan.go). Not for replicate via CLI — deliberate and documented — but client.go passes Policy.BestEffort through for any mode.

The fix

The rejection now reaches the strategy. bootstrap.Params grows

Rejected func(plumbing.ReferenceName) bool

which the syncer answers from the map it already fills from the pusher's OnRejection callback (s.rejections, one method, refRejected). The cutover consults it before deleting: a create this run cannot confirm leaves the marker at the final checkpoint, and the next run resumes from it.

Conditional delete rather than no delete, which is how #116 fixed the one-shot path: Bootstrap() rejects --prune (syncer.go), so on that route no cleaner would ever exist and every batched bootstrap would leave permanent scaffolding. Prune can own the marker only where prune can run.

Two smaller consequences:

  • The kept marker is recorded as a completed ref, since it genuinely holds that hash — the objects behind it stay usable as fetch haves for the branches planned after this one.
  • Retaining it logs and prints a line naming the marker and the hash the next run resumes from, so the one thing that was silent no longer is.

All three bootstrap entry points (the replicate resume route, sync's bootstrap route, and Bootstrap()) funnel through bootstrapWithInputs, so all three are covered. Non-best-effort callers pass no predicate and behave exactly as before — there a per-ref ng fails the push outright.

Tests

  • TestRun_IntegrationBatchedCutoverKeepsResumeMarkerOnRejectedCreate — the pin from the first commit, now passing.
  • TestRun_IntegrationBatchedCutoverResumesAfterRejectedCreate — the half that makes the marker worth keeping. Once the target accepts the create, the retry routes on bootstrap-resume-marker, finishes the import from the marker (0 checkpoint packs, 334 bytes to the target against the first run's 427 KiB) and only then deletes it.

Both share a refCreateDenier receive-pack hook, extracted from the first test and carrying the reason it cannot use syncertest.DenyRefsReport: a non-nil report short-circuits the test server before it applies anything, which would leave the temp ref at the previous checkpoint and make the delete that follows carry a stale Old — a shape a real receive-pack would reject, hiding the bug behind a second failure.

Verified the pair fails without the fix (marker gone in both). Full suite and golangci-lint green.

Still open on ENT-2060

Untouched here: partial-progress subset resume, a sync-mode resume route, stale markers under --map or branch-scoped prune, and dry-run's silence on marker lifecycle.

🤖 Generated with Claude Code

https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7


Note

Medium Risk
Changes core batched-bootstrap state machine and exit codes when refs/gitsync/* writes are blocked; behavior is heavily tested but affects large-repo imports and best-effort alerting.

Overview
Batched bootstrap no longer treats a successful push as proof the branch was created. Under --best-effort, a per-ref ng still yields a nil push error, so a cutover that advanced the temp ref but had the real branch refused used to delete refs/gitsync/bootstrap/heads/<branch> and exit 0 — leaving no resume anchor and forcing a full re-transfer on the next run.

The pusher now records per-push ref outcomes (RefOutcome / LastOutcome) without changing strict vs best-effort semantics. Bootstrap consults that signal (and, when status is unknown, a mid-run target ref listing via TargetRefsNow) before deleting markers: unconfirmed creates defer cleanup, keep the temp ref as a fetch have for later branches, and settle in resolvePendingMarkers by checking whether the branch exists at the pushed hash. Refusal to update the bootstrap temp ref is fatal even under best-effort, since checkpoint state would otherwise diverge from the target.

The syncer wires refOutcome and targetRefsNow into bootstrapWithInputs; rejection reasons are sanitized for notices. Integration tests cover refused creates, resume-after-retry, targets without report-status, and related edge cases.

Reviewed by Cursor Bugbot for commit 864584b. Configure here.

Soph and others added 2 commits August 31, 2026 13:07
Failing test, no fix. Batched bootstrap deletes its temp ref
(refs/gitsync/bootstrap/heads/<branch>) immediately after the cutover
push, on the strength of that push returning no error. Under BestEffort
that inference is invalid: gitproto hands a per-ref "ng" to OnRejection
and returns nil, so a target refusing the branch create leaves the
branch absent and the resume marker deleted, with the run reporting
success.

Every object pushed so far is then unreferenced on the target, so the
next run has no fetch have to negotiate against and re-transfers the
whole history -- on precisely the large repositories batching exists
for.

The test applies the non-denied commands itself rather than using
syncertest.DenyRefsReport: a non-nil report short-circuits the test
server before it applies anything, which would leave the temp ref at the
previous checkpoint and make the delete that follows carry a stale Old,
hiding the bug behind a second failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 01M1BFVG4TMHD8PHWE7BDMT5EB
The batched cutover deleted its temp ref on the strength of a nil push error,
and under BestEffort that error covers the request rather than each command in
it: the pusher hands a per-ref "ng" to OnRejection and returns nil. So a target
that refused the branch create — protected branch, pre-receive policy — ended
the run with neither the branch nor the marker, reported as a success. With
nothing on the target pointing at the objects already pushed, the next run had
no have to negotiate against and re-transferred the whole history, on precisely
the large repositories batching exists for (ENT-1948, >10 GiB). Reachable
wherever best-effort is implied: --all-refs turns it on for bootstrap and sync,
and client.go passes Policy.BestEffort through for any mode.

The rejection now reaches the strategy. Params.Rejected is a predicate the
syncer answers from the map it already fills from the pusher's OnRejection
callback, and the cutover consults it before deleting: a create this run cannot
confirm leaves the marker at the final checkpoint, so the next run resumes from
it. The delete is made conditional rather than dropped the way PR #116 dropped
the one-shot path's: Bootstrap() rejects --prune outright, so on that route no
cleaner would ever exist and the marker would be permanent.

The kept marker is also recorded as a completed ref, since it genuinely holds
that hash — the objects behind it stay usable as haves for the branches planned
after this one. The rejection itself was already surfaced honestly (Pushed 0,
Warned 1, plan action=warn); what was surfaced nowhere is that the run also
destroyed the checkpoint, so retaining it now logs and prints a line naming the
marker and the hash it resumes from.

Tests: the pinned test from this branch now passes, and a companion asserts the
half that makes the marker worth keeping — the retry, once the target accepts
the create, finishes the import from the marker (zero checkpoint packs, 334
bytes to the target against the first run's 427 KiB) and only then deletes it.
Both share the receive-pack denier the first test introduced, now a type
carrying the reason it cannot use syncertest.DenyRefsReport.

Remaining items on ENT-2060 are untouched: partial-progress subset resume, a
sync-mode route, and stale markers under --map or branch-scoped prune.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7
Entire-Checkpoint: 01M1BRMEV28MA421RDYNHM4X7C
@nodo
nodo force-pushed the soph/batched-cutover-drops-resume-marker branch from b820066 to 30c56db Compare August 31, 2026 11:19
@nodo nodo changed the title Failing test: batched cutover drops its resume marker on a rejected create Keep the batched cutover's resume marker when the target refuses the branch create Aug 31, 2026
@nodo

nodo commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Added the fix as a second commit (30c56dbb) — the PR is no longer intentionally red, so the title and description now describe the change rather than the pin. @nodo force-pushed the branch onto main first, because the resume test needs #116's routing; the pinning commit is unchanged in content, only rebased.

The fix is the shape ENT-2060 predicted: bootstrap.Params.Rejected, answered from the s.rejections map the syncer already fills, consulted at the cutover so a create the run cannot confirm keeps the marker. Conditional delete rather than dropping cleanup the way #116 did on the one-shot path, since Bootstrap() refuses --prune and prune therefore cannot be the cleaner on that route.

Your receivePackHook note was the load-bearing part of the second test — it's now a refCreateDenier type shared by both, with your DenyRefsReport rationale on the type. The new test asserts the marker earns its keep: the retry finishes from it in 334 bytes instead of re-pushing 427 KiB.

🤖 Generated with Claude Code

@nodo
nodo marked this pull request as ready for review August 31, 2026 11:24
…update

The conditional delete was right but under-scoped, and the fixture that proved
it was not proving it.

**The fixture applied refs whose objects never arrived.** A non-nil report from
receivePackHook short-circuits the test server before transport.ReceivePack, so
the pack on the cutover push was never unpacked: the marker resolved to a hash
the target could not resolve, ~30 objects short. That inverted the helper's own
claim that "the marker genuinely reaches the final checkpoint" — only the ref
did — and hollowed out the resume assertion, since the retry's 334 bytes were a
bare ref create against a target missing a whole batch. The server now unpacks
the pushed objects before a hook's report wins (opt-in, so no other hook's
target silently gains objects), and both tests assert the target resolves the
marker and every commit reachable from the tip, not just hashes.

Two more fixture faults: the denier prefixed its own "ng " onto a status go-git
already encodes as `ng <ref> <status>`, so the operator-visible reason arrived
doubled — a test now pins the bare reason through to plan.Reason — and it
ignored cmd.Old, leaving unenforced the very compare-and-swap whose stale-Old
hazard is why it applies commands by hand. It now answers a mismatched Old the
way a real receive-pack does.

**Three more paths inferred "landed" from a nil error.** The checkpoint loop
advanced `current` without checking the temp-ref update it just pushed, so a
target that refuses the whole request (read-only repo, blanket pre-receive
block) had every later checkpoint negotiating against a have it never accepted;
that now stops the run, naming the ref and the server's reason. The
subsumed-branch path recorded a have and counted a batch for a lone create the
target could have refused. And nothing consulted report-status: without it the
pusher decodes no report at all, every ref looks unrejected, and the marker was
deleted on a create nothing had confirmed — the exact hazard this PR fixes.
Silence is now treated as "unknown, so keep the marker", and never as a refusal,
which would fail every batched bootstrap against such a target.

**A benign rejection no longer leaks the marker.** "already exists" is a
concurrent move: the branch IS on the target, so its scaffolding is stale and
must be deleted, or it strands forever on the route that refuses --prune.
gitproto.IsConcurrentMove is exported for that one question.

**The predicate is now scoped to one push.** s.rejections accumulates over the
session, so asking it about a ref pushed in an earlier request was a false
positive waiting to happen. gitproto.Pusher records the rejections of its most
recent Push* call and answers Params.RefRefused from that; the recording rides
the existing OnRejection callback, so a caller that installed none still gets a
fatal per-ref ng rather than a silently best-effort push.

Tests: five bootstrap-package unit tests over a two-branch graph — refused
create keeps the marker and offers it as a have (the multi-branch case the
integration tests cannot reach), concurrent move deletes it, refused temp ref
stops the run, unconfirmable create keeps it, refused subsumed create is not
counted; two gitproto tests for the per-push window and the fatal path; one
integration test for a target that never advertises report-status. Verified
each new guard fails its test when removed. Full suite and golangci-lint green.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7
Entire-Checkpoint: 01M1BTSB3K4DTBP1AR616ARXTV
@nodo

nodo commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Review addressed in 1aa4f3d5. Every item, in your numbering:

1 — fixture applied refs whose objects never arrived. Confirmed exactly as described: the hook's report returns before transport.ReceivePack, so the cutover pack was never unpacked. The server now unpacks the pushed objects before a hook's report wins, behind a receivePackUnpackForHook opt-in so no other hook's target silently gains objects. Both tests now assert the target resolves what the refs name — the marker's commit, and every commit and tree reachable from the tip — since assertHeadsMatch compares hashes only. The resume test's "334 bytes" now means what it claimed.

2 — doubled ng. Fixed; the denier passes the bare reason like the DenyRefsReport call sites, and the pinned test now asserts the reason reaching plan.Reason is the target's, not the fixture's.

3 — temp-ref advance assumed landed. Guarded in the checkpoint loop: a refused temp ref stops the run, naming the ref and the server's reason, before current advances or the delete can carry a hash the target never accepted. Only a refusal stops it — see 4.

4 — blind without report-status. TargetReportsRefStatus is threaded from s.target.features.ReportStatus. Silence from a target that cannot report is now "unknown, so keep the marker", never "landed" and never "refused" — treating it as a refusal would fail every batched bootstrap against such a target. Pinned by an integration test; note go-git's server-side ReceivePack returns before applying any ref command when the client can't be told the outcome, so the fixture supplies that itself.

5 — benign "already exists" leaked the marker. gitproto.IsConcurrentMove is exported for that one question: the branch is on the target, so the scaffolding is stale and gets deleted rather than stranded on the route that refuses --prune.

6 — subsumed path. Same guard: a refused lone create no longer records a have or increments BatchCount.

Session-wide predicate. Taken, in the shape you suggested: gitproto.Pusher now records the rejections of its most recent Push* call and Params.RefRefused answers from that window, not from s.rejections. The recording rides the existing OnRejection callback, so a caller that installed none still gets a fatal per-ref ng rather than a quietly best-effort push — with a test for each half.

Nits. cmd.Old is now enforced, so the stale-Old CAS the fixture's comment rests on is real. The log field is per-branch (pushed_checkpoints). The DenyRefsReport overlap stays: it builds statuses but applies nothing, which is the one thing this hook exists to do — the type now says so.

Also added. The rejection-surfaced assertions (Warned/ActionWarn), and the bootstrap-package unit tests: a two-branch graph covering refused-create-keeps-marker-as-have (the multi-branch have preservation the integration tests can't reach), concurrent move, refused temp ref, unconfirmable create, refused subsumed create. Each new guard was verified to fail its test when removed. Full suite and golangci-lint green.

🤖 Generated with Claude Code

nodo and others added 4 commits August 31, 2026 14:58
… its prose

The previous round inferred "the branch is there" from the ng text the target
happened to write, using gitproto's concurrent-move marker set — a substring
heuristic built to classify compare-and-swap misses, and documented as
server-specific. A pre-receive message like "refusing to create
refs/heads/main: a tag with that name already exists" matched it, deleted the
resume marker with the branch still absent, and reintroduced the exact
re-transfer this PR exists to prevent. It also had no answer for a target that
never advertises report-status: markers were kept on runs that fully succeeded,
permanently, since the bootstrap route forbids --prune.

Both are the same mistake — guessing at target state — and there is one honest
answer: ask. When a push leaves a create in doubt, the marker is held back
rather than decided, and after the last branch a single ref listing settles
every doubtful case at once. Branch present, whoever created it: the marker is
stale scaffolding and goes. Branch absent: it is a resume position and stays,
with the reason stated accurately, which the old notice did not — it claimed
"did not land" for creates that had landed and promised a resume that could not
happen. Listing unavailable: everything stays, and the run still succeeds,
because a listing failure at the end of a multi-gigabyte import must not fail
an import that was delivered. The common path pays nothing: a confirmed create
still deletes its marker immediately, with no extra round trip, pinned by a
test. gitproto.IsConcurrentMove goes back to being unexported.

Also from the review, all confirmed by reading the code:

- The ng reason reached the terminal unsanitized. gitproto stores the raw
  status and only OnRejection filtered it, so the notice printed server text
  verbatim — an escape sequence could clear the warning's line and redraw it as
  a success, which is the attack internal/sanitize exists for. Filtered where
  s.rejections filters, with a test that fails without it.
- A refused delete of a *stale* marker returned nil and the run restarted from
  zero, pushing the next checkpoint with Old=zero against a ref still on the
  target: a CAS failure every subsequent run reproduces, needing a hand
  cleanup. Now stops, naming both the refusal and why the marker was stale.
- A refused delete of a *finished* marker was silent. Now logged and surfaced,
  non-fatally, since the branch is there.
- A report that omits a command git-sync sent left that ref looking unrejected.
  gitproto now treats an unauthored silence as a rejection rather than as
  success.
- The subsumed path uses the refusal directly; there is no marker to settle and
  nothing to clean up either way.

Fixture, also from the review: refCreateDenier's off/denials were shared with
the httptest goroutine unsynchronized (latent under CI's -race); the
no-report-status flag was honored only on the packful path, so delete-only
pushes silently got a reporting target; the unpack and apply-commands blocks
existed in three copies that had already drifted apart on compare-and-swap
fidelity — now one helper each, so every path enforces Old; and
assertCommitHistoryPresent claimed to check blobs while only resolving trees.
It now walks trees and blobs and compares the target's commit count with the
source's, so "the import landed" is checked rather than asserted.

Left open, and not this PR's: the one-shot path still justifies never deleting
its marker by deferring to prune, which Bootstrap() forbids — the same
permanent-marker gap, one function up, from #116. Now cheap to close with the
same listing.

Full suite green under -race, golangci-lint clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7
Entire-Checkpoint: 01M1BYCREMPSP93AWNJ6YNDWAS
…he window

The reviewer found no live data race — every push, callback, and marker
decision runs on the goroutine driving Execute, and the fixture state guarded
last round holds under -race. What it found instead were ordering faults and
one rule I got backwards.

**"Silence is a refusal" was wrong, and it was mine.** Last round's gitproto
change synthesized a rejection for a command the target's report never
mentioned. That contradicted the guard two lines above it, which says treating
silence as a refusal would fail every batched bootstrap against a
non-reporting target: an omitted status for a temp ref would have aborted a
multi-gigabyte import over something nothing refused, and it attributed a
client inference to the target in the operator-visible reason. It also had no
test.

Silence is a third answer. Pusher.LastOutcome now reports applied, refused, or
unknown per ref, computed from what the target actually said, and the strategy
treats unknown as doubt — settled against the target's ref listing, never as a
refusal and never as success. This also deletes Params.TargetReportsRefStatus:
the Pusher holds the advertisement, so the capability question belongs to it
rather than being hand-threaded across two packages. LastOutcome answers by
value, so no caller can mutate or race with push state it does not own.

**The cleanup deleted the marker whenever the branch name existed, at any
hash.** A concurrent sync that created the branch at an older commit C left our
marker holding the only reference to C..tip; the marker was deleted anyway and
those commits became unreferenced — the full re-transfer this PR exists to
prevent, in the one case where the import genuinely had not landed. Cleanup now
requires the branch to be at the hash this run pushed; anything else keeps the
marker and says which hash it found. The test that asserted the old behaviour
seeded an unrelated hash, so it was asserting the bug: it now covers both
sides.

**Ordering, all from the review:**

- The settlement ran after the tail phase, leaving the ref listing minutes stale
  while tags pushed — long enough for a concurrent run to adopt the marker and
  start resuming from it before we deleted it. It now runs as soon as the last
  create has happened.
- The deferred delete was the only push in the new code not checked for
  refusal, so a refused cleanup was silent — the opposite of the identical case
  100 lines above. Checked per marker now.
- createConfirmed could read the previous branch's answer on the one path where
  a create plan exists but no push carries it (branch and marker both already at
  the source hash). The code now tracks whether the create was pushed and treats
  that path as what it is: a branch the target already holds, marker stale.
- The tail phase re-derived its fetch haves from the plans, re-claiming exactly
  the creates the cutover withheld. It uses completedRefs, the map the run
  maintains as ground truth.

Smaller: targetRefsNow guards a nil target session (latent via Probe/Fetch) and
documents that receive.hideRefs can hide a ref that is there — safe in this
direction, since a hidden branch reads as absent and keeps a marker that was
not needed; the redundant package-level listing wrapper is gone; and three
pre-existing fixtures still emitted "ng "-prefixed statuses that go-git encodes
a second time, which this PR's own comment warns about.

Still open, and still the user's call: the one-shot path defers marker cleanup
to a prune that Bootstrap() forbids — the same permanent-marker gap, inherited
from #116, now closable with the same listing.

Full suite green under -race, golangci-lint clean. Verified each new guard fails
its test when removed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7
Entire-Checkpoint: 01M1C0CFNPR1W8S4GFMSEHXXZZ
…elog it

Three items, one of which had already been overtaken by ec4a45f.

**RefRefused / TargetReportsRefStatus are gone**, replaced by a single
Pusher-side tri-state, so the doc-versus-code disagreement the review names no
longer exists, and the probe it shows no longer reproduces: a target that
cannot report per-ref status is now settled by the ref listing, and the marker
is deleted once the branch is confirmed at the pushed hash. A stale reference
to the removed field in the cutover comment is fixed.

What survives is the remark about permanence, and it is worth making
explicitly: keepMarker now states that on the replicate and sync routes prune
reaps a marker kept unnecessarily, on the bootstrap route nothing does, and
against a target that neither reports per-ref status nor answers a listing it
is permanent — accepted, in that order of harm, because a marker is inert
scaffolding the resume route cannot act on while its branch exists, and a
marker deleted in error costs a full re-import.

**Changelog.** Added an Unreleased section covering the fix and, separately,
the exit-code change: a refused temp-ref update now fails the run even under
--best-effort, where every other per-ref refusal is a warning, so a run against
a target that blocks refs/gitsync/* moves from 0-with-warnings to non-zero.
Called out for anyone alerting on exit codes.

**The fixture located the packfile by scanning for "PACK".** A ref named
refs/heads/PACKAGING puts that literal in the command list and yields an offset
inside it. Replaced with a walk of the pkt-line framing to the flush that ends
the commands, which is exact; the request router's own hasPack test used the
same scan and now shares the helper, so the footgun is gone from both. Covered
by a test that pushes exactly that ref name.

Full suite green under -race, golangci-lint clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7
Entire-Checkpoint: 01M1C12QKHMS0PPY5W1PB22EW0
…push funcs

The subsumed path checked only for an explicit refusal, so RefOutcomeUnknown
fell through to "finalized": against a target that reports no per-ref status, a
subsumed create that silently did not land was recorded in completedRefs and
counted in BatchCount. It now asks the same question the cutover asks —
createConfirmed, not refRefused — so silence stops counting as delivery.

Settled locally rather than against a ref listing, and the comment now says why
that is the right answer here rather than answering the scaffolding question
instead: a subsumed branch has no temp ref to preserve, so all that hinges on it
is the reported count and a have that trunk's own tip already covers. Neither is
worth a round trip, and under-reporting work this run cannot confirm matches the
direction taken everywhere else. Covered by a test that compares a silent
target's BatchCount against a confirmed run's.

Nit, also from the review: pushPack, pushCommands and pushObjects took the
unexported *pushStatusSink after the last round, so an external caller could
only ever pass nil. There are none — the Pusher methods are the API — so they
are unexported now, with their doc comments and one cross-reference updated to
match.

Full suite green under -race, golangci-lint clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BKBGBh3rJ6WGnZKDbBRy7
Entire-Checkpoint: 01M1C2XXGV79TFMF3KSK4ZANW1
@nodo

nodo commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 864584b. Configure here.

@Soph
Soph merged commit 4872470 into main Aug 31, 2026
5 checks passed
@Soph
Soph deleted the soph/batched-cutover-drops-resume-marker branch August 31, 2026 14:32
nodo added a commit that referenced this pull request Aug 31, 2026
…ther

Rebased onto main after #117 (Soph's batched-cutover marker fix) and squashed:
the five original commits were an iterative refinement of the same code, and
resolving their conflicts one at a time against the moved main produced worse
results than resolving the net change once. The review narrative lives in the
PR discussion.

Once subdivision bottoms out — checkpoints split BETWEEN commits, so at one
commit per gap there is nothing left to split — a single commit whose pack
exceeded git-sync's own budget failed the run. That budget is self-imposed and
far below what the target accepts: TargetMaxPack defaults to 512 MiB while the
target announces 10 GiB, and autoTargetMaxPackBytes derives 5 GiB from that
announcement and then discards it for being larger than the default. So the
run gave up against a number git-sync chose, having never asked the server.
gh/nicschick/vc3r dies exactly here.

The batching budget stays small on purpose — it bounds the waste of a doomed
push and makes the temp ref advance often, both of which require a smaller
pack to be possible. On a one-commit gap neither is, so the ceiling for that
push is the target's announced limit instead, chosen before the push rather
than after a doomed attempt. This costs nothing: a ceiling is an abort
threshold, not an upload size, so a pack that fits sends identical bytes
either way — and it avoids fetching an indivisible multi-GiB commit twice.

That gives the failure a verdict worth classifying. An abort against our own
budget stays retryable: a larger budget or a raised server limit could still
mirror the repo. A checkpoint that is indivisible AND refused by the target —
a parsed body-limit rejection, or an attempt at its announced limit that still
overshot — returns ErrCheckpointExceedsTargetLimit, aliased into the root
package so the mirror worker can match it with errors.Is and stop redelivering
an identical pack ten times. A deadline (408/504) is availability, not size,
so it stays retryable; classifying it permanent would let one target rolling
restart permafail every large bootstrap in flight.

Also stops discarding the bootstrap Result when Execute fails. The route facts
(RelayMode, RelayReason, batch counts, temp refs, Plans) are set before
anything can fail and describe the route rather than the outcome; zeroing them
is why a failed sync could report its strategy only when it succeeded — the
gap that made ENT-2054 a source read instead of a log query. Batching is
recorded before checkpoint planning, whose commit-graph fetch is the likeliest
failure for exactly the repos that batch, and unstable.Client.Bootstrap no
longer throws the result away either.

budgetFromObservation guards against escalating past a cutoff MEASURED from
bytes actually sent (a middlebox that cuts without announcing) rather than one
the target stated. It lives beside selfImposedBudget, which spans branches:
provenance has to travel with the value, or a later branch escalates past a
limit an earlier one demonstrated. It has no test coverage — a fixture I built
did not bite when the guard was removed, and I deleted it rather than keep a
test passing for reasons I could not explain.

Verified on the rebased tree: full suite and golangci-lint green, Soph's
cutover tests pass alongside these, and removing the escalation or treating a
deadline as a size verdict each still break the new tests.

Entire-Checkpoint: 01M1C4BS40PK95XMCB8MF5YH3Q
nodo added a commit that referenced this pull request Aug 31, 2026
…ther

Rebased onto main after #117 (Soph's batched-cutover marker fix) and squashed:
the five original commits were an iterative refinement of the same code, and
resolving their conflicts one at a time against the moved main produced worse
results than resolving the net change once. The review narrative lives in the
PR discussion.

Once subdivision bottoms out — checkpoints split BETWEEN commits, so at one
commit per gap there is nothing left to split — a single commit whose pack
exceeded git-sync's own budget failed the run. That budget is self-imposed and
far below what the target accepts: TargetMaxPack defaults to 512 MiB while the
target announces 10 GiB, and autoTargetMaxPackBytes derives 5 GiB from that
announcement and then discards it for being larger than the default. So the
run gave up against a number git-sync chose, having never asked the server.
The mirror that motivated this dies exactly here.

The batching budget stays small on purpose — it bounds the waste of a doomed
push and makes the temp ref advance often, both of which require a smaller
pack to be possible. On a one-commit gap neither is, so the ceiling for that
push is the target's announced limit instead, chosen before the push rather
than after a doomed attempt. This costs nothing: a ceiling is an abort
threshold, not an upload size, so a pack that fits sends identical bytes
either way — and it avoids fetching an indivisible multi-GiB commit twice.

That gives the failure a verdict worth classifying. An abort against our own
budget stays retryable: a larger budget or a raised server limit could still
mirror the repo. A checkpoint that is indivisible AND refused by the target —
a parsed body-limit rejection, or an attempt at its announced limit that still
overshot — returns ErrCheckpointExceedsTargetLimit, aliased into the root
package so the mirror worker can match it with errors.Is and stop redelivering
an identical pack ten times. A deadline (408/504) is availability, not size,
so it stays retryable; classifying it permanent would let one target rolling
restart permafail every large bootstrap in flight.

Also stops discarding the bootstrap Result when Execute fails. The route facts
(RelayMode, RelayReason, batch counts, temp refs, Plans) are set before
anything can fail and describe the route rather than the outcome; zeroing them
is why a failed sync could report its strategy only when it succeeded — the
gap that made ENT-2054 a source read instead of a log query. Batching is
recorded before checkpoint planning, whose commit-graph fetch is the likeliest
failure for exactly the repos that batch, and unstable.Client.Bootstrap no
longer throws the result away either.

budgetFromObservation guards against escalating past a cutoff MEASURED from
bytes actually sent (a middlebox that cuts without announcing) rather than one
the target stated. It lives beside selfImposedBudget, which spans branches:
provenance has to travel with the value, or a later branch escalates past a
limit an earlier one demonstrated. It has no test coverage — a fixture I built
did not bite when the guard was removed, and I deleted it rather than keep a
test passing for reasons I could not explain.

Verified on the rebased tree: full suite and golangci-lint green, Soph's
cutover tests pass alongside these, and removing the escalation or treating a
deadline as a size verdict each still break the new tests.

Entire-Checkpoint: 01M1C4BS40PK95XMCB8MF5YH3Q
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants