Skip to content

Stamp a record lock's writes above the version it was admitted on, even when that version is ahead of the clock - #3248

Merged
kriszyp merged 38 commits into
mainfrom
fix/lock-stamp-above-admitted-version
Oct 30, 2026
Merged

kriszyp merged 38 commits into
mainfrom
fix/lock-stamp-above-admitted-version

Conversation

@kriszyp

@kriszyp kriszyp commented Oct 16, 2026 •

Copy link
Copy Markdown
Member
A meme character rejects two customers holding identical deli tickets and approves one taking a fresh ticket from a dispenser cranked past the clock

⊙ Problem

A successful table.lock() write could lose under last-write-wins when the state it read had a version ahead of this node’s clock, including a future tombstone or a predecessor’s explicit timestamp. Serialization alone did not establish version order (#2540).

🤔 Keep the correctness fix

The owner chose a stamp floor and bounded clock lift. Allocator optimization remains follow-up #3245; this PR does not require another application benchmark budget decision.

💡 Solution

Admission floors put writes above both the reloaded record version and predecessor dependencies. A future floor within storage.maxLockClockSkew (default 5s) receives a unique claimed version. Claims persist coverage before publication; each write reserves a distinct log key. Beyond the skew cap, admission fails with LockUnavailableError (503), including deliberately future-stamped local records.

🤔 Accept the allocator cost as follow-up

Optimization remains #3245. At the earlier d6f79ed head, allocator controls measured baseline-adjusted throughput losses of 15% and 64% for one and four workers; application results were inconclusive. Those measurements predate the durability barrier. Current coverage extensions block on a storage flush, amortized with one-second headroom. They also reserve up to an extra second of restart clock lead. No current application performance result is claimed.

⚖️ Alternatives

The durable issuance ceiling prevents restart from reusing future-issued keys. Its immutable recovery snapshot lives inside the ratchet buffer, so it has the same lifetime; later workers do not consume unused headroom. Returning the same future floor to multiple holders repeats keys. Sleeping holds the lock while waiting for the clock.

🤔 Refuse writes joining an older in-flight commit

The owner chose a 409 before staging when a holder write would join a native commit reserved below its stamp. Unlock, acquire a fresh lock, and retry. Existing limitations remain tracked in #3238 and #3246.

🔧 Changes

Review ordering and recovery together: one admission read, coverage before claim, durable ceiling coverage, and refused sibling cleanup. The native package has no synchronous WAL-sync primitive, so coverage uses flushSync before any claimed key escapes, including when another worker already wrote the ceiling. Observed refusal detaches all unsaved sibling operations on that handle; saved writes remain attached for replay. The atomic initial-ceiling word seeds recovery once per live registry, including a zero seed.

Repeated hold claims can reach coverage even after clock catch-up, and each worker has its own covered-through cache. A refill can therefore synchronously flush memtables on multiple workers. Skipping unnecessary barriers or sharing only a proven durable ceiling remains optimization #3245.

🤔 Preserve collision-safe hold steps

A hold step that could repeat an issued key is claimed again, including after clock catch-up. It can therefore advance beyond a later plain write; plain writes remain ungated. The rebase preserves this existing uniqueness choice. Admission returns the captured image; ungated writes can race it. The Phase 0 tradeoff remains tracked in #3246.

🤔 Preserve frozen and mixed explicit clocks

An explicit transaction that already reserved a log key or staged an unlocked write keeps its existing clock. Lock ordering stays best effort there; admission does not move unrelated writes into the future. This preserves the existing contract. Stronger enforcement remains #2918. A hold saved after that transaction can claim its deferred floor. Inside an open explicit transaction, an unreserved, unmixed snapshot advances to a hold’s floor. A frozen or mixed clock below that floor fails with a 409 without marking the handle refused.

🤔 Preserve source-read availability

When coverage fails, source reads return data uncached. A raw version could repeat an unstaged holder stamp, so the fill neither stores it nor shares it with a caching waiter. The waiter retries with a fresh allocation; a source cannot clear the internal failure guard by changing context options. Repeated failures can repeat coverage attempts and origin fetches; this error-path cost preserves read availability without storing untrusted versions.

Changed files and verification surfaces

✅ Verification

Rebased onto 9487552e4 (main). Build, repository lint, changed-file formatting, design-note budgets and unit partition pass. 525 passing, one pending across the focused compiled suites (originClosedFloor.test.js, qa5043-explicit-txn-lock-stamp.test.js, recordLock.test.js, issuanceDurability.test.js, issuanceWorkers.test.js, recordLockCluster.test.js, recordLockCoordinator.test.js, updateAttributesLock.test.js, auditRefKinds.test.js, cacheFillClose.test.js, and configValidator.test.js); seven passing targeted regressions under type stripping. New regressions fail before their fixes; verification covers the durability barrier, live worker opens, concurrent source readers, hold-lock version floors across transactions, and caught refusals with sibling operations.

Broader verification and limits

Earlier local LMDB resources: 4,241 passing, 1,176 pending. Fulltext: 346 passing, six pending after refreshing stale native dependencies. The first broad RocksDB resources run failed against stale fulltext dependencies and a worker watchdog; focused reruns passed after correction. The 30-minute core run timed out without a summary; six previously failing tests passed in focused reruns. These are not complete current-head full-gate passes. The pushed-head CI matrix is the full unit/platform/integration route; its results are recorded below.

The new POSIX recovery regression kills the issuer and drops its disposable database’s WAL, then reopens before clock catch-up. It is a WAL-loss recovery proxy, not a physical power-cut or Pro transport test. Refusal containment uses real records, a pending blob commit and an explicit transaction. Live-worker and concurrent source-reader tests use real modules and a closed audit store, with no method stubs. A broader lock/cache selection had 443 passing, one pending and two blob-reclamation failures; the isolated five-test blob group passed. That combined run is not counted as a pass.

Current-head (ac849f267) CI passed: unit/resource and platform matrices, Node/Bun/uWS integration shards, packaging, and consumer installs.

Docs: existing companion documentation#749 covers native concurrency controls; the current core note states the issuance barrier and one-second recovery reserve.

PR QA: Do simultaneous future-floor admissions survive worker restart without losing successful writes? — queued @ b4a23e6, qa-pr-harper-3248-b4a23e6

Earlier QA evidence

Earlier PR QA qa-pr-harper-3248-cc1d217 passed a four-worker future-version workload, process kill/restart before clock catch-up, and a fails-on-base control. This predates the current rebase; it is not a new-head QA claim.

Framing-Verdict: chosen-approach-sound

Refs #2540

🤖 Generated by OpenAI Codex; posted via @kriszyp.

Release target: main — new storage.maxLockClockSkew setting and lock stamping behavior; no backport requested.

Related PRs: #2918 overlaps, #3490 overlaps, #3504 overlaps, #3527 overlaps, #3229 overlaps, #3356 overlaps, #3309 overlaps, #3266 overlaps, #2498 overlaps, #3539 overlaps, #3635 overlaps, #3655 overlaps, #3649 overlaps, #3572 overlaps, #3615 overlaps, #3610 overlaps, #3629 overlaps, #3685 overlaps, #3680 overlaps, #3605 overlaps, 23 others independent

Complexity: complicated

Tracked review deferrals

Deferred findings

🤔 defer this PR's major performance finding 5b4f188d4d0353ef (Upgrade claims an unused deferred floor) to #3245 instead of fixing it here?

🤔 defer this PR's major concurrency finding bafb653dd4cca780 (Captured lock image can miss ungated writes) to #3246 instead of fixing it here?

🤔 defer this PR's major reliability finding 54dd6c0505bee6e9 (Post-claim worker-local coverage can still refuse issuance) to #3245 instead of fixing it here?

🤔 defer this PR's major performance finding 06d7d29c6b3cf1b9 (Caught-up holder saves request synchronous issuance coverage) to #3245 instead of fixing it here?

🤔 defer this PR's major performance finding 3ef7da9737e68aeb (Strict issuance latch: CAS plus sync ceiling flush on every local key after one claim) to #3245 instead of fixing it here?

🤔 defer this PR's major performance finding 1d6523614540a0b3 (sync flushSync on each new issuance ceiling during a clock lead stalls writers) to a separate issue instead of fixing it here?

Retained review snapshots below refer to their named earlier commits.

Late Grok findings

16 surviving (highest major) — Grok reviewed 36644bc, ruled against 28ba8c4
  • major — resources/originClosedFloor.ts:212 — A peer-ahead claim permanently enables shared CAS; caught-up hold saves and scoped upgrades can still perform synchronous coverage
  • major — resources/originClosedFloor.ts:218 — During worker A’s covered lead, worker B’s failing coverage repeatedly rejects its allocations
  • major — resources/Table.ts:11587 — Failed allocation falls back to a raw version that can repeat an issued version and lose a write
  • major — resources/auditStore.ts:1025 — Power loss before WAL synchronization can discard a ceiling already used for issuance, allowing reissuance after restart
  • major — resources/Table.ts:4235 — After one refusal poisons a handle, another deferred save throws before detaching its operation
  • major — resources/recordLock.ts:231 — A deliberate local timestamp 60 seconds ahead makes subsequent lock admission fail until within the cap
  • minor — resources/recordLockCoordinator.ts:1657 — A Pro serializer dropping dependencyFloor would silently weaken relayed admission; that serializer remains unverified
  • minor — resources/record-locks.md:230 — Documentation still describes replica version clamping despite distinct-version apply
  • minor — unitTests/resources/recordLock.test.js:55 — An unmapped peer fixture follows local-origin routing, hiding the remote audit path
  • minor — unitTests/resources/nativeFullTextSchemaUpgrade.test.js:250 — A valid pre-header 503 now fails a fixture requiring stream destruction
  • nit — unitTests/resources/recordLock.test.js:1686 — Runtime configuration and zero-cap admission remain unverified
  • nit — utility/common_utils.ts:903 — A unitless 5000 configures 83 minutes rather than five seconds
  • nit — unitTests/resources/recordLockAdmission.bench.js:3 — Instructions name an absent wrapper; its compiled import prevents direct TypeStrip execution
  • nit — resources/Table.ts:11586 — Misleading tie wording and added setup narration remain after the clock explanations were corrected
  • nit — resources/recordLock.ts:213 — A test-only parser alias unnecessarily expands this module’s exports
  • major — resources/Table.ts:11583 — Coverage failure falls back to a potentially reused raw source-fill version (raised by the CLI: unmeasured hot path)

Not standing: none. Grok ran 24m (24m from launch to report); the fold adjudication (cursor-grok@36644bcba8da-88016bbd, codex) took 6m.

Late Grok findings

14 surviving (highest major) — Grok reviewed 7549dc4
  • major — resources/originClosedFloor.ts:212 — One future admission permanently enables shared CAS; caught-up holder saves keep requesting synchronous coverage
  • major — resources/originClosedFloor.ts:227 — Worker-local coverage failure rejects keys another worker already durably covered
  • major — resources/recordLock.ts:513 — Scoped-to-hold upgrade claims a deferred floor before any write
  • major — resources/recordLock.ts:228 — Locking a deliberately future-stamped local record returns 503 until clock catch-up
  • major — resources/Table.ts:11839 — Raw fallback can duplicate an unstaged holder version, causing its later write to deduplicate
  • major — resources/Table.ts:4268 — Two updates after refusal leave the earlier operation queued; commit aborts unrelated writes despite a caught 409
  • minor — resources/originClosedFloor.ts:413 — Opening another live worker promotes unused persisted headroom into the issuance ratchet
  • minor — resources/Table.ts:5234 — An ungated plain write can land after the captured image; an older-stamped put can be overwritten by computed holder data
  • minor — resources/recordLockCoordinator.ts:1657 — Pro relay preservation of dependencyFloor remains unverified; retain its existing follow-up
  • nit — utility/common_utils.ts:903 — Unitless 5000 means 83 minutes rather than five seconds
  • nit — unitTests/resources/recordLockAdmission.bench.js:3 — Instructions still require an unspecified environment wrapper
  • nit — resources/recordLock.ts:213 — Parser alias adds a production export solely for tests
  • nit — resources/recordLock.ts:201 — Direct runtime-configuration and zero-cap coverage remain absent; no malfunction demonstrated
  • nit — resources/DESIGN.md:212 — Conflicting ceiling durations, misleading clock explanations, and setup narration remain

Not standing: 9 merged, 1 separate-issue, 1 resolved. Grok ran 26m (26m from launch to report); the fold adjudication (cursor-grok@7549dc445fbd-ddf7345b, codex) took 4m.

Late Grok findings

6 surviving (highest major) — Grok reviewed 9e7356b
  • major — resources/DatabaseTransaction.ts:2345 — First refusal detaches only its operation; an untouched same-handle sibling can still reject the unrelated commit
  • major — resources/recordLock.ts:229 — A locally written now+60s version makes subsequent locks fail with 503 until its lead falls within the cap
  • minor — resources/Table.ts:5220 — Abort between registration and final reload can still return a hold; retained under follow-up lock-reload-after-abort
  • nit — utility/common_utils.ts:902 — An operator interpreting unitless string 500 as milliseconds instead configures a 500-second skew cap
  • nit — resources/recordLock.ts:213 — A test-only alias adds a redundant exported parser name
  • nit — unitTests/resources/recordLock.test.js:1913 — Added comments narrate fixture setup; aggregate also includes unitTests/resources/recordLock.test.js:1671-1675 and unitTests/resources/recordLock.test.js:2009

Not standing: 1 resolved, 1 dropped-factually-wrong, 1 separate-issue. Grok ran 25m (25m from launch to report); the fold adjudication (cursor-grok@9e7356be459d-7721b719, codex) took 5m.

Late Grok findings

5 surviving (highest major) — Grok reviewed b4a23e6
  • major — resources/recordLock.ts:229 — A deliberately local write stamped 60 seconds ahead prevents subsequent lock admission under the default cap until clock catch-up
  • nit — resources/recordLock.ts:213 — Parser refactors must preserve a redundant production alias used only by tests
  • nit — resources/recordLock.ts:201 — Parser and schema tests can pass while configured or zero-cap runtime admission remains unverified
  • nit — utility/common_utils.ts:902 — Configuring unitless 500 intending milliseconds permits 500 seconds of skew
  • nit — resources/Table.ts:5037 — Added setup narration and duplicated clock explanations remain across source, tests, and design prose

Not standing: none. Grok ran 14m (14m from launch to report); the fold adjudication (cursor-grok@b4a23e642f2e-0f737888, codex) took 3m.

Late Grok findings

5 surviving (highest major) — Grok reviewed d262d9d
  • major — resources/Table.ts:5164 — An explicit transaction({ timestamp }) with a caller-supplied historical timestamp below a locked record's version is raised to admitted.stamp. For a floor at or behind the clock, admissionStamp returns acquiredAt (resources/recordLock.ts:230), which is wall time, not just above the floor.
  • minor — resources/DatabaseTransaction.ts:2397 — The fixed-clock 409 tells the caller to save the held record first or in its own transaction. The catch at :2414 calls refuseLockWrites, which sets writeRefused on the handle, so that recovery also gets a 409. The hold accepts writes again only after unlock() and a new lock()
  • minor — resources/Table.ts:11841 — When the issuance-ceiling write fails inside freshLocalKey, getFromSource catches it with an empty catch. It only sets issuanceFailed and logs nothing, so a durable coverage failure looks like a quiet cache miss on every fill. On lock() admission the same failure is rethrown raw at :5144
  • major — resources/originClosedFloor.ts:213 — One in-cap claim ahead of the clock latches WORD_STRICT for the rest of the process. After that, every local key on every thread does a compare-and-swap on the shared ratchet word, plus a synchronous flushSync for ceiling coverage. No benchmark numbers are in the PR. Deferred as `strict-issuance
  • minor — resources/recordLockCoordinator.ts:1653 — dependencyFloor survives the relay only if harper-pro's transport forwards the whole round. The in-repo test passes the object back in-process, so a reply that dropped the field would still pass. Deferred as relay-dependency-floor-wire-unverified

Not standing: 3 merged, 2 separate-issue. Grok ran 15m (16m from launch to report); the fold adjudication (cursor-grok@d262d9d5874a-309178da, claude) took 1m.

Late Grok findings

7 surviving (highest major) — Grok reviewed 8c912c0
  • major — resources/Table.ts:5164 — An explicit transaction({ timestamp: historical }) that locks a record whose stored version is older than the clock has its whole clock raised to acquiredAt (about now), not just above the floor. Every unlocked write staged after the lock then commits at about now and silently overwrites newer v
  • major — resources/originClosedFloor.ts:213 — One in-cap admission ahead of the clock sets WORD_STRICT for the rest of the process. After that, every local key does a CAS (compare-and-swap) on the shared ratchet word, plus a synchronous flushSync coverage write per thread as keys pass the headroom. No benchmark numbers are recorded.
  • minor — resources/DatabaseTransaction.ts:2388 — A hold save inside an explicit transaction uses the native handle's read-snapshot time as the "effective clock". If that time is above the ratchet, it can lag wall time. The save then sets the transaction clock to an unclaimed #stepPast step behind the wall clock, and later writes in the transacti
  • minor — resources/DatabaseTransaction.ts:2396 — The 409 refusal now detaches only the current operation. Earlier deferred writes of the same hold, linked through pendingPriorWrite, stay attached. If the caller catches the 409 and commits, they are committed.
  • minor — resources/Table.ts:11841 — An issuance-ceiling write failure in getFromSource is swallowed with nothing logged. The same failure during lock() admission is rethrown raw at resources/Table.ts:5142-5144 and reaches the client as an untyped 500.
  • minor — config-root.schema.json:719 — storage.maxLockClockSkew, the 503 returned past the cap, and the hold-save 409 have no user-facing docs and no static/defaultConfig.yaml entry. They are documented only in internal resources/record-locks.md.
  • nit — resources/recordLock.ts:348 — Past the cap, #stepPast returns a far-future step without claiming it.

Not standing: 3 merged, 2 separate-issue. Grok ran 15m (16m from launch to report); the fold adjudication (cursor-grok@8c912c0f72b8-c2c38386, claude) took 1m.

Late Grok findings

2 surviving (highest minor) — Grok reviewed 89c9888
  • minor — resources/auditStore.ts:1031 — coverLocalIssuance logs through warnContained (resources/auditStore.ts:208-212), which has no rate limit. If the ceiling write or flushSync keeps failing (disk full, store closed) after a lock claim has moved the ratchet ahead of the clock, every local key minted in that lead window logs one
  • nit — resources/indexes/DESIGN.md:683 — wording change to the streaming-errors note in a record-lock PR. Not verified as unrelated: this pass had no diff tool.

Not standing: 3 accepted-known. Grok ran 21m (21m from launch to report); the fold adjudication (cursor-grok@89c98883448f-89f218c0, claude) took 1m.

Late Grok findings

none surviving — Grok reviewed ffaba1e

No finding survived adjudication.

Not standing: none. Grok ran 7m (7m from launch to report); the fold adjudication (cursor-grok@ffaba1e9f1d5-d8a6eddd, claude) took 1m.

Review-Coverage: authored=gemini; ran=cursor-composer,codex,muse,claude,cursor-muse,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=cursor-kimi; rounds=31; full=13 @ ac849f2

Review-Attention: deep ~65m (critical: DatabaseTransaction.ts, Table.ts +3; decisions: stamp-floor-over-fencing, skew-cap-503, durable-issuance-ceiling-1s-headroom, caller-timestamp-raise, do-less-alternative, release-target-unclear, deferred-5b4f188d, deferred-bafb653d, deferred-54dd6c05, deferred-06d7d29c, deferred-3ef7da97, deferred-1d652361) @ ac849f2

@kriszyp kriszyp added this to the v5.4 milestone Oct 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements record lock admission floor stamping (harper#2540) to ensure lock holder writes outrank predecessor states, introducing configuration for maximum lock clock skew and local issuance ceiling persistence. The code review identified several key issues: a potential TypeError in highestPosition when dependencies is null, a potential lock leak in acquireRecordKey if handle creation fails, incorrect handling of numeric clock skew configurations, and comparison bugs involving undefined values in lockStampCeiling and Table.ts clock-lifting logic.

Comment thread resources/DatabaseTransaction.ts Resolved
Comment thread resources/recordLockCoordinator.ts Resolved
Comment thread resources/recordLock.ts Resolved
Comment thread resources/recordLock.ts Resolved
Comment thread resources/Table.ts Resolved
@kriszyp
kriszyp marked this pull request as ready for review October 16, 2026 22:54
@kriszyp
kriszyp requested a review from cb1kenobi October 16, 2026 22:54
@kriszyp
kriszyp force-pushed the fix/lock-stamp-above-admitted-version branch from fb7ac41 to e96c245 Compare October 20, 2026 04:41
@kriszyp
kriszyp force-pushed the fix/lock-stamp-above-admitted-version branch from fb57ab0 to 6173745 Compare October 23, 2026 02:47
@kriszyp
kriszyp marked this pull request as draft October 23, 2026 16:54
@kriszyp
kriszyp force-pushed the fix/lock-stamp-above-admitted-version branch from 6173745 to d6f79ed Compare October 25, 2026 05:22
@kriszyp
kriszyp force-pushed the fix/lock-stamp-above-admitted-version branch from d6f79ed to b4a23e6 Compare October 28, 2026 19:23
kriszyp and others added 4 commits October 29, 2026 16:40
WIP: admission floor, dependencyFloor on LockRound, explicit-clock raise, immediate-batch log-key ceiling, nextafter hold step. Docs and the log-key uniqueness decision pending.

Refs #2540

Dispatch-Task: harper-lock-stamp-above-predecessor
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KqM5AXB9v8MoxFzS6myGDM

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
An admission floor ahead of this node's clock was the same for every holder admitted on state one
future-stamped transaction left, so their explicit transactions shared it as version and log key; a
shared counter then lost an increment as a re-delivery. Within storage.maxLockClockSkew (default 5s)
the floor is now claimed from the local key ratchet, which lifts every later local key above it. Past
the cap the floor stays explicit and a rate-limited clock-skew warning is logged. Documents the stamp
floor in record-locks.md (Phase 0, Phase 1 contract, §2, §7.3, §10, §12, §14) and DESIGN.md.

Refs #2540

Dispatch-Task: harper-lock-stamp-above-predecessor
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KqM5AXB9v8MoxFzS6myGDM

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
A claimed key lived only in shared memory, so a crash and restart before the clock passed it could
issue it again. A ceiling 50 ms above each claim is now persisted before the claim is used and seeds
issuance at open. A malformed storage.maxLockClockSkew falls back to 5s with a warning. Documents the
in-flight immediate-batch residual (harper#3238).

Refs #2540

Dispatch-Task: harper-lock-stamp-above-predecessor
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KqM5AXB9v8MoxFzS6myGDM

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
…cap floors off the clock

Review round 1: the persisted ceiling now lives in the allocator, so any key issued from the ratchet
ahead of the clock (a claim, the keys after it, or keys after an open-time seed) is covered before it
is used, including by other threads issuing above a claim. A floor the clock already passed takes the
clock's next key without a claim or a metadata write; a floor past storage.maxLockClockSkew (or with a
cap of 0) moves neither the node's nor the explicit transaction's clock, so its far-future stamp does
not spread to the transaction's other writes. The cap rejects non-string, non-number values, and the
skew is measured on the monotonic clock. Lock admission checks for an unlocked staged write with one
resumable pass instead of a rescan per lock.

Refs #2540

Dispatch-Task: harper-lock-stamp-above-predecessor
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KqM5AXB9v8MoxFzS6myGDM

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
kriszyp and others added 23 commits October 29, 2026 16:41
…cleanup loudly

Review: a write refused at key reservation stayed attached and the next commit sweep staged it
after its caller had seen the error. The lock suite's issuance-lead cleanup now asserts it
cleared instead of expiring silently.

Refs #2540

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wcMn5ZU2ZtPkYfmuLKvfn

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
A failure while reserving a durable issuance ceiling used to leave the unsaved holder write attached to its transaction. Detach that write before returning the failure, and document that callers must reacquire a lock before retrying.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Keep lock-write refusal state with the native key until release so a retry through a retained resource rejects. Reset unlocked-write scan state when an unlocked operation is detached, and bound the issuance-ceiling cleanup waits.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Reject lock admissions whose state is beyond the configured skew cap, scope write refusal to the lock handle, and parse skew durations without misreading milliseconds. Add regression coverage and a multi-worker benchmark.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Validate the converted millisecond value for numeric and unit-suffixed maxLockClockSkew settings so overflow cannot silently fall back at runtime.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Reject refused lock-handle operations in the transaction save path, including deferred explicit-transaction writes. Keep off-key writes through a held resource instance independent from that handle.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Keep the explicit lock stamp assertion separate from the local transaction log key after retry.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Persist issuance coverage before publishing ratchet claims, defer unadopted holder floors until a holder write, and keep refusals scoped to their failed operation.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Keep the design-note line budget and annotate intentionally throwing generator fixtures without changing runtime behavior.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Terminate the schema-upgrade HTTP fixture after a streamed failure and assert client rejection. Allow worker startup outside the lock-wait elapsed bound.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Describe distinct record versions and log keys, primed hold upgrades, and collision-safe steps after catch-up without changing the implementation.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Flush metadata coverage before returning future-issued keys and reserve one second of headroom. Detach a caught refused deferred save so unrelated writes can commit. Add real process/WAL-loss and refused-save regressions.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Use registered peer IDs for future-state fixtures and keep benchmark imports in the selected source or compiled namespace.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Seed the persisted reserve once per live shared registry. Keep failed-coverage source outcomes uncached and reject refused mutations before staging, detaching unsaved siblings when refusal is observed.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Share ownership-aware refusal cleanup between transactions and table guards. Assert immediate first-refusal cleanup, real reservation failure containment, fresh reentry after unlock, and recovery after a full native close.

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Run noteCommittedLockVersions when a transaction commits a lease-protected write even without its own recordLocks map, and adopt or enforce a surviving hold lock's holderVersionCandidate() floor inside open explicit transactions (including after a mid-scope commit) without ratcheting normal-time hold reuse.

Dispatch-Task: harper-3248-hold-lock-residual-arms

Co-Authored-By: Gemini <noreply@google.com>
…old retryable

- Track #committedHolderVersion separately from #lastHolderVersion on
  KeyLockHandle so holderVersionFloor() does not step above acquiredAt on a
  scoped-to-hold upgrade before any holder write has committed.
- Use transaction.getTimestamp() for the unreserved native handle clock in
  DatabaseTransaction.save() when checking whether a surviving hold's floor
  exceeds the transaction's effective clock.
- Detach the operation without calling refuseLockWrites() when an open
  explicit transaction rejects a hold save whose floor exceeds an already-fixed
  transaction clock, allowing the caller to save the held record in its own
  transaction as the 409 message advises.
- Bound the child exit promise in issuanceDurability.test.js and the clock
  catch-up loop in originClosedFloor.test.js.

Dispatch-Task: harper-3248-hold-lock-residual-arms
Co-Authored-By: Gemini <noreply@google.com>
… 409

- Detach all unsaved owned writes on the same lock handle (via refuseLockWrites
  with poisonHandle=false) when an open explicit transaction rejects a hold
  save whose floor exceeds an already-fixed transaction clock, preventing
  earlier deferred updates on that handle from committing when the outer
  transaction catches the 409 and commits.
- Declare storage.maxLockClockSkew: 5s in static/defaultConfig.yaml.
- Log a rate-limited warning in Table.getFromSource when freshLocalKey throws
  and the source read falls back to an uncached response.
- Inline the exit timeout in unitTests/resources/issuanceDurability.test.js.
- Extend unitTests/resources/recordLockAdmission.bench.js to run ordinary puts
  across all worker threads and time the first post-claim lock+save (measured
  across 4 threads: 1907 op/s before claim vs 1511 op/s after claim for locked
  writes, 13702 op/s before claim vs 8608 op/s after claim for ordinary puts,
  and 52.33 ms for the first post-claim lock+save with durable ceiling flush).

Dispatch-Task: harper-3248-hold-lock-residual-arms
Co-Authored-By: Gemini <noreply@google.com>
…ng logging, and test hygiene

Dispatch-Task: harper-3248-hold-lock-residual-arms
Co-Authored-By: Gemini <noreply@google.com>
…ore merge-base DESIGN.md

Dispatch-Task: harper-3248-hold-lock-residual-arms
Co-Authored-By: Gemini <noreply@google.com>
Using database 'test' shares the root store with originClosedFloor.test.js in
the resources-core-a-q shard, where strict ratchet admission is enabled for
the rest of the process and advances WORD_RATCHET on every native commit.

Dispatch-Task: harper-3248-hold-lock-residual-arms
Co-Authored-By: Gemini <noreply@google.com>
…ESIGN.md budget after rebase

Dispatch-Task: pr-maint-c59ff75810ef1a06620560cf8efb862e
Co-Authored-By: Gemini <noreply@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants