Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
570f56c
Adopt transaction-log abort markers by default
kriszyp Nov 2, 2026
a7ae6eb
Describe both abort-marker modes in resources/DESIGN.md
kriszyp Nov 2, 2026
c31175e
Pin forced-conflict tests to the markers-off path; cover a logged com…
kriszyp Nov 2, 2026
f954683
Certify marker-era log history per process and per log; tighten the c…
kriszyp Nov 2, 2026
4e8e615
Keep a refused chain link retrying after a committed store; fix revie…
kriszyp Nov 2, 2026
d2b4e33
Keep the re-log's blob ownership flags; trim narration
kriszyp Nov 2, 2026
4fc45a8
Log an abort rocksdb-js could not mark as an error
kriszyp Nov 2, 2026
5da066c
List the abort-marker setting module in resources/MAP.md
kriszyp Nov 2, 2026
3b7ca58
Fit the abort-marker note into resources/DESIGN.md's ceiling after th…
kriszyp Nov 2, 2026
c8dfa03
Clear rotatedAfterCommit with the other scope flags
kriszyp Nov 2, 2026
f3e5cae
List transactionLogAbortMarkers.ts among the review-coverage framing …
kriszyp Nov 2, 2026
6feed99
Keep rotatedAfterCommit sticky past the end of the scope
kriszyp Nov 2, 2026
c0e134b
Release a discarded batch's blob claim on re-log; say when a capped a…
kriszyp Nov 2, 2026
1564a87
Say the outcome is unknown on a deadline 503 whose abort could not be…
kriszyp Nov 2, 2026
227635c
Add storage.rocks.transactionLogWriteIntents (default off), and state…
kriszyp Nov 2, 2026
3947f9c
Wait for submission before releasing the intent holder in the re-log …
kriszyp Nov 2, 2026
5fb516d
Turn write intents on only for an explicit true value
kriszyp Nov 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/review-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@ jobs:
resources/residencyPolicy.ts
resources/search.ts
resources/subscriptionCheckpoint.ts
resources/transactionLogAbortMarkers.ts
server/http.ts
server/transactionLogCooling.ts
utility/environment/systemInformation.ts
Expand Down
12 changes: 6 additions & 6 deletions resources/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ Each encoder has its own `StoreRecordObject` prototype, so getters stay per tabl

## Struct mode is gated to primary DBIs (downgrade compatibility)

v4 (msgpackr v1) reads struct headers `0x20–0x3f` as integers, so a struct-encoded `__dbis__` breaks a v5β†’v4 downgrade. `OpenDBIObject` allows structs only on primary stores (opt-in); `databases.ts` forces them on for custom-index object stores. `RecordEncoder` bails the write hook (`_writeStruct = () => 0`), never clears it: clearing moves the fixint boundary, so the read hook would misread `0x20–0x3f`. The read hook stays, and structon's `prepareStructures` saves records-mode structures as the plain array v4 reads (`recordEncoder.test.js`).
v4 (msgpackr v1) reads struct headers `0x20-0x3f` as integers, so a struct-encoded `__dbis__` breaks a v5β†’v4 downgrade. `OpenDBIObject` allows structs only on primary stores (opt-in); `databases.ts` forces them on for custom-index object stores. `RecordEncoder` bails the write hook (`_writeStruct = () => 0`), never clears it: clearing moves the fixint boundary, so the read hook would misread `0x20-0x3f`. The read hook stays, and structon's `prepareStructures` saves records-mode structures as the plain array v4 reads (`recordEncoder.test.js`).

## Non-`initStore` structure saves read inside their own transaction

Expand Down Expand Up @@ -146,8 +146,8 @@ Closing the owning handle releases a fill's lock, so `releaseRecordLock` ignores

Pre-commit blob files land before a commit that can still skip the write.

- A commit handler resets `write.skipped` each round and sets it only when the round staged neither record nor audit entry (an audit-only `auditRecordToStore` entry is not a skip, and passes `write` to `updateRecord` for `stagedAudit`). Skips include a duplicate tie, a newer put, a losing no-audit full update, a no-audit patch that ties the head, meets a head with no record, is already received or keeps nothing, a losing invalidate, relocate or delete, and a source version change. A skipped write's `replicatedConfirmation` falls back to the stored version (harper#3182), so an unmarked skip looks like a write that replicates nothing.
- Skips are cleaned after commit, never inline: a retry can make the skipped write win. Abort cleans all but files a durable log or audit entry owns (`hasDurableLog`, `blobsAuditReferenced`) and writes of another in-flight handle.
- A commit handler resets `write.skipped` each round and sets it only when the round staged neither record nor audit entry (an audit-only `auditRecordToStore` entry is not a skip, and passes `write` to `updateRecord` for `stagedAudit`). Skips: duplicate tie; newer put; losing no-audit full update; no-audit patch tying the head, meeting a recordless head, already received or keeping nothing; losing invalidate, relocate or delete; source version change. A skipped write's `replicatedConfirmation` falls back to the stored version (harper#3182), so an unmarked skip looks like a write that replicates nothing.
- Skips are cleaned after commit, never inline: a retry can make the skipped write win. Abort cleans all but files a durable log or audit entry owns (`abortedWithoutLog`, `blobsAuditReferenced`) and writes of another in-flight handle.
- `cleanupUnusedBlobs(blobs, retainedFileIds)` spares the committed record's files. A deleted file tombstones its `Blob`, so `saveBlob` throws on it (#2062).
- `cleanupOrphans` scans every retained audit value: history owns its files.
- `evict()` and source-miss removals unlink only from their own commit, for the version and expiry they read (`evictionBlobs.test.js`).
Expand All @@ -169,10 +169,10 @@ Pre-commit blob files land before a commit that can still skip the write.
- The owner's commit chains on `monitorCommit`. The pre-commit phase is spared `COMMIT_PHASE_GRACE` ticks (`sourceApply`/`isReplay` without limit): its writes are sealed, and poisoning there once reported success after unlinking blobs.
- After the pre-commit await, the commit rechecks liveness or resolves as a phantom. **The monitor never aborts a submitted native commit.**
- `timeoutBudget` is a sticky floor that chained links inherit.
- Conflict retries get one deadline per logical commit (503 `TransactionCommitConflictTimeoutError`), not per attempt: the per-attempt clock (`trackOutstandingCommit`) drives `checkOverloaded()` shedding and must stay per attempt.
- **A handle with a durable log entry is never aborted on retry** (harper#3236): that orphans an entry replay applies (`noAbortAfterTxnLog.test.js`).
- Conflict retries get one deadline per logical commit (503 `TransactionCommitConflictTimeoutError`); the per-attempt clock (`trackOutstandingCommit`) drives `checkOverloaded()` shedding.
- **Abort markers let a retry abort a logged handle and dedup trust its log** (harper#3236; `mustRetryLoggedCommit`, `mayReadOwnOrphan`). After a restart dedup waits ~3d for older log files to purge; meanwhile a twin delivery may double-count.
- Each hop of a sealed commit cascade gets a full idle window, never the previous hop's remainder (`renewChainForNativeCommit`).
- Long-lived handles are reported from main only, because rocksdb-js's registry is process-global (`runLongLivedTransactionSweep`, past `storage.longTransactionReportThreshold`; `0` disables); workers attribute their own by the same id (`reportIfLongLived`). Stuck-commit logs list holders from every database regardless of age: the verification table is one process-wide array (`longLivedTransactions.test.js`).
- Long-lived handles are reported from main only: rocksdb-js's registry is process-global (`runLongLivedTransactionSweep`, past `storage.longTransactionReportThreshold`); workers attribute their own by the same id (`reportIfLongLived`). Stuck-commit logs list holders from every database regardless of age: the verification table is one process-wide array (`longLivedTransactions.test.js`).

## A client disconnect cancels the request's writes

Expand Down
Loading
Loading