Hardening native rocksdb-js transaction log durability and retry-round drain tracking (follow-up to #3236)

@kriszyp

Follow-up to #3236.

Background

#3236 enforces in DatabaseTransaction that once a RocksDB transaction (or any link in a multi-store chain) has durably written its entry to the transaction log (hasDurableLog(transaction), tracked after a native transaction.commit() attempt with hasStagedLog), it is never aborted on conflict retries, commit-budget deadlines, record-lock lease expiry, or database-close commit suspension, and the caller stays awaiting the commit until it lands (matching sourceApply).

Native rocksdb-js changes and residual edges scoped out of #3236 into this follow-up issue:

Native rocksdb-js conflict-before-log ordering in Transaction::Commit (e0c5e60a1aaf, 9dcabe0ba98a):
- In rocksdb-js 2.12.0, Transaction::Commit runs executeLogWork (writing the transaction log entry and setting committedPosition) before txn->Commit() checks optimistic conflicts on the primary RocksDB column family.
- Because logging.auditLog defaults to true, every audited RocksDB write that conflicts on its first native commit attempt already has a durable log entry on disk, so under Never abort a transaction on retry once its entry has been written to the transaction log #3236 it must retry without an attempt cap or 503 deadline until the commit lands.
- Reordering or staging Transaction::Commit in rocksdb-js so optimistic conflict validation succeeds before executeLogWork flushes the log batch (or appending an abort marker that log readers skip) will allow conflicting request-path commits on audited tables to remain unlogged on conflict and preserve the Outstanding-commit wedge recurs on 5.2.6/5.2.7: native commits never settle and checkOverloaded 503s every application write on the thread until restart #2450 503 deadline and 40-attempt cap.
Native rocksdb-js durable-log state exposure (committedPosition > 0) and post-log abort refusal/reconciliation (07a62c6fd60d, 20be4feb68b1, f933af77479c, 03d6ac4d1556ea87, 95ccdec124307a0d, 6e8fbfce53749284, 0533586f774f1afc):
- Expose committedPosition > 0 (or a direct hasDurableLog accessor) from rocksdb-js on the native Transaction handle so DatabaseTransaction can read native log durability directly rather than inferring it from isRetry && hasStagedLog.
- Harden Transaction::Abort() / Close() in rocksdb-js to refuse or safely reconcile handles that already hold a durable log entry (covering a first-attempt non-conflict commit failure after executeLogWork flushed the log entry, a synchronous throw from a custom onCommit hook during retry re-save, or a store close after WAIT_FOR_DRAIN_DEADLINE_MS).
Tracking retry-round pre-commit completions awaits during waitForDrain() (e554d93b900e0d06):
- Extend outstandingCommits tracking across the full retry round (including any asynchronous this.completions await added by a custom onCommit hook on retry).
Propagating context.timestamp when creating a chained ImmediateTransaction link (fd98b84c9b2cb1a1, 8c1816872ad4f7f1):
- Copy context.timestamp into timestamp and lastContextTimestamp when constructing a .next ImmediateTransaction link in Table.ts:txnForContext so the first write to a second database inherits the caller's version.
Record-lock commit hold exclusion past commitConflictBudget (e7748380dccb1564):
- retainForCommit defers local lock unlock only up to commitConflictBudget (limitation 2 in resources/record-locks.md); once Transaction::Commit validates conflicts before executeLogWork, conflicting commits no longer outlive commitConflictBudget.
Direct raw eviction transaction commits in resources/Table.ts (ba9ebb8c973b5fe6, 9ea18b2db862):
- Single-record and batched eviction in resources/Table.ts:4129-4139 and resources/Table.ts:9947-9960 commit raw RocksTransaction handles via commitTrackedRocksTransaction outside DatabaseTransaction and call transaction.abort() if commit() rejects with ERR_BUSY. When replicateEvictions is enabled (saveAudit: true), executeLogWork() flushes an 'evict' log entry before txn_->Commit() checks for conflict, so aborting or replacing the handle on ERR_BUSY orphans that 'evict' log entry until conflict-before-log ordering lands in rocksdb-js or raw eviction commits recommit the same handle.
Per-save write-set rescan while a late immediate commit overlaps a retrying retained-iterator replay (6a76c39decccc6db):
- In that overlap activeCommitHandle is unset and the replay handle differs from this.transaction, so every re-save in the round runs backfillActiveHandleWrites over the whole write set (O(N²) for that round). Ordinary commits and replay rounds backfill once per round; once conflict-before-log ordering lands, the retry rounds that reach this path mostly disappear.
Edge cases of the stalled-commit monitor's logged-retry progress check (d5b79e28d4818b39, b6538f35ca1ef816, 3a5938b9fba8e158, 5151c6f2da9439a3):
- The check spares a logged retry only while the summed retry counter of the chain's live logged handles keeps moving (activeDurableLogRetries() / deferForCommitInFlight() in resources/DatabaseTransaction.ts); a counter that stops for LOGGED_RETRY_STALL_MS (2 × the native park timeout) is treated as a stalled native commit. Diagnostics only, no data loss: an advancing or departing sibling can hide a wedged logged handle, and a handover to the next link's handle at an equal total can read as a stall.
- The 2 × park window does not cover the up-to-1 s JS retry backoff once ROCKSDB_JS_PARK_TIMEOUT_MS is under about 1 s, and the parse of that variable differs from rocksdb-js's for non-integers and values under 50 ms.
- abortNativeTransaction skips a handle flagged commitInFlight; a synchronous throw between the flag being set and the commit settling would leave it set (nothing on that path throws today).
- Simplest direction if it matters: key the window on the per-handle retries of each live logged handle instead of one aggregate, and read the park timeout through rocksdb-js instead of mirroring its parser.

Review-Deferral: e0c5e60a1aaf severity=major category=reliability
Review-Deferral: 9dcabe0ba98a severity=major category=reliability
Review-Deferral: 07a62c6fd60d severity=nit category=code-quality
Review-Deferral: 20be4feb68b1 severity=nit category=code-quality
Review-Deferral: f933af77479c severity=minor category=test-coverage
Review-Deferral: 03d6ac4d1556ea87 severity=minor category=data-integrity
Review-Deferral: 95ccdec124307a0d severity=minor category=test-coverage
Review-Deferral: e554d93b900e0d06 severity=minor category=reliability
Review-Deferral: dd13e9a5c46676bd severity=nit category=test-coverage
Review-Deferral: 6e8fbfce53749284 severity=nit category=maintainability
Review-Deferral: 0533586f774f1afc severity=major category=reliability
Review-Deferral: fd98b84c9b2cb1a1 severity=nit category=correctness
Review-Deferral: 8c1816872ad4f7f1 severity=nit category=correctness
Review-Deferral: e7748380dccb1564 severity=nit category=concurrency
Review-Deferral: ba9ebb8c973b5fe6 severity=minor category=data-integrity
Review-Deferral: 6a76c39decccc6db severity=nit category=performance
Review-Deferral: d5b79e28d4818b39 severity=nit category=reliability
Review-Deferral: b6538f35ca1ef816 severity=nit category=reliability
Review-Deferral: 3a5938b9fba8e158 severity=nit category=reliability
Review-Deferral: 5151c6f2da9439a3 severity=nit category=reliability

🤖 Generated by Google Gemini, Claude Fable 5.1 and Claude Sonnet 5.5; posted via @kriszyp.