Skip to content

A write after ensureLoaded() on a RocksDB caching table is overwritten by the background source fill #2935

Open

Description

@kriszyp

On RocksDB, a write to a caching (sourcedFrom) table that follows ensureLoaded() on the same resource is lost: the background cache fill that ensureLoaded() started commits after the write and leaves the source's value in the table. The client's PUT is acknowledged, and nothing reports the loss. LMDB keeps the written value.

Reproduce

Observed on main at 2d81ebc (Node 24.21, HARPER_STORAGE_ENGINE=rocksdb), 3 of 3 runs:

const Target = table({
	database: 'repro',
	table: 'Target',
	attributes: [{ name: 'id', isPrimaryKey: true }, { name: 'value', indexed: true }],
});
Target.sourcedFrom(
	class extends Resource {
		get() {
			return { id: this.getId(), value: 7 };
		}
		put() {}
	}
);
class FillsFromSource extends Target {
	async put(data, target) {
		await this.ensureLoaded(); // fetches { value: 7 } and starts the background cache fill
		return super.put(data, target);
	}
}
await FillsFromSource.put('filled', { value: 8 });
(await Target.get('filled')).value; // 7 on RocksDB, 8 on LMDB

With the source's get setting this.getContext().noCacheStore = true (no fill), the record ends at 8, so the fill is what overwrites the write.

Expected

The write commits after the source fetch, so its version is newer than the fill's and the fill should be superseded (or skipped) rather than landing on top of it. Worth checking whether the fill's version or its commit ordering against a later local write is what differs between RocksDB and LMDB (getFromSource in resources/Table.ts, the background fill commit, and precedesExistingVersion).

Found while testing the write-path commit-base changes; this reproduces on main without them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P1

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions