Skip to content

SDK recovery primitive: historical public-note backfill by tag #416

Description

@haseebrabbani

Sub-issue of #357. Spike #412 benchmarked the approach and confirmed it is fast without any server-side changes.

Context

Public notes addressed to an account are on-chain, but normal sync starts from the store's global cursor. In a shared dirty store the cursor may already be past blocks containing a recovered account's notes (H < B ≤ T), and a fresh store has no efficient path at all — full syncState() is the "hours" path #357 describes.

Spike findings (#412), testnet at ~1.61M blocks:

  • Genesis→tip tag-scoped SyncNotes for one tag: 0.34 s zero-match, 2.0 s at ~3k matching blocks, 44.5 s at 135k matching blocks. Cost scales with matches, not range length.
  • For contrast: fresh-store syncState() with that dense tag took 11.8 hours (~950× slower).
  • Both the Rust and WASM/TS syncNotes paginate the full requested range internally — no client-side chunking needed.
  • Safety cap: miden-client errors (PaginationError) after 1000 pages per tag-chunk rather than truncating.
  • The creation-block anchor is therefore optional at current network scale (tracked separately; see the anchor issue) — genesis is an acceptable v1 lower bound and this primitive is unblocked now.

Scope

  • TS: backfillPublicNotesByTag(midenClient, { accountId, midenRpcEndpoint, fromBlock = 0, toBlock = tip }) using the raw WASM syncNotes.
  • Rust: backfill_public_notes_by_tag(account_id, from_block, to_block) on MultisigClient using the existing sync_notes wrapper.
  • Build the account's standard tag (NoteTag::with_account_target), scan the range, keep public notes, fetch bodies/proofs, import individually as NoteWithProof.
  • Never mutate the global sync height — normal forward sync runs afterwards.
  • Catch PaginationError and fall back to client-side range splitting; report, don't fail recovery.
  • PublicBackfillReport { scannedFrom, scannedTo, discovered, outcomes }.

Acceptance criteria

  • Both SDKs with semantic parity.
  • The dirty-store scenario test: coverage at H, note committed at B, global cursor at T with H < B ≤ T — recovery discovers the note at B without rewinding T.
  • Duplicate results and tag collisions are tolerated (tags are best-effort filters).
  • Pagination-cap fallback covered by a test.
  • Docs: notes sent with unrelated custom tags are outside this guarantee.

Unblocked (spike complete).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions