Skip to content

mbe: compile rule RHS into a pre-flattened transcription template [clap_derive -0.5%, tt-muncher -1.8%] - #83

Open
xmakro wants to merge 1 commit into
perf/mbe-lookup-cachefrom
perf/mbe-flat-template
Open

mbe: compile rule RHS into a pre-flattened transcription template [clap_derive -0.5%, tt-muncher -1.8%]#83
xmakro wants to merge 1 commit into
perf/mbe-lookup-cachefrom
perf/mbe-flat-template

Conversation

@xmakro

@xmakro xmakro commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Follow-up to #57, stacked on #82. Transcription walked the RHS tree token by token on every expansion of a rule: one dispatch, one span mark, and one buffer push per literal token, plus a metavar normalization per occurrence and a full tree re-walk for every sequence lockstep check.

This lowers each rule RHS once (lazily, on first expansion) into a compiled template:

  • Runs of literal tokens, including whole literal delimited groups, are pre-encoded as flat entries with def-site spans. Transcription copies each run into the output buffer, rebases depths and match indices, and marks the copied spans in place. The per-token interpreter dispatch is gone for these.
  • Metavar occurrences carry their MacroRulesNormalizedIdent precomputed.
  • The sequence lockstep size check walks a per-rule precollected list of the metavars in the sequence subtree instead of re-walking the tree on every sequence entry.
  • Definitions that are never invoked pay nothing (the template is built behind a OnceLock).

Diagnostics are preserved: the sequence segments keep the spans and metavar ident lists the lockstep and no-repeatable-vars errors need, and the close-bracket spacing hack (rust-lang#114571) is replicated in the pre-encoded runs.

Performance (ThinLTO stage2, jemalloc, instructions:u, full scenario, measured on top of the perf stack; A/B over 10 crates x Check/Debug, base includes #82):

Correctness: stage2 build green; x check compiler green on this branch standalone; full tests/ui run has a failure set byte-identical to the pre-existing baseline (21301 passed, zero new diffs).

Transcription walked the RHS tree token by token on every expansion:
one dispatch, span mark, and buffer push per literal token, plus a
metavar normalization per occurrence. The tree is now lowered once per
rule (lazily, on first expansion) into a template whose runs of literal
tokens, including whole literal delimited groups, are pre-encoded as
flat entries. Transcription copies each run into the output buffer,
rebases depths and match indices, and marks the copied spans in place.
Metavar occurrences carry their normalization precomputed, and the
sequence lockstep check walks a precollected list of the metavars in
the sequence subtree instead of re-walking the tree.

clap_derive check -0.52%, html5ever check -0.46%, syn check -0.27%,
tt-muncher check -1.81%, no regressions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant