Repository navigation
Plan accurate work steps and stop reading the clock per work unit - #115
Merged
Merged
Conversation
Adds the accurate-work-steps OpenSpec change: instructions as ground truth, size-aware charges, a sign-agreement acceptance test, and retiring steps behind a recall gate. WorkBudget::consume now reads the clock once per 64 units. On Indonesian and Amharic this took the deadline's cost from about 90 instructions per unit to 1.6-4.3, with work_steps unchanged. SynthesisBudget::steps() returned the work total, so it is now work_steps(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Start here:
openspec/changes/accurate-work-steps/design.md, thenrust/crates/pg-fst/src/work.rs.Adds the plan for making
work_stepsthe one work unit and retiringsteps, and lands the two pieces that are safe today. With a word deadline set, parsing no longer pays a clock read on every work unit: that cost falls from about 90 instructions per unit to 1.6-4.3, or from 3.5-4% of instructions to 0.06-0.19%.The main question for review is whether a timeout may now fire late.
WorkBudget::consumereads the clock once per 64 units, so a timeout can overshoot by at most 63 units. Timeouts are already machine-dependent and non-certifying. The cap check still runs on every reservation, andstopped()always reads the clock, so a poll-only loop still sees the deadline.a_deadline_stops_reservations_within_one_poll_intervalfails with the interval removed (checked by mutation).a_poll_observes_the_deadline_between_reservation_pollsandthe_cap_is_exact_between_clock_reads.work_stepswith and without a deadline on Indonesian and Amharic (1,613,681 and 32,238,244).SynthesisBudget::steps()returned the work total, so it is nowwork_steps(). No behaviour change.Not done here:
stepsstill exists everywhere, and no charge, cap, default, batch TSV column or ranking changes. Those are later task groups behind stated gates, including a recall gate before the attempt cap goes. The A/B test of whether a change inwork_stepspredicts a change in instructions has not run, so D5's thresholds are provisional.Base
71245076, head14eac325.pg.ps1 -Mode test: 2,244 passed, 66 skipped (before the last spec-only amend).-Mode preland: passed, with conformance skipped because no gated path changed. A forced-Allrun failed only that driver, becausehc-conformance.exefor the pin is not built here. No parse comparison against a base build was run on real grammars.Next: land the measurement tools (task group 2), then run the A/B set.
Evidence behind the plan
Measured on native Windows (i7-12700) over five real grammars, with PMC on context switch. Instructions repeat to 0.15-0.40% per word, core cycles to 5-17%. Instructions per step span 52x and per work step 2.2x. One per-grammar constant predicts a word's instructions to a median 3-14%.
Valgrind in WSL agrees with PMC at per-word log R² 0.995-0.999. Size-aware counters cut held-out median error from 18.6% to 9.0%, using bytes allocated and FST registers copied. Details and caveats are in
design.md. Raw data contains real-language words and stays local.Paths not taken
Score::keyto work now: a plain swap shrinksraw_pathsabout 30x and reopens a recorded mis-ranking, so it waits for a measured weight (D7).🤖 Generated with Claude Code