Skip to content

Exp 013: Summary growth dynamics (#119)#120

Merged
jimmytacks merged 1 commit into
mainfrom
experiment/013-summary-growth-dynamics
Apr 3, 2026
Merged

Exp 013: Summary growth dynamics (#119)#120
jimmytacks merged 1 commit into
mainfrom
experiment/013-summary-growth-dynamics

Conversation

@jimmytacks
Copy link
Copy Markdown
Collaborator

Summary

  • Ran 3 sweeps (84 total simulation runs) testing logarithmic vs fixed summary growth across all strategies, coefficient sensitivity, and interval interaction
  • Strategy rankings completely stable — lcm-subagent wins at all growth models, coefficients, and intervals tested
  • lcm-subagent advantage amplified from 8.2% to 12.1% over incremental at 200 cycles under logarithmic growth
  • 30k interval recommendation validated and strengthened — under logarithmic growth, 15k becomes 24-28% more expensive for incremental (double artefact eliminated)
  • Phase 1-3 conclusions are fully robust under more realistic summary growth modelling

Closes #119

Test plan

  • All three sweeps completed successfully with valid output
  • Results consistent with prior experiment baselines (fixed-growth numbers match Exp 003/008)
  • FINDINGS.md updated with new section, updated limitation UI: context size, cost breakdown, and cache hit rate charts #7, updated experiment index and programme status
  • Journal entry complete with hypothesis evaluation, quantitative results, and limitations

🤖 Generated with Claude Code

Ran three sweeps testing logarithmic summary growth vs fixed convergence
across all strategies, coefficient sensitivity, and interval interaction.

Key findings:
- Strategy rankings completely stable under logarithmic growth
- lcm-subagent advantage widens from 8.2% to 12.1% at 200 cycles
- 30k incrementalInterval validated (15k harmful under realistic growth)
- Coefficient sensitivity 10-24% on absolute costs, not rankings

Closes #119

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@jimmytacks jimmytacks merged commit 272786c into main Apr 3, 2026
1 check passed
@jimmytacks jimmytacks deleted the experiment/013-summary-growth-dynamics branch April 3, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exp 013: Summary growth dynamics — does logarithmic growth change cost balance or strategy rankings?

1 participant