Skip to content

Add Exp 015: combined realistic conditions capstone (#123)#127

Merged
jimmytacks merged 1 commit into
mainfrom
experiment/015-combined-realistic-capstone
Apr 3, 2026
Merged

Add Exp 015: combined realistic conditions capstone (#123)#127
jimmytacks merged 1 commit into
mainfrom
experiment/015-combined-realistic-capstone

Conversation

@jimmytacks
Copy link
Copy Markdown
Collaborator

Summary

  • Capstone experiment: all realistic model improvements combined (cacheReliability=0.9, logarithmic growth, tool compression ratio=3, calibrated reasoning)
  • lcm-subagent costs $8.84 at 200 cycles under combined conditions — 2.5% cheaper than incremental
  • Strategy rankings completely stable across all conditions tested
  • 15-20% combined advantage hypothesis rejected — reasoning calibration compressed the cost structure
  • Production-grade cost range: $6.08 (CR=1.0) to $12.22 (CR=0.8) at 200 cycles
  • FINDINGS.md updated with combined-conditions section, production cost estimates, and programme completion status

Closes #123

Test plan

  • Three sweeps completed (baseline vs combined, session length, cache reliability)
  • Journal entry with quantitative analysis
  • FINDINGS.md updated with new section, revised recommendation table, experiment index entry

🤖 Generated with Claude Code

Under combined realistic conditions (cacheReliability=0.9, logarithmic
growth, tool compression ratio=3, calibrated reasoning), lcm-subagent
costs $8.84 at 200 cycles — 2.5% cheaper than incremental. Strategy
rankings completely stable across all conditions. The 15-20% combined
advantage hypothesis is rejected; reasoning calibration compressed the
cost structure, reducing percentage advantages.

This is the final experiment in the research programme. All 15 experiments
confirm lcm-subagent as the optimal strategy for the Models Agent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jimmytacks jimmytacks merged commit fc934d6 into main Apr 3, 2026
1 check passed
@jimmytacks jimmytacks deleted the experiment/015-combined-realistic-capstone branch April 3, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exp 015: Combined realistic conditions — capstone cost estimates

1 participant