Skip to content

Conversation

@brian-hussey
Copy link
Member

🐛 Bug-fix PR

Before opening this PR please:

  1. make lint - passes ruff, mypy, pylint
  2. make test - all unit + integration tests green
  3. make coverage - ≥ 90 %
  4. make docker docker-run-ssl or make podman podman-run-ssl
  5. Update relevant documentation.
  6. Tested with sqlite and postgres + redis.
  7. Manual regression no longer fails. Ensure the UI and /version work correctly.

📌 Summary

This change addresses #1614. It works in the suggested change, ensures tests work as expected and make the tests more robust for the new model.
It removes the need for the "last_accessed" property in the CacheEntry class/objects.

🔁 Reproduction Steps

Link the issue and minimal steps to reproduce the bug.

🐞 Root Cause

Implementation of the cache could be improved.

💡 Fix Description

Using the suggested OrderedDict approach and testing around that.

🧪 Verification

Check Command Status
Lint suite make lint
Unit tests make test
Coverage ≥ 90 % make coverage
Manual regression no longer fails steps / screenshots

📐 MCP Compliance (if relevant)

  • Matches current MCP spec
  • No breaking change to MCP clients

✅ Checklist

  • Code formatted (make black isort pre-commit)
  • No secrets/credentials committed

@brian-hussey brian-hussey marked this pull request as ready for review December 17, 2025 17:06
@crivetimihai
Copy link
Member

Closes #1614

Signed-off-by: Brian Hussey <[email protected]>
@crivetimihai crivetimihai self-assigned this Dec 18, 2025
@crivetimihai crivetimihai merged commit debc41f into main Dec 18, 2025
61 of 62 checks passed
@crivetimihai crivetimihai deleted the bh/1614_LRU_perf_2 branch December 18, 2025 11:27
@crivetimihai
Copy link
Member

@brian-hussey Rebased and tested. The PR correctly implements the optimization described in issue #1614:

Aspect Original New (PR)
Storage Dict[str, CacheEntry] OrderedDict[str, CacheEntry]
LRU Tracking last_access field in CacheEntry OrderedDict maintains order
Eviction O(n) min() scan O(1) popitem(last=False)
Access Update entry.last_access = now move_to_end(key)

Bonus fix: The old code had a bug where updating an existing key at full capacity would unnecessarily evict another entry. The new code
handles this correctly.

Performance Benchmark Results

Cache Size Eviction Speedup Mixed Workload Speedup
100 8.5x 4.7x
500 35.5x 18.5x
1000 70.5x 38x
5000 305x -

The improvement scales linearly with cache size (O(n) → O(1)).

Please re-test after merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants