Skip to content

Scale tech tree with indexed API and client lazy-loading#37

Open
yodakohl wants to merge 1 commit into
mainfrom
codex/refactor-tech-tree-for-scalability
Open

Scale tech tree with indexed API and client lazy-loading#37
yodakohl wants to merge 1 commit into
mainfrom
codex/refactor-tech-tree-for-scalability

Conversation

@yodakohl
Copy link
Copy Markdown
Owner

@yodakohl yodakohl commented Jan 1, 2026

Motivation

  • The previous implementation loaded and saved the entire tech tree on every operation, which does not scale to millions of technologies.
  • Provide fast search and incremental navigation so the UI can load just the visible subset of the graph.
  • Allow per-item writes instead of writing the full dataset to avoid heavy I/O and contention.

Description

  • Add indexed server-side endpoints and helpers: buildIndexes, searchBuckets, collectNeighbors, and query handling for /api/tech-tree supporting search, roots, neighbors, ids, limit and offset.
  • Implement per-tech write endpoints POST, PUT, and DELETE on /api/tech-tree and keep indexes in sync by calling rebuildIndexes after saves.
  • Switch client to lazy-loading: load a limited set of root nodes with loadRoots, expand neighbors on demand with loadNeighbors, and ingest incremental updates with ingestTechs to avoid fetching the entire dataset.
  • Maintain compact in-memory maps on the client (techStore, prereqMap, dependentsMap, dependentsCount, edgeKeys) and edge deduplication, plus debounced server-side search usage and incremental createTech/updateTech/deleteTech flows.

Testing

  • No automated tests were run for this change.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant