Skip to content

Scale roadmap graph traversal - #15

Open
Deicyde wants to merge 2 commits into
facebookresearch:mainfrom
VivienCabannes:split/09-graph-scale
Open

Deicyde wants to merge 2 commits into
facebookresearch:mainfrom
VivienCabannes:split/09-graph-scale

Conversation

@Deicyde

@Deicyde Deicyde commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Cache containment and runtime node lookups while preserving the public graph models.
  • Replace recursive and quadratic graph walks with iterative indexed traversal.
  • Detect article mutation between graph loading and runtime projection.
  • Preserve ordering, diagnostics, and legacy Graph pickle compatibility.

Validation

  • 557 passed, 1 skipped
  • Python 3.10 graph compatibility: 20 passed
  • Randomized equivalence against the previous algorithms
  • 25,000-node traversal stress test
  • make lint
  • make check-example

Reference implementation: #8

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 19, 2026

@Deicyde Deicyde left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking correctness findings at ceca8e8.

The new indexed traversal matched the parent algorithms on 20,000 randomized graphs, and 25,000-node dependency/containment probes passed across the supported Python matrix.

Non-blocking follow-up: bulk publication still constructs each container view separately and rescans all relations in graph_pages.py:98,134-138 and graph_views.py:195,214-236. A synthetic 2,000-container graph took 5.28 seconds versus 1.42 seconds at 1,000. This appears pre-existing, so it should not block this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant