Skip to content

feat(anatomist): add package classifying main-content layout from box geometry - #934

Merged
YusukeHirao merged 4 commits into
devfrom
worktree-streamed-sprouting-backus
Jul 30, 2026
Merged

feat(anatomist): add package classifying main-content layout from box geometry#934
YusukeHirao merged 4 commits into
devfrom
worktree-streamed-sprouting-backus

Conversation

@YusukeHirao

Copy link
Copy Markdown
Member

Summary

  • Adds @d-zero/anatomist, a new CLI/library package that renders a URL in a real browser and classifies its main-content child blocks into visual layout patterns (vertical stack, horizontal row, simple/complex grid, table, float-wrap) from measured box geometry (getBoundingClientRect), using CSS properties only as corroborating signal.
  • Exports @d-zero/beholder's main-content selector priority/fallback lists (MAIN_CONTENT_SELECTORS / MAIN_CONTENT_FALLBACK_SELECTORS) as shared constants so anatomist's own main-element resolution reuses the exact same heuristic instead of duplicating it.

Details

  • capture layer (capture-layout-tree.ts / capture-layout.ts): resolves the main-content element and walks its subtree into raw geometry + style + innerHTML via a single page.evaluate. Every helper is nested inside the evaluated function since Puppeteer only serializes that function's own source text into the browser realm.
  • classify layer (pure functions, no browser dependency): row/column geometry clustering, per-pattern detectors (table/grid/horizontal-row/float-wrap/vertical-stack), a carousel-overflow guard, and single-child-wrapper collapsing so a main > div.inner > div.container chain doesn't end classification immediately after the root.
  • orchestration (analyze-page-layout.ts / run-batch.ts): one URL across multiple viewports (PC → tablet → mobile, largest first) reusing one page; many URLs concurrently via @d-zero/dealer, each in its own tab of one shared browser.
  • CLI (cli.ts): reads a URL list from stdin or --input, writes JSONL to stdout or --out.
  • Fixture-driven integration tests (classify-layout-tree.integration.spec.ts) exercise capture → classify end-to-end against real HTML fixtures for each layout pattern, using @d-zero/beholder's actual selector constants.

Test plan

  • yarn build — all 29 projects build successfully
  • yarn test — 1854 tests pass across 142 files (including new fixture-driven integration tests)
  • yarn lint — no errors
  • /code-review medium, /qa-engineer, /product-manager reviewed; all findings addressed (see commit history for the fixes: closure-free page.evaluate bug, Math.max(...spread) RangeError risk, --concurrency 0 hang, duplicate row-clustering, dead code removal, bidirectional selector-sync test, missing --input/--out coverage)

@YusukeHirao
YusukeHirao requested a review from yusasa16 as a code owner July 30, 2026 11:10
@YusukeHirao
YusukeHirao merged commit 3e1fbd9 into dev Jul 30, 2026
6 checks passed
@YusukeHirao
YusukeHirao deleted the worktree-streamed-sprouting-backus branch July 30, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant