feat: LLM review stage for enhanced reachability detection

[joshbouncesecurity]

Summary

Adds a new LLM review stage (off by default, enabled via --llm-reachability on openant scan) that uses a strong model (Opus by default) to surface additional reachability signals beyond what the structural analysis catches:

• Likely entry points the structural pass missed (framework-specific handlers, plugin/CLI registrations, message handlers).
• External content ingestion (HTTP request bodies, file/network reads, env/argv, stdin, untrusted IPC).
• Cross-process or async data flows.

Signals are advisory and only promote a unit's reachability — they never demote one that the structural analysis already kept. This matches the issue's "complements, not replaces" intent.

Output:

• llm_reachability.json in the scan dir with the full signal list.
• llm_reachability_signals: [...] field on each unit in dataset.json.
• High-confidence entry_point signals set is_entry_point: true on the target unit.

Cost & rate-limit safety: reuses the existing GlobalRateLimiter via AnthropicClient, opt-in only, and prompts are batched (default 25 units/call).

Addresses #17 (does not close — let the maintainer review the prompt + heuristics first).

Test plan

• Unit tests for analyze_reachability with mocked LLM: fixed JSON, malformed JSON, exception in client, app_context threading, batch chunking.
• Unit tests for apply_signals: promote-only semantics (high-confidence promotes, medium does not, never demotes), per-unit signal accumulation, unknown-id rejection.
• CLI plumbing tests: --llm-reachability appears in openant scan --help; default does not pass the flag through to scan_repository; setting it threads llm_reachability=True.
• Existing pytest suite passes: 112 passed, 23 skipped (env-dependent, e.g. Go binary).
• Manual: enable on a small Express fixture; verify route handlers gain entry-point signals.

[joshbouncesecurity]

Manual verification

Off by default. Requires API key. Cost note: enabling adds approximately one Opus call per 25 units.

• openant scan --help shows --llm-reachability with the cost note.
• Default behavior: openant scan <repo> (no flag) — pipeline unchanged from current behavior; no LLM reachability cost incurred.
• Enabled: openant scan <repo> --llm-reachability — emits llm_reachability.json in the scan dir; merged signals appear on dataset.json units as llm_reachability_signals.
• Promote-only: high-confidence entry_point signals on a previously-unreachable unit promote it to is_entry_point: true. Existing entry points are NEVER demoted.
• Pipeline ordering: app-context generation runs BEFORE the LLM reachability stage, so app context is threaded into the reachability prompt. Verify with --quiet step ordering.
• Markdown-wrapped JSON: model occasionally returns the response inside a fenced json ... block — the parser strips the fences and recovers the signals.
• Empty units: openant scan <empty-or-failed-parse> --llm-reachability: stage skips gracefully (no LLM call).

[joshbouncesecurity]

Local test results

Reinstalled openant-core from this branch and ran openant scan --llm-reachability end-to-end on the in-tree sample_python_repo (5 reachable units after the structural pass). Skipped enhance/verify/report/dynamic-test to keep cost minimal.

Commands run:

go build -o openant.exe ./
pip install -e libs/openant-core/

openant scan <sample_python_repo> --output <out> \
  --llm-reachability \
  --no-context --no-enhance --no-report --skip-dynamic-test \
  --limit 1 --model sonnet

Outcome (against the manual-verification checklist):

• openant scan --help lists --llm-reachability with the cost note: "Off by default — enabling this may incur additional LLM cost (one Opus call per ~25 units)" ✅
• Enabled run: llm_reachability.json written to the scan dir; the new pipeline step llm-reachability ran successfully and emitted llm-reachability.report.json with cost_usd: 0.01245, token_usage.total_tokens: 782, units_reviewed: 5 ✅
• Promote-only semantics: Opus reviewed 5 units (the structural pass had already correctly tagged the two Flask endpoints as is_entry_point: True); model returned signals: []. signals_added: 0, entry_points_promoted: 0, units_touched: 0. Existing entry points were not demoted ✅
• Pipeline ordered as documented: parse → llm-reachability → analyze. llm-reachability.report.json is timestamped before analyze.report.json ✅
• Reachability stage runs Opus (per the report) regardless of --model sonnet (which only governs analyze) — matches cost note ✅
• Did not exercise the markdown-fence recovery path or the empty-units skip path here — covered by the unit tests in the diff. Did not separately confirm app-context-threading because I passed --no-context.

Total cost: $0.024 (reachability $0.012 + analyze 1 unit $0.012). Well under the budget.

[ar7casper]

🔴 High — Architecture · core/scanner.py:228 + core/parser_adapter.py:351

Issue: The LLM stage runs after parse, but parse already applied the structural
reachability filter when processing_level != "all" (default is "reachable"). So by the
time the LLM sees the dataset, units that weren't reachable from the structural entry
points are already discarded. The headline value prop — "Likely entry points the structural
pass missed (framework-specific handlers, plugin/CLI registrations, message handlers)" —
cannot be delivered as written: if the structural pass missed an Express route handler, the
unit for that handler was filtered out at parse time. The LLM never sees it. Promoting
is_entry_point on a unit that's no longer in the dataset is impossible.

What the stage does still deliver in current form:

• Attaches llm_reachability_signals to surviving units (metadata for reports)
• Promotes is_entry_point on surviving units, which flows into
  agentic_enhancer/prompts.py:106 (so the enhancer treats those units as entry points when
  prompting)

What it cannot do:

• Bring back units the structural pass dropped
• "Surface missed entry points" — those units are gone