Skip to content

members.yml promises a collaborative phase the orchestrator never runs — implement it as a structured challenge round #36

Description

@minerva-sky

Loop: loop:self · State: status:analyzed · Class: code-feature (L0 — proposing only)

The drift

.architecture/members.yml documents a three-phase review process, including:

collaborative_phase: "Members confer to discuss findings, resolve conflicting perspectives, and prioritize recommendations."

The orchestrator (skills/architecture-review/SKILL.md, step 5) implements no such phase. Members never see each other's findings; the orchestrator aggregates solo and, on disagreement, explicitly surfaces the trade-off "rather than picking a winner." The spec promises deliberation; the implementation does summarization. Either the promise or the implementation should change.

Hypothesis

Filling the gap with a structured challenge round — rather than open-ended "discussion" — raises the precision of critical findings at modest token cost. Concretely: after parallel individual reviews, each critical/high finding is sent to the one persona best positioned to refute it (performance challenges security's mandatory-validation critical, pragmatic-enforcer challenges everyone's scope creep, etc.) with an explicit instruction to attack it or state "no objection". Findings that survive keep severity with a "challenged by X, survived" note; refuted ones are downgraded with the rebuttal recorded verbatim.

Plausible-but-wrong criticals are the most expensive review failure mode — they burn maintainer trust in the whole document. An adversarial pass is the cheapest known filter for them, and it deepens the multi-perspective mechanic itself (WORLD.md: the differentiator), because the personas finally interact instead of running as eight isolated monologues.

Evidence

  • The drift is verifiable by reading the two files side by side (members.yml review_process vs SKILL.md step 5).
  • Adversarial verification as a precision filter is standard practice in multi-agent review setups; the framework already half-endorses it (pragmatic_enforcer's whole role is challenging the others — but today it reviews the target, never the findings).
  • ADR-013 (orchestrator + subagent delegation) makes this cheap to add: it's one more dispatch wave using infrastructure the skill already has.

Experiment

A/B on one review target: run the skill as-is and with the challenge round. Count criticals in each; maintainer blind-rates each critical as actionable / overblown / wrong. Success = challenge round removes or downgrades at least one overblown critical without killing any genuine one.

Cost

Markdown-only: one new step in skills/architecture-review/SKILL.md, a "Challenges" subsection in assets/review-template.md, and a members.yml review_process wording update so spec matches implementation. Zero runtime dependencies. Token cost ≈ one extra subagent call per critical/high finding (typically 3-6 per review). Fully reversible.

Alternative (cheaper, weaker)

If the challenge round is judged too heavy: fix the drift the other way — reword collaborative_phase in members.yml to describe what the orchestrator actually does (cross-cut analysis + explicit trade-off surfacing). One-paragraph docs fix, but it spends the differentiator instead of deepening it.

Prior-art check

Closed/deferred issues and .architecture/deferrals.md reviewed: the Phase 4B "enhance collaborative discussion integration" line is a pragmatic-mode tuning deferral, not a decision about the review pipeline's missing phase. ADR-013 chose the orchestrator pattern but did not decide against inter-member interaction — it's silent on it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    loop:selfForward-thinking improvement loop (WORLD.md-aligned)status:analyzedAnalyzed, awaiting decision

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions