Skip to content

feat: per-stage traceability enforcement with deterministic JSON sensor#401

Open
jeromevdl wants to merge 2 commits into
awslabs:v2from
jeromevdl:feat/traceability-enforcement
Open

feat: per-stage traceability enforcement with deterministic JSON sensor#401
jeromevdl wants to merge 2 commits into
awslabs:v2from
jeromevdl:feat/traceability-enforcement

Conversation

@jeromevdl

Copy link
Copy Markdown

Summary

Adds element-level traceability enforcement across the AIDLC workflow — ensuring nothing from the initial intent (requirements, stories, acceptance criteria) is silently dropped between stages.

What this adds

Stable ID conventions (enforced in stage prose)

  • FR{n}, NFR{n} — requirements-analysis
  • US{n}.{m} — user-stories
  • AC{story}.{seq} (3-segment) — user-stories
  • BR{group}.{seq} — functional-design
  • NFRx.y — nfr-requirements decomposition
  • U{n} / u{n}-{desc} — units-generation naming

Per-stage traceability.json

Each stage produces a structured JSON file mapping upstream elements to downstream coverage:

{
  "stage": "functional-design",
  "unit": "u1-auth",
  "upstream_ids": ["AC1.1.1", "AC1.1.2"],
  "coverage": [
    { "id": "AC1.1.1", "status": "OK", "target": "BR01" },
    { "id": "AC1.1.2", "status": "GAP" }
  ],
  "reverse": [
    { "id": "BR03", "status": "ORPHAN" }
  ]
}

Deterministic traceability sensor

aidlc-sensor-traceability.ts fires at each gate with three checks:

  1. Minimum coverage — empty file fails
  2. Status scan — GAP/ORPHAN entries are findings; Deferred/N/A without target are findings
  3. Upstream ID cross-check — extracts IDs from upstream files via regex, compares against declared upstream_ids. Two independent sources must agree.

Per-unit stages scope via unit-of-work-story-map.md to avoid false positives across units.

Phase boundaries

  • delivery-planning (Inception exit): aggregates upstream traceability files, checks for gaps
  • build-and-test (Construction exit): final coverage gate — verifies all FRs/ACs/NFRs from inception have implementing code across all units

Other changes

  • Moved Construction→Operation boundary from ci-pipeline (conditional) to build-and-test (always runs)
  • Fixed typo in infrastructure-design.md
  • Regenerated stale t66 designer-export fixture (agents 11→13, pre-existing)
  • Updated docs and knowledge base

Known limitations

  • Ideation→Inception boundary untouched — no numbered IDs at that phase (exploratory by design)
  • Unit-name join assumes u{n} convention — advisory warning emitted on naming mismatch
  • Free-text elements (component names) not diffable — marker scan only for units-generation
  • build-and-test final gate is prose-instructed — not sensor-enforced (cross-unit aggregation)

Pre-existing test failures (not introduced by this PR)

  • t65 round-trip: YAML emitter issue on v2 branch
  • t68 version badge: v2 branch version mismatch (0.7.12 vs 2.0.0 in README)

- Stable ID conventions: FR{n}, NFR{n}, US{n}.{m}, AC{story}.{seq}, BR{group}.{seq}, NFRx.y
- Each stage produces traceability.md mapping upstream elements to outputs
- New traceability sensor (aidlc-sensor-traceability.ts) fires at gate, checks for GAP/ORPHAN markers
- Phase boundaries: delivery-planning aggregates inception tables, build-and-test does final coverage gate
- Moved Construction→Operation boundary from ci-pipeline (conditional) to build-and-test (always runs)
- NFR numbering: inception NFRs decompose to NFRx.y in construction
- AC numbering: acceptance criteria numbered as AC{story}.{seq}
- BR numbering: business rules numbered as BR{group}.{seq}
- Fixed typo in infrastructure-design.md ("ok letstages")
- Updated docs and knowledge base to reflect new verification model

Known limitations:
- Sensor checks self-reported gaps only (marker grep), not completeness (ID diffing) — v2 sensor needed
- Ideation→Inception boundary unchanged (no numbered IDs at that phase)
- Empty traceability.md passes (no minimum-rows check yet)
…nd unit scoping

Replaces the fragile markdown-table sensor with a structured JSON approach:

- traceability.json (not .md/.yaml): parsed via JSON.parse, zero ambiguity
- Three-layer verification: minimum coverage + status scan + upstream ID cross-check
- Per-unit scoping via unit-of-work-story-map.md (U{n} prefix matching)
- Deferred/N/A without target treated as gaps
- Malformed JSON emits pass:false (not exit 1 swallowed by dispatcher)
- Removed traceability from produces: (sensor-only artifact, not consumed downstream)

ID conventions enforced in stage prose:
- FR{n}, NFR{n} at requirements-analysis
- US{n}.{m} at user-stories (+ AC{story}.{seq} 3-segment)
- BR{group}.{seq} at functional-design
- NFRx.y at nfr-requirements
- U{n} / u{n}-{desc} naming at units-generation

Upstream ID cross-check catches silent omissions:
- Extracts IDs from upstream files via regex
- Compares against declared upstream_ids in the JSON
- Two independent sources must agree (LLM declaration + file extraction)

Test coverage: 14 cases covering status detection, minimum coverage,
upstream diffing, per-unit scoping, malformed JSON, and error handling.

Known limitations:
- Unit-name join assumes u{n}-{desc} dir / U{n} column convention
- Free-text elements (component names) not diffable — marker scan only
- Empty expected set on naming mismatch: advisory warning, not failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant