fix: add observability consistently across NFR stages#404
Conversation
0249166 to
e089615
Compare
e089615 to
6ee6795
Compare
Add observability as a first-class NFR category across all stages that enumerate NFR dimensions, fixing the inconsistency where only nfr-requirements.md listed it while requirements-analysis.md, nfr-design.md, and infrastructure-design.md omitted it. Changes: - requirements-analysis.md: add observability to Step 6 and Step 10 - nfr-requirements.md: add observability-requirements artifact (produces, outputs, Step 6), update condition description - nfr-design.md: add observability-design artifact (produces, consumes, outputs, Step 5, Step 6), add observability-requirements to consumes - infrastructure-design.md: add observability-design to consumes list - Regenerate dist/ for all harnesses - Update golden fixture and t66 element count assertion Closes awslabs#398
6ee6795 to
4b0ff30
Compare
apackeer
left a comment
There was a problem hiding this comment.
Thanks for the fix; the shape is right and the artifact graph is internally consistent. Every new produces has a consumer, and every new required consumes has a producer in every scope that runs the consumer stage. Requesting changes for rebase and same-commit hygiene the current v2 requires.
Blockers
-
Rebase required. Base is 2.1.3 (c95070e); v2 is now 2.2.0 (b67798c) with six intervening commits (2.1.4 - 2.2.0).
git merge-tree origin/v2 refs/pull/404/headreports a hard conflict ontests/integration/t66.test.ts; the stage .md files and dist trees auto-merge textually but everydist/*/tools/data/stage-graph.jsonandtests/fixtures/designer-export/export.jsonneeds a real regeneration on the rebased tree. -
Version trio missing.
core/tools/aidlc-version.tsstill reads 2.1.3, and there is no matching heading inCHANGELOG.mdor update to the README badge.AGENTS.mdChangelog Policy requires all three in the same commit andtests/unit/t68-version-changelog-sync.test.tsenforces it. Versions 2.2.1 - 2.2.13 are already claimed by open PRs, so 2.2.14 is the next free slot (re-bump if a sibling merges first). -
t66agent count stale. The PR setsartifacts=123, agents=13intests/integration/t66.test.ts; v2 hasartifacts=122, agents=14after the composer agent landed in 2.2.0. On rebase the target isstages=32, scopes=9, artifacts=123, agents=14, with bothexpect(...).toBe(...)lines matching. Your artifact math (net +1: two added, one removed) is correct. -
Golden fixture must be re-exported after rebase.
tests/fixtures/designer-export/export.jsonwas regenerated at 2.1.3; the byte-exact assertion int66needs a fresh export on the rebased tree so it includes the composer agent plus the observability additions. -
Docs still name
monitoring-design.md. Six references the PR removes fromcore/but leaves in the shipped docs:docs/reference/03-orchestrator.md:565docs/reference/04-stages/construction.md:450,:515,:538,:550,:1044
AGENTS.mdDocumentation Policy asks for these to update in the same commit. While there, add rows forobservability-requirements.mdandobservability-design.mdwherever the docs enumerate produced artifacts by name.
-
infrastructure-design.mdcontent preservation. Step 5 drops the "Monitoring and Observability" bullet (four areas becomes three) and Step 6 drops themonitoring-design.mdbullet. The short "observability tooling" phrase added into Infrastructure Services does not fully preserve the operational-visibility detail (metrics platform, log aggregation, tracing backend, alerting) that lived in the removed bullets. Please either expand the Infrastructure Services phrasing or referenceobservability-design.mdexplicitly, so the reader still sees where that content lives.
Non-blocking
-
The PR summary claims
(122 -> 124 artifacts, 11 -> 13 agents). The actual diff is122 -> 123, andagentsis untouched by the diff. Worth fixing in the PR body so reviewers do not reconcile silently. -
core/knowledge/aidlc-quality-agent/nfr-reliability-guide.mdis titled "NFR Reliability and Observability Guide", which pre-dates promoting observability to a peer NFR category. Optional pass to reshape the knowledge companions so observability reads like a peer rather than a sub-topic of reliability. Not a merge blocker. -
Cross-PR interaction with #402 (same author, also touches
nfr-design.md+infrastructure-design.md). Whichever merges second will re-apply on top of the first. Heads-up only. -
Post-rebase, run
t66and confirm the advisory-edge baseline still holds. Adding required consumes changes the dangling calculation for scopes that skip the producer stage; a diff there is a design question, not a mechanical pin update.
Design confirmation (for the record)
The observability chain closes cleanly in scopes that run every stage from nfr-requirements through infrastructure-design. nfr-requirements produces observability-requirements; nfr-design consumes it and produces observability-design; infrastructure-design and observability-setup consume observability-design. Scopes that skip nfr-design do not consume observability-design either, and the dangling-consume machinery from 2.1.7 covers the case where nfr-requirements produces observability-requirements without a downstream consumer in scope.
Ping me post-rebase and I will re-review.
Summary
Fixes #398 — NFR categories were inconsistent across stages:
nfr-requirements.mdlisted 5 categories (Performance, Security, Scalability, Reliability, Observability) whilerequirements-analysis.md,nfr-design.md, andinfrastructure-design.mdonly listed 4 (missing Observability).Changes
Core stage files (
core/aidlc-common/stages/)inception/requirements-analysis.md: Added "observability" to the NFR category list in Step 6 (completeness analysis) and Step 10 (generate requirements)construction/nfr-requirements.md: Addedobservability-requirementsartifact toproduces:,outputs:, and Step 6; updatedcondition:to list all 5 NFR categoriesconstruction/nfr-design.md: Addedobservability-designartifact toproduces:,outputs:, Step 5, and Step 6; addedobservability-requirementstoconsumes:; updated upstream-coverage sensor listconstruction/infrastructure-design.md: Addedobservability-designtoconsumes:list and upstream-coverage sensor descriptionGenerated dist/
bun scripts/package.ts(pass--check)Tests
tests/fixtures/designer-export/export.jsongolden fixturetests/integration/t66.test.tselement count assertion (122→124 artifacts, 11→13 agents) and test labelArtifact flow (after fix)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.