Skip to content

Bug: Prevent QA Bulk Validation from succeeding with an empty HEAD cache #2696

Description

@filiperochalopes

User story

As a CIEL curator, I want QA Bulk Validation to fail when the HEAD cache refresh produces no concepts so that an empty validation run cannot be reported as successful.

Use case

QA Bulk Validation relies on the CIEL HEAD concept cache. When the cache is missing or stale, the existing workflow automatically refreshes it from the OCL export before scanning concepts.

A cache refresh that produces zero concepts must not be considered valid, and QA Bulk Validation must never report SUCCEEDED when no concepts were actually scanned.

Requirements

  • Validate the HEAD cache after an automatic refresh and ensure that it contains concepts before starting QA evaluation.
  • Treat a refreshed cache containing zero concepts as an error.
  • Prevent QA Bulk Validation from finishing as SUCCEEDED when zero concepts were scanned because the HEAD cache is empty.
  • Preserve the existing behavior where a missing or stale HEAD cache triggers an automatic refresh.
  • Return a clear failure message indicating that the HEAD cache refresh completed without usable concepts.
  • Add regression coverage for an apparently successful cache refresh that produces zero concepts.

Acceptance criteria

  • Given a missing or stale HEAD cache, when QA Bulk Validation starts, then the existing automatic cache refresh is attempted.
  • Given a cache refresh that successfully loads concepts, when QA continues, then those concepts are scanned normally.
  • Given a cache refresh that produces zero concepts, when QA attempts to continue, then the job fails instead of being marked SUCCEEDED.
  • A QA Bulk Validation job cannot report success with scanned_concepts=0 as a consequence of an empty HEAD cache.
  • The failure clearly identifies the empty HEAD cache/export condition.

More details (collapsible)

Evidence and investigation

Evidence from August 20, 2026:

A QA Bulk Validation execution started at approximately 00:56 and was persisted as SUCCEEDED with:

  • total_items=0
  • processed_items=0
  • scanned_concepts=0
  • total_failures=0

However, the same execution reported that the concept total had been resolved to 55,644 active concepts and that the Valkey HEAD cache had been repopulated from the OCL export.

No concept pages were subsequently scanned.

A later execution on August 20, 2026 at approximately 02:36 processed approximately 55,000 concepts normally.

Additional evidence from July 2, 2026:

Executions at approximately 11:50 and 12:02 also had processed_items=0, but these executions correctly failed because the HEAD cache was empty/stale and the required export was not yet available.

The August 20 case differs because the cache refresh itself returned success despite producing an empty cache.

Investigation indicates the current workflow already detects an unavailable or stale cache before refresh. The missing guard is after repopulation.

Relevant implementation points identified during investigation:

  • api/src/adapters/secondary/cache/head_concept_cache.py:326 detects an invalid/stale cache.
  • api/src/tasks/head_refresh.py:401 considers the refresh successful when population does not return None.
  • api/src/adapters/secondary/cache/head_concept_cache.py:779 allows total_count=0 to be persisted.
  • api/src/services/qa_bulk_service.py:2881 derives total_concepts from the cached payloads and currently accepts zero.
  • api/src/services/qa_bulk_service.py:2889 consequently processes no pages when the resulting collection is empty.

The failure mode is therefore not a QA run with zero findings. It is a QA run that scanned zero concepts after an apparently successful HEAD cache refresh.

The fix should include a post-refresh guard at the cache population layer and/or a defensive validation in QA Bulk Validation before scanning begins.

Metadata

Metadata

Labels

signal/high-riskSignificant potential impact if incorrectsignal/small-scopeLimited to a small part of the codebasesignal/well-specifiedClear requirements and acceptance criteriastage/triagedAI triage complete — scored and classifiedtype/bugSomething is broken

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions