Skip to content

modified ChecklistValidator to use a call to the clb api#1439

Open
marcos-lg wants to merge 5 commits into
dev-validatorfrom
#1262_checklist_validation
Open

modified ChecklistValidator to use a call to the clb api#1439
marcos-lg wants to merge 5 commits into
dev-validatorfrom
#1262_checklist_validation

Conversation

@marcos-lg

@marcos-lg marcos-lg commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

When merged it can be tested with the rest of the changes in the validator in whaterver env is deployed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates validator-checklists from the embedded ChecklistBank normalizer/Neo4j-based validation to using the Checklistbank.org API, simplifying dependencies and shifting checklist metrics collection to remote validation responses.

Changes:

  • Replaced local ChecklistBank CLI/Neo4j normalization flow with Checklistbank.org API calls (/validator, /importer/{key}, /dataset/{key}/verbatim).
  • Updated CLI configuration and callback wiring to use CLB API URL/user/password instead of Neo4j/GBIF lookup settings.
  • Refactored and replaced unit test fixtures to mock CLB API responses (JSON) rather than building local normalized usage objects.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
gbif/validator/validator-checklists/src/test/resources/checklists/api_response_without_extensions.json Adds mocked importer API response fixture (no extensions).
gbif/validator/validator-checklists/src/test/resources/checklists/api_response_with_extensions.json Adds mocked importer API response fixture (with extensions).
gbif/validator/validator-checklists/src/test/resources/checklists/api_response_verbatim.json Adds mocked verbatim API response fixture for issue sampling/file-name inference.
gbif/validator/validator-checklists/src/test/resources/checklists/00000001-c6af-11e2-9b88-00145eb45e9a/taxa.txt Removes legacy local DWCA test resource used by the old normalizer-based approach.
gbif/validator/validator-checklists/src/test/java/org/gbif/pipelines/validator/checklists/ws/ChecklistbankWsManualTest.java Adds a disabled manual test for running against a real CLB API endpoint.
gbif/validator/validator-checklists/src/test/java/org/gbif/pipelines/validator/checklists/ws/ChecklistbankWsClientMock.java Adds a mock CLB WS client used by unit tests.
gbif/validator/validator-checklists/src/test/java/org/gbif/pipelines/validator/checklists/model/TestData.java Removes old object-based test data builder used by the term-frequency collector tests.
gbif/validator/validator-checklists/src/test/java/org/gbif/pipelines/validator/checklists/collector/TermFrequencyCollectorTest.java Removes tests for the removed local term-frequency collector flow.
gbif/validator/validator-checklists/src/test/java/org/gbif/pipelines/validator/checklists/cli/ChecklistValidatorCallbackIT.java Partially updates integration test setup (currently still disabled) after config/model changes.
gbif/validator/validator-checklists/src/test/java/org/gbif/pipelines/validator/checklists/ChecklistValidatorTest.java Reworks tests to validate output from CLB API-based evaluation using mocked JSON responses.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/ws/IdLookupWsClient.java Removes the old species lookup WS client interface.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/ws/IdLookupClient.java Removes the old IdLookup wrapper client and its custom client builder logic.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/ws/ChecklistbankWsClient.java Introduces a Feign/Spring WS client interface for the Checklistbank.org validator API.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/model/ObjectToTermMapper.java Removes object→term mapping utilities used only by the old local-normalizer path.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/model/NormalizedTermMapUsageData.java Removes intermediate normalized term-map model used by old collectors.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/model/NormalizedNameUsageData.java Removes normalized usage holder used by old local-normalizer path.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/collector/ValidationDataCollector.java Removes old issue/term aggregation collector tied to NameUsage/VerbatimUsage objects.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/collector/TermFrequencyCollector.java Removes term-frequency aggregation logic tied to the local-normalizer approach.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/cli/config/ChecklistValidatorConfiguration.java Replaces Neo4j/GBIF lookup config with CLB API URL/user/password fields.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/cli/ChecklistValidatorCallback.java Wires the callback to instantiate the new CLB API-based ChecklistValidator.
gbif/validator/validator-checklists/src/main/java/org/gbif/pipelines/validator/checklists/ChecklistValidator.java Replaces local normalizer execution with CLB API submission/polling and response mapping to Metrics.FileInfo.
gbif/validator/validator-checklists/pom.xml Removes ChecklistBank CLI/Neo4j/Lucene dependencies now obsolete with API-based validation.
gbif/validator/pom.xml Drops properties related to removed ChecklistBank/Neo4j/Lucene dependency versions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants