Skip to content

[Router] 2381 semantic_similarity classifier (M5)#2482

Closed
SlawomirNowaczyk wants to merge 2 commits into
mainfrom
router/2381-semantic_similarity_classifier
Closed

[Router] 2381 semantic_similarity classifier (M5)#2482
SlawomirNowaczyk wants to merge 2 commits into
mainfrom
router/2381-semantic_similarity_classifier

Conversation

@SlawomirNowaczyk

Copy link
Copy Markdown
Collaborator

Implements the semantic_similarity classifier — the first model-backed classifier in the generic routing engine — against the existing Classifier interface. It scores an input by max cosine similarity against a set of reference_phrases, reported as a single score under the empty-string key and consumed via a classifier-band condition. Closes #2381.

Changes

  • SemanticSimilarityClassifier (routing_policy.cpp): embeds the input via ClassifierServices::embed, computes cosine against each reference exemplar, and returns max as Score::labels[""] (read by conditions through Score::primary()). Declares no labels.
  • Reference-phrase caching: exemplars are embedded once and cached on the instance. Since classifiers are shared across concurrent router requests (AGENTS.md invariant Lock transformers version for cpu #8), the lazy cache is guarded by a mutex; the input embedding is recomputed per request.
  • Registry wiring (make_classifier): semantic_similarity now constructs the real classifier instead of throwing "not implemented". Requires model + non-empty reference_phrases; rejects labels/default_label.
  • Failure handling: any embed failure (missing service, throw, empty/mismatched vectors) yields Score::ok = false so the owning band applies its on_error policy rather than letting an exception escape.
  • Tests: extended FakeClassifierServices with per-(model, text) embeddings and embed call counters; added test_routing_policy_semantic.cpp covering max-cosine selection, reference-phrase caching (embed-once), inclusive band boundaries incl. the default min_score: 0.5, and on_error; updated the registry test for the now-accepted type.

Notes

  • The cosine score is clamped into [0,1] to satisfy the band contract (guards against floating-point overshoot past 1.0 for near-identical vectors).
  • Out of scope per the issue: the real embed() ↔ Router wiring ([Router] ClassifierServices to Router wiring (M-svc) #2384) and the other classifier types (classifier/llm). No schema change — the existing route_policy.schema.json already describes reference_phrases.

Testing

ctest -R RoutingPolicy — all four suites (Contract, Evaluator, Registry, Semantic) pass.

@eddierichter-amd eddierichter-amd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Only feedback is can you add the RoutingPolicySemanticTest to the CI?

@SlawomirNowaczyk

Copy link
Copy Markdown
Collaborator Author

This looks great! Only feedback is can you add the RoutingPolicySemanticTest to the CI?

Thanks, I forgot about it. Should be done now :)

@ramkrishna2910

Copy link
Copy Markdown
Contributor

Thanks for this — it's clean and was approved, but we're consolidating #2381 on the unified version in #2483, which builds on this work and additionally collapses semantic_similarity onto the generic classifier label→score contract (no magic empty key) and supports multi-concept scoring. Since the schema is still unreleased, taking the breaking cleanup now is the cheap moment to do it.

No action needed here. I'll close this once #2483 merges (it carries your initial commit forward).

@SlawomirNowaczyk

Copy link
Copy Markdown
Collaborator Author

Made obsolete by #2483

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cpp enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Router] semantic_similarity classifier (M5)

3 participants