docs: fix drift in Hybrid Search - #321
Draft
jack-arturo wants to merge 1 commit into
Draft
Conversation
- final_score is a raw weighted sum, not normalized to [0.0, 1.0] (3 places) - SEARCH_WEIGHT_RELEVANCE weights relevance_score; context_bonus is unweighted - recency decays from timestamp, never last_accessed; window/curve configurable - missing importance/confidence score 0.0, not 0.5/0.7 - repin source-note permalinks to automem@42ba8b6 Verified against automem@42ba8b61b7d0b24ecaeb7feb4ceef59f09fc7cd0 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014iwvkKvNhquqnG4MQ6EEun
Deploying automem-website with
|
| Latest commit: |
7237c1a
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://2abbea2f.automem-website.pages.dev |
| Branch Preview URL: | https://docs-audit-core-concepts-hyb-t6y2.automem-website.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated docs-accuracy audit of
core-concepts/hybrid-search. First review of this page — no prior review state existed.The "sum to 1.95, then normalize to [0.0, 1.0]" claim was already proven false on
core-concepts/recall-tuning(PR #290). It survives on this page in three more places;_compute_metadata_score()returns the raw weighted sum with no normalization step anywhere in the function.All ten weight defaults, all six
RECALL_*limits, thelimitdefault of 5, and all nine source-note file paths were checked and are correct — the drift is concentrated in how the score is combined and where recency comes from._compute_metadata_score()buildsfinalas a plain weighted sum and returns it — no normalization, clamping, or division follows. With saturated components it exceeds 1.0context_bonus × SEARCH_WEIGHT_RELEVANCE (default: 0.0); weights table labels the row "Context — Context profile scoring bonus"SEARCH_WEIGHT_RELEVANCEmultipliesrelevance_score(consolidation decay relevance).context_bonusis added unweighted and is unaffected by that variable — so the table implied the context bonus is disabled by default when it is notlast_accessedfalls back totimestamp"_compute_recency_score(memory.get("timestamp"))— the stringlast_accesseddoes not appear anywhere inscoring.py. Access time never influences rankingtimestamponly; documentedSEARCH_RECENCY_WINDOW_DAYS(180) andSEARCH_RECENCY_CURVE(linear/exp) as the real controlsimportancedefaults to 0.5"; "Missingconfidencedefaults to 0.7"0.0when absent or non-numeric. (0.5 / 0.7 are write-time defaults onPOST /memory, not scoring fallbacks — a memory lacking the field scores zero for that component)0720da242ba8b6; every path verified to still existVerified against: automem@42ba8b61b7d0b24ecaeb7feb4ceef59f09fc7cd0
Questions
componentsdict returns eleven entries (vector,keyword,metadata,relation,tag,importance,confidence,recency,exact,relevance,context). But "10-Component" is used as a site-wide framing — it also appears inreference/api/recall-operationsandarchitecture/overview. Renaming it here alone would make this page inconsistent with two pages outside this audit's scope, so the heading was left as-is. Worth deciding globally.SEARCH_WEIGHT_TEMPORAL(default0.1) exists inconfig.pybut is absent from this page. It only applies when therecency_biasre-rank runs, which is a separate path from_compute_metadata_score()— so its omission may well be deliberate scoping rather than drift. Not touched.Unverified
automemand are sourced from its verification trail, not from this checkout. Left alone.embeddingsaves 200-500ms — no measurement in source. Left alone.Follow-ups
Beyond the 5-fix cap:
RECALL_RELEVANCE_GATEis undocumented on this page. When set above 0, results whose topical evidence falls below the gate getimportance,confidence,recency,tag, andrelevancescaled down linearly byevidence / gatebefore weighting. This materially changes scoring inside tag-scoped pools and deserves its own subsection.SEARCH_TAG_SCORE_TOKEN_CAPcaps the tag-score denominator so long queries aren't penalized — also undocumented.evidencesignal (max(vector, keyword, metadata, exact), deliberately excluding tag overlap as scope-confounded) is a notable design decision with an explanatory comment in source, and has no counterpart in the docs.Generated by Claude Code