fix(search): gate exact-match rank on trust and order tiers by adoption#3058
Conversation
An exact name match with no strong trust signal (official flag, provenance/rebuild verification) and no measurable adoption now ranks with the lexical tier, and a log-scale identity-deduped adoption bucket orders results before raw text score within each tier. Shared seam in convex/lib/searchRanking.ts covers package and skill catalog search. Closes #3054
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7d2ed33466
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return ( | ||
| a.rankTier - b.rankTier || | ||
| b.score - a.score || | ||
| compareRankedSearchKeys(rankedSearchKey(a, signals(a)), rankedSearchKey(b, signals(b))) || |
There was a problem hiding this comment.
The new trust key only affects this local sort; the entries returned to the HTTP layer still carry the original rankTier/score, and the API fan-in resorts combined results with compareCatalogSearchEntries in convex/httpApiV1/packagesV1.ts using the old rankTier-then-score order. For /api/v1/plugins/search across code+bundle plugins, or /api/v1/packages/search with skills included, a zero-adoption exact-name result from one source can be promoted back above an adopted lexical result from another source, so the public API still exposes the squat vector this change is meant to close. Return an adjusted sort key/rank metadata or make the HTTP combiner use the shared trust-aware ordering.
Useful? React with 👍 / 👎.
| const adoption = adoptionBucket(signals); | ||
| const demoteExact = match.rankTier === 0 && adoption === 0 && !hasStrongTrustSignal(signals); | ||
| return { | ||
| tier: demoteExact ? 1 : match.rankTier, |
There was a problem hiding this comment.
Demoting an untrusted exact match only works after all plausible candidates have been gathered, but both search collectors still decide whether to scan more from matches.length < targetCount before this adjusted key is applied. With limit=1, an exact slug/name hit from direct recall fills the page and prevents the fallback scan from ever seeing an adopted substring/topic/summary candidate, so the exact-name squat can remain the top and only result despite this demotion. Overfetch to the scan budget before sorting, or otherwise continue recall whenever collected entries may be demoted.
Useful? React with 👍 / 👎.
…s are collected A demoted exact-name hit filled the collection quota before the fallback scan ran, so top-1 queries returned the squat unchallenged. Demoted exact matches no longer count toward the quota in package or skill catalog search; regression tests cover the limit-1 scenario on both surfaces.
1 participant
Summary
convex/lib/searchRanking.ts) and wired it into both package catalog search (convex/packages.ts) and skill-as-package catalog search (convex/skills.ts). Two behavior changes: (1) an exact-name match with no strong trust signal (not official, no provenance/rebuild verification) and no measurable adoption is ranked with the lexical tier instead of the exact tier; (2) within a tier, a log-scale adoption bucket (identity-deduped downloads + installs) orders results before the raw text-match score. Existing tie-breakers (official, verification tier, stars, installs, downloads, recency) are unchanged.specs/search-relevance.mdis amended with the new "Exact-Match Squat Gate" contract.Linked Issue
Screenshots
N/A(backend ranking change; no UI surface altered)Behavioural Proof
convex/packages.public.test.ts,convex/skills.packageCatalog.test.ts).convex/lib/searchRanking.test.ts(15 tests), including the observed scenario: a zero-adoption, unverified exact-name match ("squat") now ranks below an adopted substring match, while official/verified/adopted exact matches keep today's top placement.convex/packages.public.test.ts,convex/skills.packageCatalog.test.ts,convex/httpApiV1.handlers.test.ts,src/lib/packageApi.test.ts(707 tests total in those files).Security / Trust Impact
This is abuse hardening for the search trust boundary: it removes the cheap top-slot path for name-squatting on generic queries and prices ranking influence in identity-deduped adoption or curated/provenance trust. Intended trade-off (documented in the spec): a brand-new package that exactly matches a query no longer outranks adopted related packages until it earns adoption or verification; it remains discoverable at the top of the unproven band by text relevance.
Data / Deploy Impact
Verification
bun run ci:staticbunx vitest run convex/lib/searchRanking.test.ts convex/packages.public.test.ts convex/skills.packageCatalog.test.ts convex/httpApiV1.handlers.test.ts src/lib/packageApi.test.tsbun run ci:unit— the three isolated-pass/full-suite-timeout failures on this host (home hero, scan results, user badge; all unrelated React render tests) reproduce on a cleanmainbaseline run as well (main shows 19 local failures), so they are a local-environment artifact; hosted CI is the arbiterbun run ci:types-buildN/A