feat: efficiency leaderboard, ranked by tokens per dollar - #118
Merged
Conversation
Lands the idea from #70 by kevinfosterNG, reworked so the ranking is true. That PR sorted client-side over the ~25 rows already fetched, which answers "who is most efficient among the top 25 by cost" rather than "who is most efficient" — and it forced hasMore to false, so the board silently offered fewer pages than it had. It also scored zero-cost submissions as Infinity, which would pin a $0 row at #1 permanently. Ranking now happens in the database on a stored tokens_per_dollar column, NULL for zero cost so division by zero has no answer instead of a wrong one, with a partial index matching the query. The spend floor is the part that makes this a real metric rather than a noise generator, and it came from looking at the data: unfiltered, the top of the efficiency board was a $0.01 submission at 7M tokens/$ against a median of 1.2M — the ratio of a rounding error. Above $100 the ranking spans $112 to $11K of real spend and reads as a genuine second axis, which matters on a board whose #1 by cost is a $205K outlier. Efficiency is disabled while a date filter is active rather than quietly falling back: the date-range board aggregates daily rows and has no stored ratio to order by, so offering the toggle there would show a cost ranking under an efficiency label. Selecting a date range while on efficiency drops back to cost so the lit toggle never disagrees with the rows. Verified against production through the anon key the browser actually uses: 1030 eligible submissions, correctly ordered, noise rows excluded. Co-Authored-By: kevinfosterNG <kevinfosterNG@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the idea from #70 by @kevinfosterNG, reworked so the ranking is actually true.
Why it needed reworking
That PR sorted client-side over the ~25 rows already fetched, which answers "who is most efficient among the top 25 by cost" — a different and much less interesting question. It also forced
hasMoretofalse(so the board silently offered fewer pages than it had) and scored zero-cost submissions asInfinity, which would pin a $0 row at #1 permanently.Ranking now happens in the database on a stored
tokens_per_dollarcolumn —NULLfor zero cost, so division by zero has no answer rather than a wrong one — with a partial index matching the query.The spend floor is what makes it a metric
This came from looking at the data, not from theory. Unfiltered, the top of the efficiency board was:
psource /Users/pawelkiszczak/…/activateAgainst a median of 1.2M tok/$. Those are rounding errors, not ways of working.
Above $100 the ranking spans $112 to$11K of real spend (4.1M → 2.6M tok/$ ) and reads as a genuine second axis — which matters on a board whose #1 by cost is a $205K outlier.
Date filters
Efficiency is disabled while a date filter is active rather than quietly falling back. The date-range board aggregates daily rows and has no stored ratio to order by, so offering the toggle there would show a cost ranking under an efficiency label. Selecting a date range while on efficiency drops back to cost, so the lit toggle never disagrees with the rows.
Separate finding
That $28.89 row is a username that's a shell fragment —
psource /Users/pawelkiszczak/Projects/…/activate. Someone'sgit config user.namepicked up corrupted text and it's now a public leaderboard entry. Filing separately; not fixed here.Verified against production through the anon key the browser actually uses: 1,030 eligible submissions, correctly ordered, noise excluded. Migration 011 applied.
npm test,tsc,eslint,next buildclean.🤖 Generated with Claude Code