Skip to content

Add pandas asv example - #11

Merged
ev-br merged 2 commits into
benchx-dev:mainfrom
MarcoGorelli:pandas-example
Nov 18, 2026
Merged

ev-br merged 2 commits into
benchx-dev:mainfrom
MarcoGorelli:pandas-example

Conversation

@MarcoGorelli

Copy link
Copy Markdown
Collaborator

No description provided.

@MarcoGorelli
MarcoGorelli marked this pull request as ready for review November 6, 2026 11:12

@ev-br ev-br left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!
Several minor quibbles below:

Tried it locally with a released benchx and uv, and it seems to hard require an editable benchx install?

$ uv venv
$ source .venv/bin/activate
$ uv pip install benchx
$ ./run.sh 
run.sh: cannot find a benchx checkout to install in the venvs.
  Install benchx in editable mode in the active environment (pip install -e <checkout>/prototype),
  or set BENCHX_SRC to the directory with benchx's pyproject.toml.

Uninstalled the released benchx, and followed the suggestion: an editable install of benchx main branch plus BENCHX_SRC env variable.

Output:

== 4. compare: the sides differ in the pandas the collector read
run pandas-1794483557  profile environments  baseline pandas-2.3.3  contender pandas-3.1.1
  differs: coordinates.environment.system.deps.pandas, coordinates.environment.system.deps.sha256
  caveats: baseline: setup-per-call-unbatched, contender: setup-per-call-unbatched

IMPROVED (1)
  strings.Methods.time_lower [dtype='str'] wall-time  effect  -86.96%  noise   0.13%  rounds=3
  • Do I read it correctly that the benchmark improved by 86% in pandas 3.1.1 relative to pandas 2.3.3?
  • differs: coordinates.environment.system.deps.sha256 --- is this the benchx way to express that the two pandas versions have different dependencies?

One other minor quibble, probably unrelated to this example: the output I see on this particular local machine is

== 5. the series: one per environment, each point at the suite's revision
adf31f791760  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
093c55ee7a86  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
430404914063  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
4e589de29f24  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
e5bd8352d8b3  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
1e69eb0e3acb  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
49c8f078a2f2  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
044d99446e1e  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
c169e68fef35  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
b166b9363e8a  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
71f7bf5bd6dd  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
3e0c43614cdb  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
abf0001dea0f  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
f99ad2a290b3  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
a6b5cc9e1658  local/qgpu3  bench_matmul  wall-time [s]  est=benchx/median/v1
b740429975e1  local/qgpu3  bench_matmul  load-average [1]  est=benchx/median/v1
1043c2b39898  pandas  strings.Methods.time_lower  wall-time [s]  est=benchx/median/v1
606e237c68ed  pandas  strings.Methods.time_lower  load-average [1]  est=benchx/median/v1
c24f12a13bb7  pandas  strings.Methods.time_lower  wall-time [s]  est=benchx/median/v1
6e3d35b1375f  pandas  strings.Methods.time_lower  load-average [1]  est=benchx/median/v1

So it looks like some unrelated results, local/qgpu3 bench_matmul, which I probably run at some point earlier and have forgotten about, contaminate the local store.
Maybe the example should use a local store?

Comment thread examples/pandas-asv/README.md Outdated
```console
$ git clone https://github.com/benchx-dev/benchx.git
$ python3 -m venv .venv && . .venv/bin/activate # Python >= 3.11, or any environment you like
$ pip install -e benchx/prototype

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can pip install -e benchx or install it from PyPI now

Comment thread examples/pandas-asv/README.md Outdated
```

The script installs that same checkout in both venvs. It finds it from the benchx in the active
environment; `BENCHX_SRC=<checkout>/prototype ./run.sh` names it instead.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more prototype path

@MarcoGorelli

Copy link
Copy Markdown
Collaborator Author

thanks for your review!

Do I read it correctly that the benchmark improved by 86% in pandas 3.1.1 relative to pandas 2.3.3?

yup that's right!

differs: coordinates.environment.system.deps.sha256 --- is this the benchx way to express that the two pandas versions have different dependencies?

As in, that the pandas are of different versions? If so, yes, 2.3.3 vs 3.1.1

@ev-br ev-br left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MarcoGorelli

I reran the updated version locally with benchx from PyPI, and it indeed works without setting BENCHX_SRC. I'm still not very clear if that variable is very useful, but that's separate and does not need to block adding a great example.

@ev-br
ev-br merged commit c8f8270 into benchx-dev:main Nov 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants