Repository navigation
Add pandas asv example - #11
Conversation
ev-br
left a comment
There was a problem hiding this comment.
This is great!
Several minor quibbles below:
Tried it locally with a released benchx and uv, and it seems to hard require an editable benchx install?
$ uv venv
$ source .venv/bin/activate
$ uv pip install benchx
$ ./run.sh
run.sh: cannot find a benchx checkout to install in the venvs.
Install benchx in editable mode in the active environment (pip install -e <checkout>/prototype),
or set BENCHX_SRC to the directory with benchx's pyproject.toml.
Uninstalled the released benchx, and followed the suggestion: an editable install of benchx main branch plus BENCHX_SRC env variable.
Output:
== 4. compare: the sides differ in the pandas the collector read
run pandas-1794483557 profile environments baseline pandas-2.3.3 contender pandas-3.1.1
differs: coordinates.environment.system.deps.pandas, coordinates.environment.system.deps.sha256
caveats: baseline: setup-per-call-unbatched, contender: setup-per-call-unbatched
IMPROVED (1)
strings.Methods.time_lower [dtype='str'] wall-time effect -86.96% noise 0.13% rounds=3
- Do I read it correctly that the benchmark improved by 86% in pandas 3.1.1 relative to pandas 2.3.3?
differs: coordinates.environment.system.deps.sha256--- is this the benchx way to express that the two pandas versions have different dependencies?
One other minor quibble, probably unrelated to this example: the output I see on this particular local machine is
== 5. the series: one per environment, each point at the suite's revision
adf31f791760 local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
093c55ee7a86 local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
430404914063 local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
4e589de29f24 local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
e5bd8352d8b3 local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
1e69eb0e3acb local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
49c8f078a2f2 local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
044d99446e1e local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
c169e68fef35 local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
b166b9363e8a local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
71f7bf5bd6dd local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
3e0c43614cdb local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
abf0001dea0f local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
f99ad2a290b3 local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
a6b5cc9e1658 local/qgpu3 bench_matmul wall-time [s] est=benchx/median/v1
b740429975e1 local/qgpu3 bench_matmul load-average [1] est=benchx/median/v1
1043c2b39898 pandas strings.Methods.time_lower wall-time [s] est=benchx/median/v1
606e237c68ed pandas strings.Methods.time_lower load-average [1] est=benchx/median/v1
c24f12a13bb7 pandas strings.Methods.time_lower wall-time [s] est=benchx/median/v1
6e3d35b1375f pandas strings.Methods.time_lower load-average [1] est=benchx/median/v1
So it looks like some unrelated results, local/qgpu3 bench_matmul, which I probably run at some point earlier and have forgotten about, contaminate the local store.
Maybe the example should use a local store?
| ```console | ||
| $ git clone https://github.com/benchx-dev/benchx.git | ||
| $ python3 -m venv .venv && . .venv/bin/activate # Python >= 3.11, or any environment you like | ||
| $ pip install -e benchx/prototype |
There was a problem hiding this comment.
can pip install -e benchx or install it from PyPI now
| ``` | ||
|
|
||
| The script installs that same checkout in both venvs. It finds it from the benchx in the active | ||
| environment; `BENCHX_SRC=<checkout>/prototype ./run.sh` names it instead. |
|
thanks for your review!
yup that's right!
As in, that the pandas are of different versions? If so, yes, 2.3.3 vs 3.1.1 |
ev-br
left a comment
There was a problem hiding this comment.
Thanks @MarcoGorelli
I reran the updated version locally with benchx from PyPI, and it indeed works without setting BENCHX_SRC. I'm still not very clear if that variable is very useful, but that's separate and does not need to block adding a great example.
No description provided.