Repository navigation
Conversation
Move pyarrow>=19 into the optional benchx[parquet] extra, replace jsonschema[format-nongpl] with rfc3339-validator and rfc3986-validator, remove jsonpointer, and decouple JSON-only execution and comparison paths from Parquet storage. Closes benchx-dev#116
- Catch FormatCheckerError in CLI main() and return EXIT_REFUSED (with test in test_refusals_not_crashes.py). - Add --no-ingest to bx compare measure mode using MemoryStore (works for revisions, environments, and variants without pyarrow), validate flags before opening the store, and point missing-pyarrow hints to --no-ingest. - Remove duplicated pyarrow ModuleNotFoundError wrapper in store.py (delegates to parquet.py). - Replace module-level pyarrow skips with lazy Store imports and per-test requires_pyarrow marks across test files (+194 more tests run in the minimal no-pyarrow job). - Narrow the minimal CI import check to pyarrow/jsonpointer and verify benchx's default vs parquet Requires-Dist directly on the built wheel.
| if isinstance(suites, list) and suites: | ||
| first = suites[0] | ||
| if isinstance(first, dict) and first.get("suite"): | ||
| return str(first["suite"]).rsplit("/", 1)[-1] |
There was a problem hiding this comment.
Suite basename collision for nested Python suites: stripping everything before the last / matches Google Benchmark binary paths (release/arrow-bit-util-benchmark), but two nested Python suites sharing a filename (e.g. benchmarks/a/bench.py and benchmarks/b/bench.py) merge into one "bench.py" suite. Worth deciding whether to keep relative paths for non-binary suites (or strip only a leading build-dir prefix) in a follow-up.
| """ | ||
| grouped: dict[tuple, list[dict]] = defaultdict(list) | ||
| for pt in points: | ||
| key = (pt["sha"], pt["node"], pt["suite"], pt["case"], pt["params"], pt["quantity"]) |
There was a problem hiding this comment.
Cross-environment stores collide on the same commit: dashboard series are keyed by (suite, case, params, quantity, node) rather than series_fingerprint. If a store holds bx compare --profile environments --label env=a,b (or two different subject.configuration / protocol variants) on the same commit and host, _collapse_duplicate_points will pool both sides into one point. Worth noting in docs/benchx.md that bx view currently assumes per-commit revision history, or incorporating labels/configuration into the key in a follow-up.
| ingest_problems = sum(r["ingest_problems"] for r in runs) | ||
| raw_points: list[dict] = [] | ||
| for path in sorted(runs_path.glob("*/*/results.parquet")): | ||
| raw_points.extend(_read_parquet_points(path, distinguish_dirty=False)) |
There was a problem hiding this comment.
Scaling note for multi-year runs/ trees: read_runs_dir loads every */*/results.parquet into Python dicts on each build. Fine for local stores and months of CI runs, but once a project accumulates hundreds of thousands of rows we should benchmark build time and consider incremental/monthly rollups.
…e filenames, and polish UI labels
Moves the static benchmark dashboard and derived-table pipeline from
benchx-arrowintobenchxasbx view(benchx.views), so users can generate deployable static sites or view local benchmark results in a browser directly frombenchx.Summary of changes
src/benchx/views/):chdbwith pure Python stdlib for deriving flat tables (commits,points,changes,latest,movers) and static JSON view payloads (landing,series,sparks,history/<suite>,commits/<sha>).--store,--runs) or writing the optional Parquet marts underdata/.store.parquet/$BENCHX_HOME, or--store PATH), collapsing multi-round attempts per commit, distinguishing dirty working trees (<sha>-dirty-<tree>), and enriching commit metadata viagit cat-file --batch(--repo).--results PATH...) viaMemoryStorewith zero PyArrow dependency.runs/<sha>/<node>/{run.parquet, results.parquet}layout (--runs DIR), pluswrite_run()/run_schema()for CI publishers.src/benchx/views/static/(keeps CDN uPlot with graceful offline table fallback; vendoring uPlot tracked in Vendor uPlot assets in wheel for offlinebx view#129).bx view):bx view(no--out): builds into a temporary directory and serves over local HTTP until Ctrl-C.bx view --out DIR [--serve] [--open]: builds a deployable static site with cache-busted asset URLs (?v=).tests/test_views.py(15 tests covering marts arithmetic, view payloads, Arrow-free JSON--results, local store round collapsing,--runswith logs and failed builds, and HTTP serving).tests/test_minimal_deps.pyto verifyimport benchx.viewsdoes not importpyarrow.bx viewindocs/benchx.md.Verification
pytest tests: 844 passed, 6 skipped.bx view --runsagainstbenchx-arrow's existing chDB-builtsite/data/acrossmanifest.json,landing.json,series.json,sparks.json,history/*.json, andcommits/*.json.node --checkonsrc/benchx/views/static/app.jsand confirmeduv build --wheelincludesbenchx/views/static/*andbenchx/tests/test_views.py.