Skip to content

add mobench support to ProveKit main#450

Open
dcbuild3r wants to merge 2 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures
Open

add mobench support to ProveKit main#450
dcbuild3r wants to merge 2 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures

Conversation

@dcbuild3r
Copy link
Copy Markdown
Collaborator

Reopened from #429 using the new dcbuilder/* branch naming.

Old PR: #429
Replacement branch: dcbuilder/pr429-main-mobench-fixtures
Old head branch: dcbuild3r/main-mobench-fixtures
Head SHA: e14fa60beadc25bf1e7d238c9bdbf8f6237fa9f2

Previous PR CI / benchmark runs from #429:

Summary

  • add main-based mobench fixtures for passport, OPRF, and P-256 bigcurve
  • run BrowserStack triad profile across Vivo Y21, Pixel 7, S24 and iOS fallback triad
  • fix Android prove timing so fixture clone/setup is outside the measured prove span
  • use jemalloc as the Android fallback allocator under the native C FFI allocator
  • harden Android incomplete-fixture reporting with timeout/build/kill-evidence fields

Android mobench fix note

The previous failing run was https://github.com/worldfnd/provekit/actions/runs/26007468825. The missing Vivo Y21 monolithic cell did not recover a BrowserStack session payload or summary.json; available artifacts only show the BrowserStack fetch timeout after 7200s for build c943753d95fd0b34f5775aa0a3bc6ff58cbcc3ca.

Before/failure memory from that run:

  • Vivo Y21 OPRF: 328 MB process peak, 10.061s
  • Vivo Y21 fragmented age check: 1187 MB process peak, 65.342s
  • Vivo Y21 monolithic age check: no row; BrowserStack timeout/no summary
  • S24 OPRF: 426 MB process peak, 2.809s

What changed:

  • Android and iOS now use the same semantic boundary for prove benches: per-iteration fixture setup happens before the measured closure, and profile_phase("prove") wraps only the prover entry point. Previously prepared.clone() ran inside profile_phase("prove"), so clone cost and peak memory were charged to proving.
  • Android native C FFI fallback allocation now routes through jemalloc instead of Bionic malloc when no host callback/mmap allocator is active.
  • If BrowserStack returns no summary, failure.json now records attempts, fetch timeout seconds, build id, and any LMK/OOM/SIGKILL lines recovered from attempt/device logs.

Latest benchmark numbers were reposted as a PR comment from #429.

Validation

  • cargo fmt --all
  • cargo test -p bench-mobile --lib
  • cargo test -p bench-mobile --test examples_smoke
  • cargo test -p bench-mobile --test passport_smoke
  • cargo check -p provekit-ffi --target aarch64-linux-android with NDK 26.1 aarch64-linux-android34-clang
  • ruby -e 'require "yaml"; YAML.load_file(".github/workflows/mobile-bench-reusable.yml")'
  • git diff --check

Copy link
Copy Markdown
Collaborator Author

Latest mobile benchmark numbers

Reposted from #429 comment #429 (comment).

Source: Mobile Bench PR Auto run 26368378239 on dcbuild3r/main-mobench-fixtures at e14fa60b.
Mobench version: 0.1.42. Artifact summaries report 2 measured runs per function/device plus 1 warmup run.

iOS

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF iPhone 16 Pro Max 2 1 766.622ms 766.622ms 775.181ms 758.064ms 775.181ms 1.533s 3.364s 6.728s 438.8% 10.95 MB 10.95 MB 362.28 MB
OPRF iPhone 14 2 1 1.096s 1.096s 1.118s 1.075s 1.118s 2.192s 4.497s 8.994s 410.2% 32.55 MB 32.55 MB 206.95 MB
OPRF iPhone SE 2022 2 1 1.085s 1.085s 1.086s 1.085s 1.086s 2.171s 4.486s 8.972s 413.3% 33.48 MB 33.48 MB 209.44 MB
Passport fragmented iPhone 16 Pro Max 2 1 4.751s 4.751s 4.805s 4.696s 4.805s 9.501s 21.524s 43.049s 453.1% 150.95 MB 150.95 MB 1042.44 MB
Passport fragmented iPhone 14 2 1 6.598s 6.598s 6.625s 6.570s 6.625s 13.195s 28.438s 56.877s 431.0% 264.30 MB 264.30 MB 845.34 MB
Passport fragmented iPhone SE 2022 2 1 7.053s 7.053s 7.233s 6.874s 7.233s 14.106s 30.661s 61.323s 434.7% 265.72 MB 265.72 MB 851.73 MB
Passport complete iPhone 16 Pro Max 2 1 4.499s 4.499s 4.514s 4.484s 4.514s 8.997s 19.933s 39.866s 443.1% 376.08 MB 376.08 MB 1594.42 MB
Passport complete iPhone 14 2 1 7.772s 7.772s 8.889s 6.654s 8.889s 15.543s 28.034s 56.069s 360.7% 487.56 MB 487.56 MB 1220.12 MB
Passport complete iPhone SE 2022 2 1 7.443s 7.443s 7.505s 7.381s 7.505s 14.886s 30.253s 60.507s 406.5% 494.86 MB 494.86 MB 1104.30 MB

Android

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF Samsung Galaxy S24 2 1 2.725s 2.725s 2.762s 2.688s 2.762s 5.449s 9.657s 19.315s 354.4% 89.95 MB 89.95 MB 488.93 MB
OPRF Google Pixel 7 2 1 3.900s 3.900s 3.943s 3.858s 3.943s 7.800s 11.518s 23.036s 295.3% 18.23 MB 18.23 MB 237.57 MB
OPRF Samsung Galaxy M32 2 1 6.681s 6.681s 6.926s 6.436s 6.926s 13.362s 22.641s 45.282s 338.9% n/a n/a 352.76 MB
Passport fragmented Samsung Galaxy S24 2 1 14.516s 14.516s 14.697s 14.336s 14.697s 29.033s 54.556s 109.112s 375.8% 202.75 MB 202.75 MB 929.28 MB
Passport fragmented Google Pixel 7 2 1 26.237s 26.237s 28.870s 23.605s 28.870s 52.475s 74.610s 149.220s 284.4% 217.21 MB 217.21 MB 789.64 MB
Passport fragmented Samsung Galaxy M32 2 1 40.629s 40.629s 41.183s 40.075s 41.183s 81.257s 137.225s 274.451s 337.8% 284.89 MB 284.89 MB 859.60 MB
Passport complete Samsung Galaxy S24 2 1 11.250s 11.250s 11.671s 10.829s 11.671s 22.500s 46.655s 93.310s 414.7% 376.77 MB 376.77 MB 1157.10 MB
Passport complete Google Pixel 7 2 1 19.856s 19.856s 22.382s 17.330s 22.382s 39.712s 63.704s 127.408s 320.8% 304.69 MB 304.69 MB 980.12 MB
Passport complete Samsung Galaxy M32 2 1 28.921s 28.921s 30.002s 27.840s 30.002s 57.842s 102.707s 205.414s 355.1% 365.05 MB 365.05 MB 1220.69 MB

Memory growth is the benchmark-reported peak growth above baseline; process peak is the reported process RSS peak.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 3, 2026

CSP benchmarks

Metric Value
Workflow status [PASS] success
Commit 66c476dcea8c
Run #26893108970
Circuits benchmarked 21
Iterations averaged per circuit 3

Prover time, peak RSS, peak heap, and verifier time are arithmetic means across the iterations. Peak heap comes from the largest peak memory entry in provekit-cli prove's tracing output; peak RSS is reported by /usr/bin/time -v (max-resident-set-size).

Each metric cell shows the current value followed by the percentage delta against the latest successful main run #26627835985. (new) marks circuits absent from the baseline.

Results
Circuit Constraints Witnesses Prover time Peak RSS Peak heap Verifier time Proof size PKP size
ecdsa_p256 143,282 (±0.0%) 258,158 (±0.0%) 2.98 s (±0.0%) 257 MB (±0.0%) 225 MB (-0.1%) 340 ms (±0.0%) 2.76 MB (-1.0%) 810 KB (±0.0%)
keccak_1024 822,870 (±0.0%) 1,543,366 (±0.0%) 6.34 s (-0.7%) 985 MB (±0.0%) 953 MB (±0.0%) 843 ms (-0.8%) 3.13 MB (-0.4%) 6.07 MB (±0.0%)
keccak_128 163,058 (±0.0%) 313,707 (±0.0%) 2.12 s (+0.2%) 274 MB (+0.1%) 242 MB (-0.1%) 363 ms (-1.8%) 2.81 MB (+0.4%) 1.22 MB (±0.0%)
keccak_2048 1,575,606 (±0.0%) 2,945,822 (±0.0%) 11.78 s (-0.6%) 1.81 GB (±0.0%) 1.80 GB (±0.0%) 1.51 s (+5.1%) 3.28 MB (-0.5%) 12.36 MB (±0.0%)
keccak_256 256,206 (±0.0%) 487,012 (±0.0%) 2.35 s (+1.4%) 328 MB (-0.1%) 290 MB (-0.2%) 420 ms (+2.4%) 2.86 MB (-0.1%) 1.97 MB (±0.0%)
keccak_512 445,094 (±0.0%) 839,130 (±0.0%) 3.68 s (+1.4%) 594 MB (±0.0%) 509 MB (-0.2%) 560 ms (+2.4%) 2.99 MB (+0.5%) 3.40 MB (±0.0%)
poseidon2_12 479 (±0.0%) 563 (±0.0%) 353 ms (+0.9%) 23.90 MB (+0.2%) 14.69 MB (±0.0%) 100 ms (±0.0%) 1.01 MB (-4.1%) 436 KB (±0.0%)
poseidon2_16 556 (±0.0%) 719 (±0.0%) 360 ms (+0.9%) 24.22 MB (+0.2%) 14.88 MB (±0.0%) 100 ms (±0.0%) 1.01 MB (-2.6%) 530 KB (±0.0%)
poseidon2_2 231 (±0.0%) 278 (±0.0%) 347 ms (-1.9%) 22.96 MB (-0.4%) 14.11 MB (±0.0%) 103 ms (+3.3%) 1.05 MB (+2.6%) 108 KB (±0.0%)
poseidon2_4 529 (±0.0%) 535 (±0.0%) 350 ms (+1.0%) 23.32 MB (-0.2%) 14.31 MB (±0.0%) 100 ms (±0.0%) 1.05 MB (+0.5%) 31.67 KB (±0.0%)
poseidon2_8 363 (±0.0%) 423 (±0.0%) 353 ms (+0.9%) 24.18 MB (+0.1%) 14.50 MB (±0.0%) 103 ms (+3.3%) 1.03 MB (+1.8%) 365 KB (±0.0%)
poseidon_12 504 (±0.0%) 524 (±0.0%) 363 ms (+2.8%) 24.19 MB (±0.0%) 14.69 MB (±0.0%) 103 ms (+3.3%) 1.06 MB (+2.7%) 410 KB (±0.0%)
poseidon_16 609 (±0.0%) 633 (±0.0%) 363 ms (+1.9%) 24.21 MB (-0.2%) 14.97 MB (±0.0%) 103 ms (+3.3%) 1.05 MB (+1.0%) 536 KB (±0.0%)
poseidon_2 240 (±0.0%) 249 (±0.0%) 343 ms (+1.0%) 22.95 MB (+0.4%) 14.02 MB (±0.0%) 100 ms (±0.0%) 1.02 MB (-3.6%) 53.79 KB (±0.0%)
poseidon_4 297 (±0.0%) 309 (±0.0%) 350 ms (±0.0%) 23.35 MB (-0.4%) 14.31 MB (±0.0%) 100 ms (±0.0%) 1.04 MB (-0.5%) 210 KB (±0.0%)
poseidon_8 402 (±0.0%) 418 (±0.0%) 353 ms (±0.0%) 23.50 MB (+0.9%) 14.50 MB (±0.0%) 100 ms (±0.0%) 1.03 MB (±0.0%) 305 KB (±0.0%)
sha256_1024 196,940 (±0.0%) 339,764 (±0.0%) 2.25 s (+1.5%) 305 MB (-2.4%) 274 MB (±0.0%) 420 ms (±0.0%) 2.82 MB (+1.3%) 1.89 MB (±0.0%)
sha256_128 46,398 (±0.0%) 80,974 (±0.0%) 1.09 s (±0.0%) 99.70 MB (-0.6%) 83.41 MB (-0.4%) 260 ms (±0.0%) 2.51 MB (+0.6%) 511 KB (+1.7%)
sha256_2048 345,399 (±0.0%) 612,724 (±0.0%) 3.55 s (-1.3%) 548 MB (-0.6%) 483 MB (-0.1%) 587 ms (-2.2%) 2.95 MB (-2.1%) 2.99 MB (-1.4%)
sha256_256 67,904 (±0.0%) 117,944 (±0.0%) 1.38 s (-0.7%) 151 MB (+0.4%) 130 MB (+0.2%) 290 ms (±0.0%) 2.63 MB (-0.6%) 709 KB (-0.3%)
sha256_512 110,916 (±0.0%) 191,884 (±0.0%) 1.50 s (-1.3%) 179 MB (-2.1%) 158 MB (±0.0%) 320 ms (±0.0%) 2.69 MB (±0.0%) 1.09 MB (-0.6%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant