[benchmark] Protocol v1 CPU — Apple M4 / macOS 26.5.2 / PyTorch 2.13.0

[sergio12S]

Commit SHA

2a91c6b199d3f3608bbebacb36addb37949200b9

Command

python scripts/benchmark_tensor_factors.py \
  --device cpu --n-dates 750 --n-stocks 1000 --window 20 \
  --repeat 10 --warmup 3 --threads 1 --interop-threads 1 --seed 42 \
  --json-out artifacts/benchmark-v1.json

The canonical protocol v1 line from docs/benchmarking.md rather than make benchmark, so every flag is visible.

Environment

Field	Value
Protocol	v1
Python	3.10.16
Platform	macOS-26.5.2-arm64-arm-64bit
CPU	Apple M4 (the script reports arm)
Logical CPUs	10
PyTorch	2.13.0
PyTorch threads	1
PyTorch interop threads	1
CUDA available	False
CUDA device	-
Synthetic panel	750 dates x 1000 stocks
Window	20
Warmup / repeat	3 / 10
Seed	42

Fresh shallow clone and a fresh virtual environment from pip install -e '.[dev]'. No other deliberate compute-heavy workload was running.

Result table

Device	Case	Mean	Std	Peak CUDA memory
cpu	cs_rank(close)	24.4 ms	218.0 us	-
cpu	ts_mean(close,20)	7.1 ms	499.6 us	-
cpu	ts_rank(close,20)	15.9 ms	489.3 us	-
cpu	ts_corr(close,returns,20)	36.9 ms	4.7 ms	-
cpu	ewma(close,0.05)	3.6 ms	158.5 us	-
cpu	compute_legacy_set(6 factors)	136.3 ms	6.6 ms	-

JSON report

Attachments are not available through the CLI, so the report is pasted as the template allows.

{
  "environment": {
    "cpu": "arm",
    "cuda_available": false,
    "cuda_device": null,
    "logical_cpus": 10,
    "n_dates": 750,
    "n_stocks": 1000,
    "platform": "macOS-26.5.2-arm64-arm-64bit",
    "protocol": "v1",
    "python": "3.10.16",
    "pytorch": "2.13.0",
    "pytorch_interop_threads": 1,
    "pytorch_threads": 1,
    "repeat": 10,
    "seed": 42,
    "warmup": 3,
    "window": 20
  },
  "results": [
    {
      "case": "cs_rank(close)",
      "device": "cpu",
      "mean_seconds": 0.02436975009914022,
      "peak_cuda_memory": "-",
      "std_seconds": 0.00021803943878439828
    },
    {
      "case": "ts_mean(close,20)",
      "device": "cpu",
      "mean_seconds": 0.007112212300853571,
      "peak_cuda_memory": "-",
      "std_seconds": 0.0004996006231893428
    },
    {
      "case": "ts_rank(close,20)",
      "device": "cpu",
      "mean_seconds": 0.01594553349932539,
      "peak_cuda_memory": "-",
      "std_seconds": 0.000489293930147845
    },
    {
      "case": "ts_corr(close,returns,20)",
      "device": "cpu",
      "mean_seconds": 0.03694394149788423,
      "peak_cuda_memory": "-",
      "std_seconds": 0.004703237368120022
    },
    {
      "case": "ewma(close,0.05)",
      "device": "cpu",
      "mean_seconds": 0.0036402627003553787,
      "peak_cuda_memory": "-",
      "std_seconds": 0.0001584816944583692
    },
    {
      "case": "compute_legacy_set(6 factors)",
      "device": "cpu",
      "mean_seconds": 0.13632488329967601,
      "peak_cuda_memory": "-",
      "std_seconds": 0.006604538647831815
    }
  ],
  "schema_version": 1
}

Notes

ts_corr does not settle on this machine. A second identical run of the same command gave 43.4 ms with a standard deviation of 12.1 ms, against 36.9 ms and 4.7 ms in the first. Every other case reproduced within 5% across the two runs:

Case	Run 1	Run 2	Spread
cs_rank(close)	24.4 ms	24.5 ms	0.4%
ts_mean(close,20)	7.1 ms	6.8 ms	4.8%
ts_rank(close,20)	15.9 ms	15.5 ms	2.6%
ts_corr(close,returns,20)	36.9 ms	43.4 ms	17.4%
ewma(close,0.05)	3.6 ms	4.0 ms	9.4%
compute_legacy_set(6 factors)	136.3 ms	133.2 ms	2.3%

If other submissions show the same spread on ts_corr specifically, the 10-repetition budget may be too small for that case.

A factor of two on ts_rank, with a confound. The same physical machine ran this protocol on 2026-08-17 at 94d39d3 under Python 3.9.6 and PyTorch 2.8.0, where ts_rank(close,20) measured 30.6 ms against 15.9 ms now. This is not offered as a PyTorch speedup: the benchmark script itself changed between those commits, so harness and runtime moved together and the comparison cannot be attributed. It is recorded only because a factor of two on one case is worth someone with a clean A/B checking on purpose.

This is a single machine snapshot, not a controlled hardware ranking.

[initial-d]

Thanks, @sergio12S — this report is now formally included in the repository's Community Benchmark Board under Comparable Protocol v1 Results.

The board preserves the full environment, commit, exact protocol, and raw result caveats. In particular, the second-run ts_corr variance and the confounded historical ts_rank comparison remain attached to the report. We are treating this as a reproducibility and hardware-context snapshot, not a controlled CPU ranking or trading-performance claim.