[dsh-benchmark] Protocol v1 CPU — maintainer seed / Windows i7-1255U

[initial-d]

Commit SHA

960de1953f8589467456ad92bf9e4f978c3f0f1b

DeepSeek Harness version or source

@deepseek-ai/dsh 0.1.0-rc.7, launched with pnpm dlx.

Model route used for the headless agent run:

• model: deepseek/deepseek-v4-flash
• endpoint: private OpenAI-compatible endpoint
• local patch: deepseek-official provider with a private OpenAI-compatible base URL

No API key is included in this report.

Agent prompt

Read AGENTS.md, docs/deepseek_harness_recipe.md, docs/benchmarking.md, and docs/reality_check.md. Run this exact protocol v1 CPU benchmark using PowerShell: $env:PYTHONPATH='src'; & 'C:\Users\duyimin1\AppData\Local\Python\bin\python.exe' scripts\benchmark_tensor_factors.py --device cpu --n-dates 750 --n-stocks 1000 --window 20 --repeat 10 --warmup 3 --threads 1 --interop-threads 1 --seed 42 --json-out artifacts\benchmark-v1-dsh-headless.json . Then return an issue-ready DeepSeek Harness benchmark report summary with commit SHA, dsh version 0.1.0-rc.7, model deepseek/deepseek-v4-flash via private OpenAI-compatible endpoint, exact command, environment table, result table, JSON path, caveats, and the statement that this is an engineering benchmark only, not trading alpha.

Note: the prompt accidentally included a trailing . after the command. DeepSeek Harness attempted the supplied command, observed the click error for the unexpected extra argument, then reran the identical protocol command with the stray trailing . removed.

Command

$env:PYTHONPATH='src'; & 'C:\Users\duyimin1\AppData\Local\Python\bin\python.exe' scripts\benchmark_tensor_factors.py --device cpu --n-dates 750 --n-stocks 1000 --window 20 --repeat 10 --warmup 3 --threads 1 --interop-threads 1 --seed 42 --json-out artifacts\benchmark-v1-dsh-headless.json

Environment

Field	Value
Protocol	v1
Python	3.14.4
Platform	Windows-11-10.0.26200-SP0
CPU	Intel64 Family 6 Model 154 Stepping 4, GenuineIntel
Logical CPUs	12
PyTorch	2.11.0+cpu
PyTorch threads	1
PyTorch interop threads	1
CUDA available	False
CUDA device	-
Synthetic panel	750 dates x 1000 stocks
Window	20
Warmup / repeat	3 / 10
Seed	42

Result table

Device	Case	Mean	Std	Peak CUDA memory
cpu	cs_rank(close)	209.5 ms	33.4 ms	-
cpu	ts_mean(close,20)	37.6 ms	6.1 ms	-
cpu	ts_rank(close,20)	177.3 ms	61.0 ms	-
cpu	ts_corr(close,returns,20)	295.8 ms	151.5 ms	-
cpu	ewma(close,0.05)	51.7 ms	9.0 ms	-
cpu	compute_legacy_set(6 factors)	756.9 ms	131.5 ms	-

JSON report

{
  "environment": {
    "cpu": "Intel64 Family 6 Model 154 Stepping 4, GenuineIntel",
    "cuda_available": false,
    "cuda_device": null,
    "logical_cpus": 12,
    "n_dates": 750,
    "n_stocks": 1000,
    "platform": "Windows-11-10.0.26200-SP0",
    "protocol": "v1",
    "python": "3.14.4",
    "pytorch": "2.11.0+cpu",
    "pytorch_interop_threads": 1,
    "pytorch_threads": 1,
    "repeat": 10,
    "seed": 42,
    "warmup": 3,
    "window": 20
  },
  "results": [
    {
      "case": "cs_rank(close)",
      "device": "cpu",
      "mean_seconds": 0.20951630999916232,
      "peak_cuda_memory": "-",
      "std_seconds": 0.0333845963657337
    },
    {
      "case": "ts_mean(close,20)",
      "device": "cpu",
      "mean_seconds": 0.03761682999902405,
      "peak_cuda_memory": "-",
      "std_seconds": 0.006076283240144468
    },
    {
      "case": "ts_rank(close,20)",
      "device": "cpu",
      "mean_seconds": 0.1772975300147664,
      "peak_cuda_memory": "-",
      "std_seconds": 0.061030517096814854
    },
    {
      "case": "ts_corr(close,returns,20)",
      "device": "cpu",
      "mean_seconds": 0.29584634000784715,
      "peak_cuda_memory": "-",
      "std_seconds": 0.15146333213705085
    },
    {
      "case": "ewma(close,0.05)",
      "device": "cpu",
      "mean_seconds": 0.05165958000579849,
      "peak_cuda_memory": "-",
      "std_seconds": 0.00897780335181658
    },
    {
      "case": "compute_legacy_set(6 factors)",
      "device": "cpu",
      "mean_seconds": 0.7569349399884231,
      "peak_cuda_memory": "-",
      "std_seconds": 0.13146516003125394
    }
  ],
  "schema_version": 1
}

Agent transcript or summary

DeepSeek Harness read the project agent instructions and benchmark/reality-check docs, attempted the exact supplied command, detected the trailing-dot click usage error, reran the same protocol v1 command without the stray argument, verified exit code 0, checked that artifacts/benchmark-v1-dsh-headless.json existed and contained schema_version: 1, then returned this issue-ready summary.

Caveats

• Maintainer seed report, not an independent third-party machine report.
• Synthetic benchmark only; no market data was used.
• This is an engineering throughput benchmark, not trading alpha, investment evidence, or live-performance evidence.
• CPU-only PyTorch build; CUDA availability was False.
• High variance on ts_corr(close,returns,20) (151.5 ms std, about 51% of mean) and ts_rank(close,20) (61.0 ms std, about 34% of mean). Treat those means as unstable on this host.
• The run used a low-power laptop CPU and one PyTorch intra-op / inter-op thread, so results are machine-specific and should not be interpreted as a controlled hardware ranking.
• The local shell wrapper emitted repeated CreateFile() Error: 5 lines on stderr in this environment; the Python benchmark completed and wrote valid JSON with exit code 0.

[initial-d]

Follow-up: the DSH-facing benchmark path now has a short announcement and a released helper plugin.

• Announcement: DeepSeek Harness benchmark path: run, validate, report #62
• Plugin release: https://github.com/initial-d/dsh-plugin-mlquant-benchmark/releases/tag/v0.1.0
• Install: dsh plugin --profile web add github:initial-d/dsh-plugin-mlquant-benchmark

The plugin is only a reproducibility helper: it runs the fixed protocol v1 CPU benchmark, validates the JSON artifact, reads the result table, and drafts an issue-ready report. It does not add a trading agent, market-data access, or automatic GitHub posting.

[initial-d]

Follow-up: this DSH seed report is now included in the Community Benchmark Board under the protocol v1 results. It is listed as an agent-assisted reproducibility run, with the high-variance caveats kept attached, rather than as an independent hardware ranking.\n\nBoard: https://github.com/initial-d/ml-quant-trading/blob/main/docs/benchmark_board.md