Major refactoring by DanieleDeSensi · Pull Request #7 · HLC-Lab/blink

DanieleDeSensi · 2026-06-09T17:29:55Z

No description provided.

Bug fixes: - Fix duplicate Irecv/Send pair in o2o_bsnbr (was posting each op twice) - Fix MPI_STATUS_IGNORE -> MPI_STATUSES_IGNORE in all Waitall calls - Fix checker.c log filename (was using locale timestamp, now checker_YYYYMMDD_HHMMSS.log) - Fix spurious 'receiver rank' in alltoall/allreduce print strings - Fix null_dummy.c to properly call MPI_Init/Finalize - Add MPI_Init/Finalize to null_dummy New benchmarks: - allgather_b/nb (MPI_Allgather / MPI_Iallgather) - gather_b/nb (MPI_Gather / MPI_Igather) - allreduce_b/nb (MPI_Allreduce / MPI_Iallreduce, renamed from ardc) - broadcast_b/nb (MPI_Bcast / MPI_Ibcast, renamed from bdc) - reduce_b/nb (MPI_Reduce / MPI_Ireduce) - scatter_b/nb (MPI_Scatter / MPI_Iscatter) - reduce_scatter_b/nb (MPI_Reduce_scatter / MPI_Ireduce_scatter) - barrier_nb (MPI_Ibarrier) - pairwise_b (MPI_Sendrecv one-to-one, renamed from o2o) - stencil_2d_nb: 2D Cartesian halo exchange via MPI_Cart_create - kpartners_nb: k-regular random traffic (-k flag); k=1 ~ pairwise, k=N-1 ~ alltoall Removed: - bursty_noise_a2a.cpp and bursty_noise_incast.cpp (redundant: -endl -blength -bpause flags on standard benchmarks cover the same use case) Restructure: - All sources moved under src/ with one subdirectory per traffic pattern - Files renamed to full descriptive names (agtr->allgather, ardc->allreduce, bdc->broadcast, inc->incast, o2o->pairwise, a2a->alltoall, barrier->barrier_b) - common.h moved to src/ Build system: - Replace Makefile with CMake (minimum 3.16) - src/CMakeLists.txt uses GLOB_RECURSE CONFIGURE_DEPENDS: adding a new .c/.cpp file requires zero CMake edits - Add .gitignore (build/) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Workflow (.github/workflows/ci.yml) builds every benchmark and the test suite on Ubuntu with OpenMPI and runs ctest on each push and pull request; MPI oversubscribe is enabled so the 8-rank suites run on smaller runners. Adds a BLINK_COVERAGE CMake option (-O0 --coverage -fprofile-update=atomic so concurrent MPI ranks do not corrupt shared .gcda files); CI runs gcovr over src/, prints it to the job summary, uploads a coverage.xml artifact, and best-effort uploads to Codecov (continue-on-error, so a missing token never fails CI). codecov.yml marks coverage informational. README: add CI and Codecov status badges plus a Continuous integration section documenting the flow and the one-time Codecov token setup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Implemented entirely in common.h, so every benchmark gets it for free. New flags: -plot (additive, printed after the CSV/table), -plotstat (avg|min|max|median|mainrank, default max), -plotbins, -plotbinsize (accepts a unit e.g. 2ms/500us, aligns bin edges to multiples of the width, caps at 64 bins folding the tail), and -plotlog (geometric bins). Includes a parse_duration() helper, startup validation, and a percentile (min/mean/p50/p90/p99/max) footer. Also: README flags + Output-format section, and test_misc smoke tests (render, stat/log/fixed selectors, invalid-stat and binsize+log conflict guards). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Extract print_histogram_row() into common.h (range label + scaled bar + empirical %, with an optional theoretical-% overlay and inf-edge handling). print_runtime_histogram and dist_test's print_histogram now both delegate row rendering to it; dist_test keeps its mean-relative bins and CDF overlay (its purpose) but no longer duplicates the bar-drawing/formatting, and the dead HIST_BAR_MAX is removed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

LICENSE: MIT (2024-2026). codeql.yml: GitHub-native C/C++ static analysis on pushes/PRs to main/dev and weekly; manual CMake+OpenMPI build with tests off so it scans the benchmark sources. README: add License and CodeQL badges (CodeQL pinned to dev). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

malloc_align() aborts internally on failure and never returns NULL, so the subsequent if(buf==NULL){fprintf;MPI_Abort;} blocks in the benchmarks were dead code. Removed 35 such blocks; common.h-s genuine plain-malloc check (all_data/sorting_buf/plot_vals) is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Tests: drive -bldist/-bpdist/-bpause (covers the samplers, rand_duration, dsleep and sample_* from the benchmark side, which merges correctly) plus checker/null_dummy smoke tests. gcovr: exclude unreachable/throw branches and MPI_Abort error-exit lines (uncoverable - the process dies before .gcda is flushed). .gitignore: ignore coverage artifacts. Raises src/ coverage to ~83% lines / 86% functions / 60% branches; 8/8 suites pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Move the definitions out of common.h (a header of static-inline functions instantiated in ~40 TUs) into common.c, compiled once into a blink_common static library and linked into every benchmark. common.h is now declarations only, wrapped in extern "C" so the C++ comm_only benchmarks link to the C library. Cleaner, faster to build, and -- the motivation -- gcov now counts the shared code from a single .gcda instead of per-TU copies it cannot merge: coverage of src/ rises to ~88% lines / 100% functions / 67% branches with no metric gaming. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add a burstdist case to each suite (collectives via the 2 parameterised fixtures, plus incast/pairwise/ring/pingpong/kpartners/stencil) running -bldist/-bpause/-bpdist, so the if(burst_length_rand) / if(burst_pause!=0) / if(burst_pause_rand) branches are exercised in all ~35 benchmarks (previously only barrier_nb). Lifts src/ coverage to ~92% lines / 100% functions / 72% branches; 8/8 suites pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-advanced-security · 2026-06-09T17:29:58Z

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

codecov · 2026-06-09T17:33:54Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment

Thanks for integrating Codecov - We've got you covered ☂️

Copilot

Pull request overview

Refactors the project into a CMake-based build with a shared common.c runtime library, adds structured -debug output/self-checks to MPI benchmarks, and introduces black-box GTest suites that launch the benchmarks under mpirun and validate correctness by parsing DEBUG ... lines.

Changes:

Introduce CMake build + CI/CodeQL workflows, remove legacy Makefile/top-level sources, and standardize output to build/bin.
Add shared runtime API (src/common.h) and update benchmarks to use common argument parsing, shutdown handling, safer buffering for -grty > 1, and structured debug self-check output.
Add GTest-based black-box correctness tests (tests/test_*.cpp) plus shared helpers for spawning and parsing debug output.

Reviewed changes

Copilot reviewed 68 out of 70 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/test_stencil.cpp	Adds black-box correctness tests for `stencil_2d_nb` (coverage, nprocs, integrity, neighbor symmetry, periodicity).
tests/test_pingpong.cpp	Adds black-box correctness tests for `pingpong_b` and `pingpong_pairwise_b`.
tests/test_kpartners.cpp	Adds black-box correctness tests for `kpartners_nb` including `k` validation.
tests/popen_helpers.h	Adds shared mpirun spawning + DEBUG-line parsing and common assertions for tests.
tests/CMakeLists.txt	Adds `add_benchmark_test()` helper to build/register GTest runners and set ctest `PROCESSORS`.
src/tools/null_dummy.c	Adds minimal MPI init/finalize tool.
src/tools/checker.c	Updates tool to use shared runtime, adds per-iteration timestamp logging.
src/stencil/stencil_2d_nb.c	Refactors to shared runtime + adds cartesian topology + debug integrity output for stencil exchanges.
src/scatter/scatter_nb.c	Refactors to shared runtime, fixes buffering for concurrent `MPI_Iscatter` under `-grty > 1`, adds debug output.
src/scatter/scatter_b.c	Refactors to shared runtime and adds debug output.
src/reduce/reduce_nb.c	Refactors to shared runtime, fixes buffering for concurrent `MPI_Ireduce`, adds debug output.
src/reduce/reduce_b.c	Refactors to shared runtime and adds debug output.
src/reduce_scatter/reduce_scatter_nb.c	Refactors to shared runtime, fixes buffering for concurrent `MPI_Ireduce_scatter`, adds debug output.
src/reduce_scatter/reduce_scatter_b.c	Refactors to shared runtime and adds debug output.
src/pingpong/pingpong_pairwise_b.c	Refactors to shared runtime, replaces old signal handling, adds deterministic debug check output.
src/pingpong/pingpong_b.c	Adds/refactors benchmark to shared runtime and emits structured debug self-check output.
src/pairwise/pairwise_b.c	Adds/refactors benchmark to shared runtime and adds a deterministic post-loop debug exchange to validate partner identity.
src/incast/incast_put.c	Refactors to shared runtime, fixes types/offsets and adds debug verification of RMA payload placement.
src/incast/incast_nb.c	Refactors to shared runtime, fixes buffering for concurrent nonblocking receives under `-grty > 1`, adds debug verification path.
src/incast/incast_get.c	Refactors to shared runtime and adds debug verification for Get-based incast.
src/incast/incast_bsnbr.c	Refactors to shared runtime and adds deterministic debug verification path.
src/incast/incast_b.c	Refactors to shared runtime and adds deterministic debug verification path.
src/gather/gather_nb.c	Adds/refactors benchmark to shared runtime, fixes buffering for concurrent `MPI_Igather`, adds debug output.
src/gather/gather_b.c	Adds/refactors benchmark to shared runtime and adds debug output.
src/common.h	Introduces shared runtime declarations (globals + helpers) implemented in `common.c`.
src/CMakeLists.txt	Auto-discovers sources, builds one executable per source, and links against a shared `blink_common` library.
src/broadcast/broadcast_nb.c	Refactors to shared runtime, fixes buffer typing, adds debug self-check output.
src/broadcast/broadcast_b.c	Refactors to shared runtime and adds debug self-check output.
src/barrier/barrier_nb.c	Adds nonblocking barrier benchmark with shared runtime + debug output.
src/barrier/barrier_b.c	Refactors to shared runtime and adds debug output.
src/alltoall/alltoall_nb.c	Refactors to shared runtime, fixes buffering for concurrent `MPI_Ialltoall` under `-grty > 1`, adds debug output.
src/alltoall/alltoall_man.c	Refactors to shared runtime, fixes buffering for concurrent point-to-point all-to-all, adds debug output.
src/alltoall/alltoall_comm_only.cpp	Adds/refactors comm-only all-to-all benchmark to shared runtime + debug output.
src/alltoall/alltoall_b.c	Adds/refactors blocking all-to-all benchmark to shared runtime + debug output.
src/allreduce/allreduce_nb.c	Adds/refactors nonblocking allreduce benchmark to shared runtime, fixes buffering for concurrent ops, adds debug output.
src/allreduce/allreduce_b.c	Adds/refactors blocking allreduce benchmark to shared runtime + debug output.
src/allgather/allgather_nb.c	Adds/refactors nonblocking allgather benchmark to shared runtime, fixes buffering for concurrent ops, adds debug output.
src/allgather/allgather_comm_only.cpp	Refactors comm-only allgather to shared runtime and aligns semantics with other allgather benchmarks, adds debug output.
src/allgather/allgather_b.c	Adds/refactors blocking allgather benchmark to shared runtime + debug output.
ping-pong_b.c	Removes legacy top-level pingpong source (superseded by `src/pingpong/pingpong_b.c`).
null_dummy.c	Removes legacy top-level dummy source (superseded by `src/tools/null_dummy.c`).
Makefile	Removes legacy Makefile build in favor of CMake.
LICENSE	Adds MIT license file.
common.h	Removes legacy top-level header implementation (replaced by `src/common.h` + `src/common.c`).
codecov.yml	Adds Codecov configuration (informational coverage status).
CMakeLists.txt	Adds top-level CMake configuration (MPI, tests, coverage option, FetchContent for GTest).
checker.c	Removes legacy top-level checker source (moved to `src/tools/checker.c`).
bursty_noise_incast.cpp	Removes legacy noise generator source.
bursty_noise_a2a.cpp	Removes legacy noise generator source.
ardc_nb.c	Removes legacy top-level benchmark source.
ardc_b.c	Removes legacy top-level benchmark source.
a2a_comm_only.cpp	Removes legacy top-level comm-only source (superseded by `src/alltoall/alltoall_comm_only.cpp`).
a2a_b.c	Removes legacy top-level benchmark source.
.gitignore	Adds ignore rules for build and coverage artifacts.
.github/workflows/codeql.yml	Adds CodeQL workflow for C/C++ with manual CMake build.
.github/workflows/ci.yml	Adds CI workflow: build, run tests, and best-effort gcovr + Codecov upload.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+#include <gtest/gtest.h>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <map>


+/* Assert that every rank that emits nprocs= reports the expected value. */
+inline void check_nprocs(const DebugMap& m, int expected, const std::string& ctx)
+{
+    for (const auto& [r, fields] : m) {
+        auto it = fields.find("nprocs");
+        if (it == fields.end()) continue;
+        EXPECT_EQ(std::stoi(it->second), expected)
+            << ctx << ": rank " << r << " nprocs=" << it->second
+            << " expected " << expected;
+    }
+}


+    set_target_properties(${name} PROPERTIES
+        RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
+    add_test(NAME ${name} COMMAND ${name})
+    set_tests_properties(${name} PROPERTIES PROCESSORS ${nprocs})


+    if (debug_mode) {
+        int _d, _b, _ok = 1;
+        const int _dirs[4] = {north, south, west, east};
+        for (_d = 0; _d < 4 && _ok; _d++) {
+            if (_dirs[_d] < 0) continue; /* MPI_PROC_NULL — no neighbour, no data */
+            for (_b = 0; _b < msg_size && _ok; _b++)
+                if (recv_buf[(size_t)_d * msg_size + _b] != (unsigned char)_dirs[_d]) _ok = 0;
+        }


parse_common_args now recognises -h, --help, and -help: prints common options on rank 0, then any benchmark-specific options from a weak benchmark_help symbol, then MPI_Finalize+exit(0). Benchmarks with custom flags (pairwise_b/_nb/_bsnbr, kpartners_nb, ring_nb/_bsnbr, stencil_2d_nb) override the weak NULL with a one-line help string at file scope; the other 31 benchmarks need no change and just inherit the common block. tests/test_misc.cpp: 4 new Help.* tests cover the common-only output, benchmark-specific section, all 3 flag forms, and rank-0-only printing. README.md: -h/--help row added to the common-flags table. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Previously parsed with atof(), which silently truncated unit suffixes: "-blength 1ms" became 1.0 (= 1 SECOND) instead of 1 millisecond. Anyone who tried to use a suffix got a result 1000-1000000x off the intent without any warning. parse_common_args now routes both flags through a new parse_duration_arg() wrapper that uses parse_duration() (same parser as -plotbinsize, already suffix-aware) and aborts with a clear diagnostic on NAN or negative input. Bare numbers continue to mean seconds, so existing scripts are unaffected. Help text and README now document the available suffixes; README example switched to "10ms" / "50ms" for readability. tests/test_misc.cpp: - DurationSuffixesAccepted exercises the suffix path. - InvalidDurationAbortsCleanly regression-guards the new validation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

DanieleDeSensi and others added 22 commits May 22, 2026 19:06

[IMP] Refactoring, pretty print and README.

cfce9ff

[ADD] More distributions for random bursts.

4f2bac5

Update README.md

883440c

[ADD] Logo.

0d0cc34

Merge branch 'dev' of github.com:DanieleDeSensi/blink into dev

2ee941a

[ADD] Logo.

e7d734e

[ADD] Logo.ush

397e83c

[ADD] Fixes and refactoring.

2b29e0a

[ADD] New tests.

ad45df4

docs: pin CI and coverage badges to the dev branch

c37f18c

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

[ADD] Coverage.

54be384

docs: point badges at the canonical hlc-lab/blink repo

e065eb9

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

[FIX] Minor

e76e505

DanieleDeSensi requested a review from Copilot June 9, 2026 17:30

Copilot started reviewing on behalf of DanieleDeSensi June 9, 2026 17:30 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

DanieleDeSensi added 3 commits June 10, 2026 10:12

[FIX] Compilation.

e3b3927

[FIX] Compilation.

55bddc8

[FIX] Compilation.

daa9c63

DanieleDeSensi and others added 2 commits June 10, 2026 10:45

DanieleDeSensi merged commit 37486c6 into main Jun 10, 2026
6 checks passed

DanieleDeSensi deleted the dev branch June 10, 2026 09:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Major refactoring#7

Major refactoring#7
DanieleDeSensi merged 27 commits into
mainfrom
dev

DanieleDeSensi commented Jun 9, 2026

Uh oh!

github-advanced-security AI commented Jun 9, 2026

Uh oh!

codecov Bot commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

DanieleDeSensi commented Jun 9, 2026

Uh oh!

github-advanced-security AI commented Jun 9, 2026

What Enabling Code Scanning Means:

Uh oh!

codecov Bot commented Jun 9, 2026

Welcome to Codecov 🎉

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants