Skip to content

toolshed: fix copyright-year check to work outside staged-file workflows - #3352

Merged
juenglin merged 10 commits into
NVIDIA:mainfrom
juenglin:fix/spdx-year-check
Nov 4, 2026
Merged

juenglin merged 10 commits into
NVIDIA:mainfrom
juenglin:fix/spdx-year-check

Conversation

@juenglin

@juenglin juenglin commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Description

closes #2273

toolshed/check_spdx.py used git diff --staged as a proxy for "was this file changed recently?" That signal is only valid during a git commit. Under pre-commit run --all-files or CI nothing is staged, so the gate short-circuited to return True and the copyright-year check was silently skipped for every file.

Root fix (toolshed/check_spdx.py): Replace is_staged() with last_modified_year(). It returns the current year when git status --porcelain reports any working-tree change (staged, unstaged, or untracked), and otherwise the year of the last commit that touched the file (git log -1). A file passes when its copyright end year is at least that year, so the result is the same under a git commit hook, pre-commit run --all-files, and CI. --fix is unchanged and always writes the current year, because committing the fix is itself a change made this year.

CI: .github/workflows/pre-commit.yml changes fetch-depth from 1 to 0 and adds filter: blob:none, because git log -1 -- <file> needs the full commit history. In a depth-1 clone every file appears to be introduced by the tip commit, so each file would be judged against the tip commit's year and files with older copyright years would fail. blob:none keeps all commits and trees but skips historical file contents, which a path-limited git log does not read; in a local test, a blobless clone returned the same last-change dates as full history without fetching any blobs. The other hooks only read the checked-out tree. An alternative design is to pass only the files changed against a base ref (--from-ref/--to-ref), which avoids the deep fetch but needs the year rule scoped away from --all-files; this PR does not do that. .github/workflows/ci-nightly.yml also installs pathspec for the CI-tools unit tests.

Backfill: Running the corrected check surfaced 20 files that were last committed in 2026 but still carried an earlier copyright end year. They are fixed in the second commit.

Tests: ci/tools/tests/test_check_spdx.py builds temporary git repositories with controlled commit dates and runs the script against them. It covers a file committed this year with a stale copyright (the bug), a file untouched since last year (no false positive), a staged edit checked and then rewritten by --fix, and a clean stale file that --fix rewrites and that is then accepted after the fix is committed. Years are derived from the current year so the tests do not expire. Run with python -m pytest ci/tools/tests/test_check_spdx.py -v --noconftest. Against the previous check_spdx.py, the stale-copyright case and the clean-stale-file case fail; the other two pass on both and guard existing behavior.

Checklist

  • I have read the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@juenglin juenglin added this to the CI milestone Nov 3, 2026
@juenglin juenglin added bug Something isn't working CI/CD CI/CD infrastructure labels Nov 3, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@juenglin juenglin self-assigned this Nov 3, 2026
@juenglin
juenglin requested a review from rwgk November 3, 2026 23:39
@github-actions github-actions Bot added cuda.core Everything related to the cuda.core module cuda.pathfinder Everything related to the cuda.pathfinder module labels Nov 3, 2026
Replace is_staged() with last_modified_year(), which consults git status
for working-tree changes and git log -1 for committed files.  The old
is_staged() gate used git diff --staged as a proxy for 'was this file
changed recently', which only holds during a git commit.  Under
pre-commit run --all-files or CI nothing is staged, so the gate
short-circuited to 'return True' and the year was never checked.

Also change fetch-depth from 1 to 0 in the pre-commit CI workflow.
git log -1 needs enough history to find when each file was last touched;
a depth-1 shallow clone returns empty output for files not changed in the
tip commit, causing silent false passes on stale copyright years.

Fixes NVIDIA#2273.
Running check_spdx.py --fix --all-files after the previous commit
updated files last committed in 2026 that still had a 2025 (or earlier)
copyright end year.  These were invisible to the hook before because
is_staged() always returned False in the --all-files invocation mode.
@juenglin
juenglin force-pushed the fix/spdx-year-check branch from 8944326 to 09ab7f6 Compare November 3, 2026 23:41
--fix wrote the current year even for files whose last change was in an
earlier year, which would re-date untouched files whenever the year
rolled over.  Use the same last_modified_year() the check compares
against, and report it in the OUTDATED message.

Add ci/tools/tests/test_check_spdx.py, which builds temporary git
repositories with controlled commit dates and covers committed, staged,
unstaged and untracked files, plus --fix.  Install pathspec in the
nightly CI-tools job so the tests run there instead of skipping.
…behavior

Drop cases that exercise the same git-status branch or restate another
test: unstaged and untracked edits, the current-copyright happy path,
copyright-ahead-of-commit, and the stale-relative-to-last-commit case
already pinned by the --fix test.  Fold the staged-edit check and the
--fix rewrite of a modified file into one test.
Writing the last-commit year for a clean, stale file is undone by
committing the fix, since that commit is itself a change made this year
(the cycle described in NVIDIA#1514).  Revert to writing the current year and
restore the original OUTDATED message.

Replace the last-commit-year test with one that fixes a clean stale file,
commits the result, and checks that it is accepted.
@juenglin
juenglin requested a review from mdboom November 3, 2026 23:59
check_spdx.py needs commit history to find when each file last changed.
Correct the comment: in a shallow clone every file appears to be
introduced by the tip commit, so each would be judged against the tip
commit's year (not a false pass, as the comment previously claimed).

Add filter: blob:none so the job fetches all commits and trees but not
historical file contents, which a path-limited git log does not read.
@juenglin
juenglin marked this pull request as ready for review November 4, 2026 00:07
@juenglin

juenglin commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 8f6000a

@juenglin juenglin modified the milestones: CI, cuda.bindings next Nov 4, 2026
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review
Summary

Summary by CodeRabbit

  • Maintenance
    • Updated copyright notices across project materials to include 2026.
    • Copyright checks now account for each file’s latest changes and include expanded test coverage.
    • CI setup now pins an additional testing utility and retrieves full commit history while omitting historical file contents.
  • Tests
    • Marked a CUDA memory-pool test as thread-unsafe due to a synchronization race.
  • Other
    • No application behavior changes are included in this update.

The SPDX checker now uses working-tree changes or the file’s latest commit to determine the year for copyright validation. Tests cover committed and edited files. CI fetches commit history and installs pathspec. Copyright notices were updated across project files.

Changes: SPDX year validation

Layer / File(s) Summary
Year detection and validation
toolshed/check_spdx.py, ci/tools/tests/test_check_spdx.py, .github/workflows/pre-commit.yml, .github/workflows/ci-nightly.yml
The checker uses the current UTC year for files with staged or unstaged changes. Otherwise, it uses the latest commit year, with a current-year fallback. Tests cover committed files, edits, fixes, UTC boundaries, and signed commits. CI fetches full history with blob:none and installs pathspec==0.12.1.
Copyright year updates
.github/actions/*, .github/copy-pr-bot.yaml, ci/tools/guess_latest.sh, cuda_core/*, cuda_pathfinder/*
Copyright notices were extended through 2026 where changed. Some notices were replaced with identical text.

CUDA test marker

Layer / File(s) Summary
Mempool test marker
cuda_core/tests/test_memory.py
test_mempool_trim is marked thread-unsafe with a reason identifying a CUDA legacy-default-stream mempool teardown and synchronization race.

Assessment against linked issues

Objective Addressed Explanation
[#2273] Detect stale copyright years for files changed outside staged-file workflows. ✅

Out-of-scope changes

Code Change Explanation
Mark test_mempool_trim as thread-unsafe (cuda_core/tests/test_memory.py:2949) The linked issue concerns SPDX copyright-year validation and does not state an objective for CUDA test thread-safety markers.

Change: Bug fix

Merge Risk: Low · up to 93b8f

The implementation is broadly mergeable, but one diagnostic can prompt unnecessary edits and the new unstaged-file behavior should receive a focused regression test.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 32ba752 and 8f6000a.

Files selected for processing: 24
  • .github/actions/install_unix_deps/action.yml
  • .github/copy-pr-bot.yaml
  • .github/workflows/ci-nightly.yml
  • .github/workflows/pre-commit.yml
  • ci/tools/guess_latest.sh
  • ci/tools/tests/test_check_spdx.py
  • cuda_core/DESCRIPTION.rst
  • cuda_core/cuda/core/_memory/_graph_memory_resource.pxd
  • cuda_core/cuda/core/_utils/__init__.py
  • cuda_core/docs/source/_templates/autosummary/cyclass.rst
  • cuda_core/docs/source/release.rst
  • cuda_core/docs/source/release/0.3.0-notes.rst
  • cuda_core/docs/source/release/0.4.0-notes.rst
  • cuda_core/docs/source/release/0.5.1-notes.rst
  • cuda_core/tests/cython/build_tests.sh
  • cuda_core/tests/helpers/layout.py
  • cuda_core/tests/test_context.py
  • cuda_pathfinder/cuda/pathfinder/_utils/env_vars.py
  • cuda_pathfinder/cuda/pathfinder/_utils/find_sub_dirs.py
  • cuda_pathfinder/cuda/pathfinder/_utils/platform_aware.py
  • cuda_pathfinder/tests/conftest.py
  • cuda_pathfinder/tests/test_utils_env_vars.py
  • cuda_pathfinder/tests/test_utils_find_sub_dirs.py
  • toolshed/check_spdx.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

Comment thread toolshed/check_spdx.py Resolved
@juenglin
juenglin enabled auto-merge (squash) November 4, 2026 00:22
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

@juenglin
juenglin disabled auto-merge November 4, 2026 00:36
@juenglin
juenglin enabled auto-merge (squash) November 4, 2026 00:39
@rwgk

rwgk commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Drive-by comment (I'll review soon):

Under pre-commit run --all-files or CI nothing is staged

I was wondering about it the whole time, but always stopped, assuming that can't be the case.

That could explain a few "that's weird" observations when merging internally.

@rwgk rwgk left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

codex GPT-6.1-Sol ultra:

No actionable findings in PR #3352, reviewed at 8f6000a.

  • All four new regression cases pass.
  • Full-repository SPDX validation passes without fixes.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 8f6000a and b71d04f.

Files selected for processing: 1
  • toolshed/check_spdx.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between b71d04f and fcb4484.

Files selected for processing: 1
  • toolshed/check_spdx.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Comment thread toolshed/check_spdx.py Resolved
— codex GPT-6 on behalf of Ralf —

Read committer timestamps and convert them explicitly to UTC so clean files
use the same year boundary as staged changes. Disable signature display so
signed commits cannot contaminate the timestamp parsed by the checker.

Cover both New Year timezone directions and signed history with regression
tests. All 312 CI-tool tests and the full pre-commit suite pass.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

rwgk commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

— codex GPT-6 on behalf of Ralf —

/ok to test 0f8ff10

rwgk commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

/ok to test 0f8ff10

— codex GPT-6 on behalf of Ralf —

Mark only test_mempool_trim thread-unsafe, as proposed in NVIDIAgh-3350. Concurrent
legacy-default-stream pool teardown and context synchronization reproduce a
native driver crash without cuda.core; this test hit that failure twice in
the Windows A100 CI configuration.

Keep all parameter cases and assertions enabled while avoiding the known
concurrent lifecycle. Full pre-commit validation passes.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

rwgk commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

/ok to test 93b8f25

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Minor · 🎯 Functional Correctness · check_spdx.py:178-181

toolshed/check_spdx.py:178-181
Minor · Functional Correctness · Quick win

important: Report the year used by validation

For a clean file, validation requires the latest commit year, but the diagnostic always reports CURRENT_YEAR. This can direct users to update the notice beyond the required year. Use the computed validation year and keep --fix writing CURRENT_YEAR.

Suggested fix
-    if int(end_year) >= last_modified_year(filepath):
+    expected_year = last_modified_year(filepath)
+    if int(end_year) >= expected_year:
         return True, blob
 
-    print(f"OUTDATED copyright {years!r} (expected {CURRENT_YEAR!r}) in {filepath!r}")
+    print(f"OUTDATED copyright {years!r} (expected {str(expected_year)!r}) in {filepath!r}")
Nitpick comments: 1
ci/tools/tests/test_check_spdx.py: 1

111-128: Trivial · Maintainability & Code Quality · Quick win

Add a test for an unstaged working-tree edit.

The current suite exercises only staged edits and clean committed files. The working-tree branch in toolshed/check_spdx.py:76-82 has no test. A regression that ignored unstaged changes would pass the suite. The uncovered path is the one that the pre-commit run --all-files and CI cases depend on.

Do not add this assertion to test_staged_edit_requires_current_year_and_fix_writes_it. That test stages its edit on purpose. Add a separate test instead:

Suggested fix
     # The rewrite is stable: a second run accepts the file.
     assert run_checker(repo).returncode == 0
 
+
+def test_unstaged_edit_requires_current_year(repo):
+    # Working-tree edit with nothing staged: `git status` reports it.
+    path = commit(repo, SAMPLE, str(CURRENT_YEAR - 3), year=CURRENT_YEAR - 2)
+    path.write_text(path.read_text(encoding="utf-8") + "# edited\n", encoding="utf-8")
+
+    check = run_checker(repo)
+    assert check.returncode == 1, check.stdout + check.stderr
+    assert f"(expected '{CURRENT_YEAR}')" in check.stdout
+
 
 @pytest.mark.agent_authored(model="claude-sonnet-5.5")
 def test_fix_of_clean_stale_file_survives_being_committed(repo):
Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 0f8ff10 and 93b8f25.

Files selected for processing: 1
  • cuda_core/tests/test_memory.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@juenglin
juenglin merged commit b34498c into NVIDIA:main Nov 4, 2026
119 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working CI/CD CI/CD infrastructure cuda.core Everything related to the cuda.core module cuda.pathfinder Everything related to the cuda.pathfinder module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: check_spdx.py can miss stale copyright years outside staged-file workflows

2 participants