Check script lockfiles in CI by AlexWaygood · Pull Request #29021 · astral-sh/ruff

AlexWaygood · 2026-09-30T12:51:55Z

Check that repository PEP 723 scripts have up-to-date lockfiles in CI and dogfood ty on each script, including scripts outside scripts/. The helper can also update the lockfiles and check the scripts locally with uv run scripts/check-scripts.py. It builds ty once unless --ty-binary supplies an existing executable. CI passes --check to verify lockfiles without updating them.

I started with uv's equivalent check, but a few small modifications were required to make the check Ruff-compatible. After making those changes, I was unhappy with the readability of the script, so I rewrote it in Python for better readability and maintenance.

This PR also enables the missing-exclude-newer-package-lock preview feature, which omits exclude-newer-package overrides from a script lockfile when the corresponding package is absent from that lockfile's resolution. Without that feature, changing an unrelated exemption in our global pyproject.toml file can cause every script's lockfile to be updated on its next run. The feature requires uv 0.12.10 or later, so our pyproject.toml file now enforces that minimum version of uv using the tool.uv.required-version configuration setting.

The broader ty check found a bug in the AST generator: if a source-order field is missing, the generator could reuse the last field or append None. It now raises an error instead.

Closes #27845.

astral-sh-bot · 2026-09-30T12:53:34Z

Typing conformance results

No changes detected ✅

Current numbers

The percentage of diagnostics emitted that were expected errors held steady at 98.24%. The percentage of expected errors that received a diagnostic held steady at 98.24%. The number of fully passing files held steady at 134/146.

astral-sh-bot · 2026-09-30T12:54:49Z

Memory usage report

Memory usage unchanged ✅

astral-sh-bot · 2026-09-30T12:57:23Z

`ecosystem-analyzer` results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

AlexWaygood · 2026-09-30T12:59:04Z

Closes #27845.

Okay, codex only reminded me that this PR already existed after I'd already spent a bunch of time on this. But that PR has merge conflicts, and I think the script being added here is slightly superior in several other ways too:

Having the script as a separate file rather than just embedding shell into the CI workflow means that we can easily run it locally to update all script lockfiles at once
Changes to exclude-newer exemptions don't cause uv lock to update a lockfile unless you pass --refresh, which that PR doesn't do
That PR has merge conflicts 😆

astral-sh-bot · 2026-09-30T13:03:33Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

MichaReiser · 2026-09-30T13:12:51Z

+COMPLETION_FIXTURES = Path("crates/ty_completion_eval/truth")
+
+
+def scripts() -> Iterator[Path]:


Can't we use ty for this?

TY_UV=scripts UV_LOCKED=1 ty check ./scripts

which we already have in the dogfood step

ruff/.github/workflows/ci.yaml

Lines 426 to 431 in fad83a7

- name: Dogfood ty on scripts

env:

# Enable experimental support for installing PEP 723 script dependencies.

TY_UV: scripts

UV_LOCKED: "true"

run: cargo run --quiet -p ty check scripts/*.py --color=always

we have a bunch of scripts that are not located in the scripts/ directory, which also need to be kept up to date

You can pass them as CLI arguments. It just feels a bit silly to have this huge script.

meh, I don't want to have to remember to update this workflow every time we add a new script. I'd much prefer automated discovery, which is what the new script I'm adding does.

this huge script

It doesn't seem that big to me 😆

Ideally, we'd probably move all the scripts in one places, which would simplify this. I also think we do want the dogfood step to run on all scripts, meaning you still need to update that one whenever you add a misplaced script.

Maybe. I do think there's some sense in having scripts that really only pertain to a single crate be located in those crates. The script for generating files in the ruff_python_ast crate does in some sense belong in that crate; the mdtest runner for ty_python_semantic does feel like it belongs in that crate.

Also, even if we did move all scripts inside scripts/, I still think the script I'm adding provides value, because it gives you a really easy way to update all lockfiles at once locally if this CI step fails.

I don't mind the script for updating, but I don't think we should duplicate the dogfood step. Instead, scripts that should be checked should be added there.

MichaReiser · 2026-09-30T13:31:20Z

+            ):
+                continue
+
+            if b"# /// script" in full_path.read_bytes().splitlines():


You can replace parts (most, all) of this by calling uv workspace list --scripts --preview-features workspace-list-scripts

that silently skips scripts with invalid configuration rather than rejecting them

This seems unfortunate. Good that ty check catches that for you :) (I guess that might not be true, it doesn't report scripts with a missing closing tag, but they're technically not scripts)

Maybe we need a much better ty project list --scripts 😆

I updated the new script so that it runs ty on each script as it discovers them, and updated the dogfooding CI job to just use this script, and removed the new CI job that an earlier version of this PR added. It found ty errors in scripts that were not inside the scripts/ directory, so the PR now fixes those as well

MichaReiser · 2026-09-30T15:10:56Z

-        env:
-          # Enable experimental support for installing PEP 723 script dependencies.
-          TY_UV: scripts
-          UV_LOCKED: "true"


Hmm, I think this is worse. The point of dogfooding is that we dogfood it. I also find this much more difficult to understand.

Well, I just wanted a CI check that verified that all our lockfiles were up to date, including ones that were not included in the ty-dogfooding check 😆

I can revert to the previous version where it was an entirely separate check? I don't particularly think it's important for every script in the repo to be included in the ty dogfooding check TBH. We get the vast majority of them checked in the existing dogfooding check on main, and I think that's enough.

But I really would like all scripts in the repo checked to see if their lockfiles are up to date. It's very annoying for me when crates/ty_python_semantic/mdtest.py.lock changes locally after I invoke it when working on a PR branch, because some previous PR that touched pyproject.toml should have updated that lockfile but didn't.

My preferred solution would be:

Keep your script to update the locks.

I would probably just change the existing dogfooding step and add the two missing scripts. That's the easiest change

But if you want to keep your script, I'd probably use your script to find all scripts, and pass them to ty check in the dogfooding step (you can write them to a @ file or use echo or something like that)

But if you want to keep your script, I'd probably use your script to find all scripts, and pass them to ty check in the dogfooding step (you can write them to a @ file or use echo or something like that)

Okay, I did that. We still need a separate check to confirm the lockfiles are up to date, though. We set UV_LOCKED: 1 in CI while doing the ty dogfooding check, but UV_LOCKED=1 doesn't imply --refresh, and there is no environment variable that you can set right now that does imply --refresh. This is one reason why I kept running into issues where scripts had stale lockfiles that would be updated as soon as I ran them locally, despite them passing the ty-dogfooding step in CI.

Check script lockfiles in CI

fad83a7

AlexWaygood added the ci Related to internal CI tooling label Sep 30, 2026

AlexWaygood marked this pull request as ready for review September 30, 2026 12:59

AlexWaygood requested a review from a team as a code owner September 30, 2026 12:59

astral-sh-bot Bot requested a review from sharkdp September 30, 2026 12:59

MichaReiser reviewed Sep 30, 2026

View reviewed changes

Check script lockfiles and dogfood ty together

1451ee9

sharkdp removed their request for review September 30, 2026 15:04

AlexWaygood requested a review from MichaReiser September 30, 2026 15:04

MichaReiser reviewed Sep 30, 2026

View reviewed changes

Pass script paths to ty via an argument file

9689857

AlexWaygood requested a review from MichaReiser September 30, 2026 16:55

		COMPLETION_FIXTURES = Path("crates/ty_completion_eval/truth")


		def scripts() -> Iterator[Path]:

	- name: Dogfood ty on scripts
	env:
	# Enable experimental support for installing PEP 723 script dependencies.
	TY_UV: scripts
	UV_LOCKED: "true"
	run: cargo run --quiet -p ty check scripts/*.py --color=always

Conversation

AlexWaygood commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

astral-sh-bot Bot commented Sep 30, 2026

No changes detected ✅

Uh oh!

astral-sh-bot Bot commented Sep 30, 2026

Memory usage report

Uh oh!

astral-sh-bot Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ecosystem-analyzer results

Uh oh!

AlexWaygood commented Sep 30, 2026

Uh oh!

astral-sh-bot Bot commented Sep 30, 2026

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

Uh oh!

MichaReiser Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaReiser Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Sep 30, 2026 • edited by MichaReiser Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AlexWaygood commented Sep 30, 2026 •

edited

Loading

astral-sh-bot Bot commented Sep 30, 2026 •

edited

Loading

`ecosystem-analyzer` results

`ruff-ecosystem` results

MichaReiser Sep 30, 2026 •

edited

Loading

MichaReiser Sep 30, 2026 •

edited

Loading

AlexWaygood Sep 30, 2026 •

edited by MichaReiser

Loading