Conversation
|
HyperFrames walkthrough of this PR. |
`lfs.serve_via = "signed_url"` now applies to the upload half of the batch as
well: the `upload` action becomes a presigned PUT straight at the store, the way
GitHub and GitLab hand out LFS uploads. Downloads already did this; uploads always
proxied through `PUT /info/lfs/objects/{oid}`. `proxy` stays the default.
A signed PUT is only ever handed out bound to the oid. The LFS keyspace is
content-addressed and served back to everyone as immutable, so a PUT that accepts
any bytes is a write primitive for every oid a client can name.
`ObjectStore::signed_put_url(key, ttl, checksum_sha256)` therefore answers
`Ok(None)` unless the backend both rejects a body whose sha256 differs and keeps
the header carrying that checksum inside the signature:
* S3 signs `x-amz-checksum-sha256` (mismatch: BadDigest; dropped header:
SignatureDoesNotMatch), and the URL is discarded if the header ever falls
outside `X-Amz-SignedHeaders`.
* GCS cannot: `x-goog-hash` validates only CRC32C/MD5, neither of which walgit
knows for an object it has never seen, and `x-goog-content-sha256` is
UNSIGNED-PAYLOAD on the signed-URL path. Uploads stay proxied there.
* memory signs nothing.
`verify` stays walgit's either way. Since `authenticated: true` keeps git-lfs
from putting walgit's credential on the store's URL — and git-lfs applies that
flag to the verify POST too — the verify action carries the credential the client
used on the batch. `lfs.max_object_bytes` is only enforceable where the bytes
pass through, so an object over the cap is not signed and the proxy href refuses
it. The proxy path's own size + sha256 gate is untouched.
Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
… bucket The memory store's fake signed PUT takes a base URL instead of a boolean, so a test can point it at a bucket it controls. `lfs_signed_url.rs` runs a real `git lfs push` at one that checks the signed checksum the way S3 does, then asserts the object landed and `verify` came back to walgit. Both halves of the `authenticated` decision are pinned: with the flag dropped the shape test fails, and with the credential off the `verify` action git-lfs reports "Authorization error: .../info/lfs/verify" and the push fails. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
A table and a paragraph needed the blank line that separates them from what precedes. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
A signed upload changes the cost model, not the request count: signing is local crypto, and the object's bytes stop passing through walgit entirely. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Upstream reworked `client_authorization` to read the edge-forwarded copy only when an edge announced the `client-authorization` capability; this branch had only widened the old body's visibility. Keep upstream's semantics and re-apply the `pub(crate)` the LFS batch needs, then clear the workspace clippy gate: `if let` for the single-pattern signed-upload match, char-safe `get` for the oid path segments, and `is_ok_and` for the git-lfs probe.
3b10467 to
03f0b42
Compare
main removed bundles in the packfiles work (tobi#52-tobi#58) and renamed the shared BundleServe enum to LfsServe, which is what conflicted: - lfs.rs: took main's LfsServe and its wildcard arm, and kept this branch's `authenticated: true` on a signed download href. main had folded the fallback into the match; keeping the Option lets the authenticated signal stay tied to whether the URL was actually signed. - Renamed the remaining BundleServe references, here and in tests/lfs_signed_url.rs. - justfile: main's test list plus this branch's lfs_signed_url. - docs/ROUNDTRIPS.md: kept both sets of rows; the LFS batch and signed-upload rows and main's bundle-removal and checkpoint rows describe different protocols. just warnings, just clippy and just test are green, including the five lfs_signed_url tests.
|
Merged main. The packfiles work renamed the shared In In
|

Walkthrough
HyperFrames walkthrough of this PR.
mp4
lfs.serve_via = "signed_url"now applies to the upload half of the batch as well: theuploadaction becomes a presigned PUT straight at the store, the way GitHub and GitLab hand out LFS uploads. Downloads already did this; uploads always proxied throughPUT /info/lfs/objects/{oid}.proxystays the default, and its size + sha256 gate is untouched.A signed PUT is only ever handed out bound to the oid — the LFS keyspace is content-addressed and served back to everyone as immutable, so a PUT that accepts any bytes is a write primitive for every oid a client can name.
ObjectStore::signed_put_url(key, ttl, checksum_sha256) -> Result<Option<SignedPut>>therefore answersOk(None)unless the backend both rejects a body whose sha256 differs and keeps the header carrying that checksum inside the signature:x-amz-checksum-sha256, signed. Mismatching body:BadDigest; dropped header:SignatureDoesNotMatch. The URL is discarded if the header ever falls outsideX-Amz-SignedHeaders.None.x-goog-hashvalidates only CRC32C/MD5, neither of which walgit knows for an object it has never seen, andx-goog-content-sha256isUNSIGNED-PAYLOADon the signed-URL path. Uploads stay proxied.None.A signing failure logs a WARN and falls back to the proxying href; it never fails a push.
lfs.max_object_bytesis only enforceable where the bytes pass through, so an object over the cap is not signed and the proxy href refuses it with 413.verifystays walgit's: the store guarantees the content, we still confirm the object arrived at the promised size. Sinceauthenticated: truekeeps git-lfs from putting walgit's credential on the store's URL — and git-lfs applies that flag to the verify POST too (tq/verify.go) — the verify action carries the credential the client used on the batch, as GitHub does. Dropping it makesgit lfs pushfail withAuthorization error: …/info/lfs/verify, which the test below asserts.No new config key.
signed_get_urldownloads now also sendauthenticated: true, so git-lfs stops asking a credential helper for the bucket's host.Round trips
Request counts are unchanged: S3 presigning is offline, GCS returns without a call, and
signed_put_urlis only reached underserve_via = "signed_url". What changes is where the bytes go — walgit no longer PUTs the object at all — sodocs/ROUNDTRIPS.md§2 gains a row for the LFS batch and one for the signed upload.Tests
crates/walgit-store/src/s3.rs— presigning needs no bucket, so the guarantee is a unit test: the checksum header is required of the client and listed inX-Amz-SignedHeaders.crates/walgit-store/tests/contract.rs— every backend:signed_put_urlis bound to the sha256 or absent; memory is absent.crates/walgit-server/tests/lfs_signed_url.rs— the batch response under a signing store, a store that cannot sign, a denied signer, and the cap; plus a realgit lfs pushagainst a mock bucket that checks the signed checksum the way S3 does and hands the object to the store, thenverifyhere.just test(withlfs_signed_urladded to the tier),just e2eandcargo test -p walgit-server --test simpass,lfs_upstreamincluded.Notes
e2e::lfs_roundtrip_when_availablefails on this branch and identically onmainat 6d8fa54: git-lfs 3.7.1 ignores-c filter.lfs.*on a clone without a localgit lfs install, so the checkout keeps the pointer. Left alone.just clippyis likewise red before and after (the 1.97.1 gate added in 5ccc405); this change adds no new finding except oneexpect()incontract.rs, which already has 66 of them.