Skip to content

feat(napplet): uploads land on this device first and lose their metadata - #157

Closed
Origami74 wants to merge 1 commit into
mainfrom
feat/upload-local-first
Closed

Origami74 wants to merge 1 commit into
mainfrom
feat/upload-local-first

Conversation

@Origami74

@Origami74 Origami74 commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

What

NAP-UPLOAD always works, at least on this device, and strips metadata before anything is stored or sent.

Why

An upload failed offline, and files went up exactly as the napplet sent them — camera EXIF (GPS) included.

How

  • Every upload is stripped, hashed, written to the kept Blossom store (never evicted), then sent to the configured servers best effort. Offline-only, no internet, no server or every server refusing still answers ok with url: "blossom:sha256:<hex>", localOnly: true and remoteError (Myco additions; status stays complete, the spec's state list is closed). Only a napplet without the grant gets policy denied.
  • Stripping (nap/upload/strip.rs): JPEG drops APPn except JFIF/ICC/Adobe, COM and trailing bytes; PNG keeps an allow-list of chunks; WebP drops EXIF/XMP and fixes VP8X and the RIFF size; GLB drops generator, every extras and KHR_xmp_json_ld, and re-strips embedded images (skipped for meshopt or overlapping views, documented). A known type that won't parse is refused, never uploaded unstripped.
  • Orientation kept: when the first EXIF block (JPEG APP1, PNG eXIf, WebP EXIF) has an Orientation (0x0112) of 2–8, it is replaced by a new minimal EXIF — TIFF header in the original byte order, IFD0 with only that entry, no next IFD (26 bytes). JPEG puts it right after SOI/APP0; PNG in place with a new CRC; WebP in place with the VP8X EXIF flag set. Orientation 1, none, or malformed EXIF → no EXIF at all. GPS, camera, timestamps, thumbnails, XMP, IPTC and comments still go. If stripping changed the bytes the result carries originalSha256, which is not published.
  • Paired peers fetch kept blobs from the mesh Blossom (:24243).
  • Docs: NAP-UPLOAD.md (This device first, Metadata, Offline, Results), CHANGELOG.

Testing

  • cargo test -p myco-napplet-runtime passes; per-format stripping tests with synthesized files, plus orientation tests: Orientation 6/8 with a GPS IFD, Make and DateTime in both byte orders → output EXIF is exactly the 26-byte orientation-only block (JPEG, PNG, WebP incl. Exif\0\0-prefixed chunk); Orientation 1 / absent / malformed → no EXIF; truncated, out-of-range offsets and counts, wrong type, invalid value → no panic, nothing kept; cargo test -p myco-core upload passes. blossom tests: 3 LMDB ENOSPC failures locally, unrelated.
  • Not yet tested on device. Manual test: two paired phones; on A turn on Offline-only and upload a camera photo from a napplet (Doodleduo) → localOnly: true, the kept file has no EXIF beyond Orientation, and a portrait photo shows upright; on B fetch blossom:sha256:<hex> through resource / mesh.blobs → served by A over the mesh.

Every NAP-UPLOAD is stripped, hashed and written to the kept Blossom store
before it is sent to the configured servers best effort. Offline-only, no
internet or no server still answers ok with a blossom: URL and localOnly.
Paired peers fetch kept blobs over the mesh Blossom.

Stripping (nap/upload/strip.rs) walks segments and chunks, never re-encodes:
JPEG drops APPn but JFIF/ICC/Adobe, COM and trailing bytes; PNG keeps an
allow-list of chunks; WebP drops EXIF/XMP and fixes VP8X and the RIFF size;
GLB drops generator, extras and KHR_xmp_json_ld and strips embedded images.
A known type that won't parse is refused.

The EXIF Orientation is kept, so portrait photos stay upright: when the
first EXIF block (JPEG APP1, PNG eXIf, WebP EXIF) has an Orientation of
2-8, it is replaced by a minimal 26-byte TIFF holding only that tag, in the
original byte order. GPS, camera, timestamps and thumbnails still go;
malformed EXIF is dropped whole.
@Origami74
Origami74 force-pushed the feat/upload-local-first branch from e7f7bc6 to 8d3a0b1 Compare October 19, 2026 16:23
@Origami74

Copy link
Copy Markdown
Owner Author

Fixed the orientation blocker in 8d3a0b1. Stripping now keeps the EXIF Orientation tag, and nothing else from the EXIF, for JPEG, PNG and WebP. When it is 2–8, the old EXIF is replaced by a minimal TIFF with IFD0 holding just tag 0x0112, in the original byte order. For JPEG it goes right after SOI/APP0. PNG gets a new CRC. WebP gets the VP8X EXIF flag. Orientation 1, a missing tag or malformed EXIF means no EXIF at all, and the parse is bounds-checked. Tests cover both byte orders, GPS-bearing EXIF and malformed input. Also rebased on main.

@Origami74

Copy link
Copy Markdown
Owner Author

Closing: metadata stripping belongs on the client side (in the napplet), not in Myco's NAP-UPLOAD host. The host should store and send files as the napplet hands them over. The branch feat/upload-local-first stays for reference.

@Origami74 Origami74 closed this Oct 23, 2026
Origami74 added a commit that referenced this pull request Oct 23, 2026
…ploads

Symptom: a napplet could not share a file without internet. With
offline-only on, no internet or no server, NAP-UPLOAD failed and nothing
was kept, so Circle peers couldn't get the file either.

Cause: uploads had one rail, the user's Blossom servers, and kept a copy
on the device only after a server took it.

Fix, within the NAP-UPLOAD spec (a shell may name its own rails):
- New rail blossom-local: the file goes to this device's own Blossom
  store, no signature, no internet, works with offline-only on. It
  answers complete with a blossom:sha256:<hex> URL that NAP-RESOURCE
  resolves here and from Circle peers. upload.info lists it with
  returns ["blossom"]; blossom keeps returns ["https"].
- A blossom upload no server could take still fails, but a copy is kept
  for a retry or a peer. Nothing is kept for an upload the user did not
  approve (any signing failure) or one refused because another runs:
  UploadError::keep_locally says which.
- Keeping without a server is budgeted per napplet, 64 MiB per 24 h
  (LocalUploads); past it blossom-local is quota exceeded. Kept files
  are always written to the kept store, never left in the cache.
- No metadata stripping: that stays the napplet's job.

Replaces #157.
Origami74 added a commit that referenced this pull request Oct 23, 2026
…ploads (#160)

Symptom: a napplet could not share a file without internet. With
offline-only on, no internet or no server, NAP-UPLOAD failed and nothing
was kept, so Circle peers couldn't get the file either.

Cause: uploads had one rail, the user's Blossom servers, and kept a copy
on the device only after a server took it.

Fix, within the NAP-UPLOAD spec (a shell may name its own rails):
- New rail blossom-local: the file goes to this device's own Blossom
  store, no signature, no internet, works with offline-only on. It
  answers complete with a blossom:sha256:<hex> URL that NAP-RESOURCE
  resolves here and from Circle peers. upload.info lists it with
  returns ["blossom"]; blossom keeps returns ["https"].
- A blossom upload no server could take still fails, but a copy is kept
  for a retry or a peer. Nothing is kept for an upload the user did not
  approve (any signing failure) or one refused because another runs:
  UploadError::keep_locally says which.
- Keeping without a server is budgeted per napplet, 64 MiB per 24 h
  (LocalUploads); past it blossom-local is quota exceeded. Kept files
  are always written to the kept store, never left in the cache.
- No metadata stripping: that stays the napplet's job.

Replaces #157.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant