Skip to content

S3 on Forge #4

Closed
Closed

Description

@alanshaw

Track the prototype-to-production path for an S3-compatible API on top of the Forge stack. The S3 listener (codename ms3t) currently lives at pkg/ms3t/ in this repo. See pkg/ms3t/architectural.md for the prototype-level design notes, the choice points, and open questions.

Why sprue, not a separate repo? A pragmatic prototype-stage call: keep ms3t in-process with sprue so it inherits sprue's identities + clients with zero auth coordination. Up for re-evaluation once we settle a tenant model.

In scope for this epic

Status snapshot

# Title Status
#37 Define S3 API subset for the spike proposed
#38 Bootstrap S3 facade in flight (#34)
#39 Bucket mutation data storage proposal proposed
#40 ms3t in smelt + S3 smoke tests proposed
#41 ms3t telemetry + Grafana dashboards partial (#35, storacha/smelt#23)
#42 Staging deployment + production infra (spike) proposed
#43 Tenant management API (spike) proposed

Explicitly deferred

  • Sharding. Assume all data received is <256 MB. The sharding proposal in flight is fil-one/RFC#2; revisit once it lands. Adjacent prior art: storacha/RFC#65 (Filepack archive format), storacha/RFC#66 (Virtual DAG in Sharded DAG Index), and shard.rfc in this repo.
  • Multipart upload, aws-chunked body decoding, AWS Sig V4 auth, pre-signed URLs, server-side encryption, x-amz-meta-* headers, versioning, object lock — these are FilOne hard requirements but are out of scope until build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.19.11 to 1.19.31 #43's spike proposes a sequencing.
  • GC of unreferenced bodies, multi-instance / sticky-bucket routing, direct-passthrough to piri — see pkg/ms3t/architectural.md#future-directions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions