Repository navigation
Closed
Description
Track the prototype-to-production path for an S3-compatible API on top of the Forge stack. The S3 listener (codename ms3t) currently lives at pkg/ms3t/ in this repo. See pkg/ms3t/architectural.md for the prototype-level design notes, the choice points, and open questions.
Why sprue, not a separate repo? A pragmatic prototype-stage call: keep ms3t in-process with sprue so it inherits sprue's identities + clients with zero auth coordination. Up for re-evaluation once we settle a tenant model.
In scope for this epic
- build(deps): bump github.com/testcontainers/testcontainers-go from 0.42.0 to 0.43.0 #37 — Define a sensible S3 API subset for the initial spike
- build(deps): bump github.com/aws/aws-sdk-go-v2/service/dynamodb from 1.57.3 to 1.62.1 #38 — Bootstrap S3 facade (HTTP listener + MST + Forge integration) — anchored on refactor: make did:plc resolution always on and config directory URL optional with default #34
- build(deps): bump github.com/testcontainers/testcontainers-go/modules/minio from 0.40.0 to 0.43.0 #39 — Bucket mutation data storage: investigate "separate space for block diffs"
- build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.96.4 to 1.106.1 #40 — Add ms3t to smelt + S3 smoke tests
- build(deps): bump the github-actions group with 8 updates #41 — ms3t telemetry + Grafana dashboards
- build(deps): bump github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue from 1.20.34 to 1.20.55 #42 — Stand up staging deployment + production infra prep (spike)
- build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.19.11 to 1.19.31 #43 — Flesh out tenant management API (spike)
Status snapshot
| # | Title | Status |
|---|---|---|
| #37 | Define S3 API subset for the spike | proposed |
| #38 | Bootstrap S3 facade | in flight (#34) |
| #39 | Bucket mutation data storage proposal | proposed |
| #40 | ms3t in smelt + S3 smoke tests | proposed |
| #41 | ms3t telemetry + Grafana dashboards | partial (#35, storacha/smelt#23) |
| #42 | Staging deployment + production infra (spike) | proposed |
| #43 | Tenant management API (spike) | proposed |
Explicitly deferred
- Sharding. Assume all data received is <256 MB. The sharding proposal in flight is fil-one/RFC#2; revisit once it lands. Adjacent prior art: storacha/RFC#65 (Filepack archive format), storacha/RFC#66 (Virtual DAG in Sharded DAG Index), and
shard.rfcin this repo. - Multipart upload,
aws-chunkedbody decoding, AWS Sig V4 auth, pre-signed URLs, server-side encryption,x-amz-meta-*headers, versioning, object lock — these are FilOne hard requirements but are out of scope until build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.19.11 to 1.19.31 #43's spike proposes a sequencing. - GC of unreferenced bodies, multi-instance / sticky-bucket routing, direct-passthrough to piri — see
pkg/ms3t/architectural.md#future-directions.
Metadata
Metadata
Assignees
Labels
No labels
Activity