Add codec FFI layer with Zstd encoding support

[anylots]

This PR reorganizes the Go bindings from decoder-only to codec, adds Zstd compression support alongside decompression, and introduces encoder/codec FFI crates. It also updates platform static libraries for Linux amd64 and Darwin arm64, plus related docs, tests, and build configuration.

[Copilot]

Pull request overview

This PR upgrades the project from a decoder-only Go binding to a full codec (compress + decompress) by adding a Rust encoder pipeline, aggregating decoder+encoder FFI into a single staticlib for cgo linking, and updating build/test/docs accordingly.

Changes:

• Introduce Rust encoder core + encoder FFI, and add a codec-ffi aggregator crate that builds the single Go-linked libmorph_da_zstd.a.
• Replace bindings/decoder with bindings/codec, adding Go APIs/tests for both CompressMorphDABatch and DecompressMorphDABatch.
• Update build tooling (Makefile + manylinux Docker build) and workspace dependencies to support the new codec layout.

Reviewed changes

Copilot reviewed 21 out of 26 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
README.md	Updates repo-level docs to describe codec semantics and new Go binding path/targets.
Makefile	Switches build/install/test targets from decoder-only to codec staticlib workflow.
crates/encoder/zstd-ffi/src/lib.rs	Adds C ABI wrappers for compression functions used by Go via cgo.
crates/encoder/zstd-ffi/Cargo.toml	Defines encoder FFI crate as rlib for aggregation into the single staticlib.
crates/encoder/core/src/lib.rs	Implements encoder core logic using zstd with Morph DA-specific parameters.
crates/encoder/core/Cargo.toml	Adds encoder core crate and zstd dependency (plus decoder-core for round-trip tests).
crates/decoder/zstd-ffi/Cargo.toml	Changes decoder FFI to rlib to avoid duplicate Rust runtime symbols.
crates/codec-ffi/src/lib.rs	Re-exports decoder+encoder FFI to keep no_mangle symbols reachable in the final staticlib.
crates/codec-ffi/Cargo.toml	Introduces the single staticlib (libmorph_da_zstd.a) linked by Go.
Cargo.toml	Adds new workspace members and introduces git-based zstd dependency.
Cargo.lock	Locks new dependency graph including zstd/zstd-sys and new crates.
build/Dockerfile	Updates manylinux build to produce the new codec static library artifact.
bindings/decoder/zstd/libmorph_da_zstd_decode_linux_amd64.go	Removes old decoder-only cgo link stub (linux/amd64).
bindings/decoder/README.md	Removes decoder-only integration guide (replaced by codec guide).
bindings/decoder/go.mod	Removes decoder-only Go module definition.
bindings/codec/zstd/libmorph_da_zstd_linux_amd64.go	Adds codec cgo link stub for linux/amd64.
bindings/codec/zstd/libmorph_da_zstd_darwin_arm64.go	Updates darwin/arm64 cgo link stub to new archive name.
bindings/codec/zstd/decompress.go	Adds Go decompression wrapper over Rust FFI.
bindings/codec/zstd/decompress_test.go	Adds Go tests for decompression behavior and edge cases.
bindings/codec/zstd/compress.go	Adds Go compression wrapper over Rust FFI.
bindings/codec/zstd/compress_test.go	Adds Go tests for compression + round-trip semantics.
bindings/codec/zstd/batch_test.go	Extends existing batch tests to check recompression equivalence.
bindings/codec/README.md	Adds new codec integration guide for Go consumers.
bindings/codec/go.mod	Adds new Go module definition for codec bindings.

Comments suppressed due to low confidence (1)

bindings/codec/zstd/batch_test.go:48

• batchData[:len(recompressed)] can panic if the recompressed output is larger than the original payload. This makes the test flaky and can hide real mismatches behind a runtime panic; add an explicit length guard before slicing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.