Summary
Moon has no probabilistic cardinality estimation. Every Redis-based analytics pipeline that uses unique-visitor counting, distinct-event counting, or dashboard UV metrics depends on HLL and cannot migrate.
Missing commands
Design notes
- Redis uses the original Flajolet et al. HyperLogLog with 14-bit registers (~12KB per key, ~0.81% std error). Moon should match the wire format exactly —
PFADD / PFCOUNT output must be bit-identical to Redis for consistency tests.
- Redis uses two encodings (sparse + dense) and switches automatically. We likely need both to pass the byte-identical consistency harness.
- Storage: new value type in the shard's entry enum, alongside
String/List/Hash/Set/ZSet/Stream. No new allocation pattern — just another Bytes-backed payload.
- No hot-path allocations for
PFADD on an existing key.
PFMERGE is the merge-of-dense-registers operation — SIMD candidate (AVX2 max-of-registers is trivial).
Acceptance criteria
- Byte-identical output vs Redis for the canonical test vectors.
- Consistency test coverage for sparse→dense promotion boundary.
- Benchmark vs Redis — HLL is allocation-light, Moon should be faster on the add path.
- Works on both architectures and both runtimes.
References
- Redis source:
src/hyperloglog.c
- Paper: Flajolet, Fusy, Gandouet, Meunier (2007)
Summary
Moon has no probabilistic cardinality estimation. Every Redis-based analytics pipeline that uses unique-visitor counting, distinct-event counting, or dashboard UV metrics depends on HLL and cannot migrate.
Missing commands
PFADD key [element ...]PFCOUNT key [key ...]PFMERGE destkey sourcekey [sourcekey ...]Design notes
PFADD/PFCOUNToutput must be bit-identical to Redis for consistency tests.String/List/Hash/Set/ZSet/Stream. No new allocation pattern — just anotherBytes-backed payload.PFADDon an existing key.PFMERGEis the merge-of-dense-registers operation — SIMD candidate (AVX2 max-of-registers is trivial).Acceptance criteria
References
src/hyperloglog.c