A software engineer building NeoCloud infrastructure, with a focus on inference optimization, KV-cache systems, and heterogeneous resource scheduling.
- Apache Asyncband - Runtime-agnostic concurrency control and async coordination primitives for Rust.
- LMCache - A KV cache layer for offloading and reuse in LLM inference.
- 3FS - A distributed file system for AI training and inference, built on SSDs and RDMA.
- llm-d - A Kubernetes-native stack for distributed LLM serving.
True passions are absolutely running and reading.
I'm also a big fan of blogs and podcasts.
- Book: Lines: A Brief History by Tim Ingold
- Blog: Compression is prediction by Annie Sexton
- Podcast: Vol.127 足球是方言:一期一会,告别墨西哥 - 鹰眼时间
- Paper: An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age