A distributed, highly available large-scale microservices blog system, which covers a wide range of popular technology stacks and personally written components, featuring note publishing, likes, favorites, following, and more. The platform is designed to handle massive user traffic with high-concurrency read/write and strong data consistency, ensuring real-time user interactions. High availability and scalability are achieved through a distributed microservices architecture.
- Microservices Distributed Architecture
- Maven Multi-Module
- Spring Boot Starter Components
- Frontend-Backend Separation
- Spring Cloud Alibaba
- Nacos
- Gateway
- Feign
- Mybatis
- MySQL
- Redis
- RocketMQ
- SaToken
- Minio
- Cassandra
- Zookeeper
- Led requirements analysis, technology selection, architecture design, project setup, API and database schema design.
- Built a microservices architecture from scratch, ensuring high scalability, availability, and performance to support massive user operations.
- Led the modularization of common functionalities to improve code reusability and maintainability.
- Developed user context components, API aspect logging, Jackson auto-configuration, global exception handling, custom parameter validation annotations, response utilities, etc.
- Split services by business domain and responsibility boundaries: gateway, authentication, user, note, user relationship, counter, OSS storage, distributed ID, KV short text storage, etc.
- Ensured each microservice has a single responsibility, low coupling, and supports independent deployment and horizontal scaling.
- Built and developed the Gateway service for unified routing and load balancing.
- Implemented unified authentication with SaToken; global filters parse tokens and pass user IDs downstream.
- Built authentication service based on RBAC, using SaToken for JWT login (account/password & SMS), logout, self-registration, password change.
- Integrated Aliyun SMS, custom thread pool for async SMS sending, and anti-abuse for SMS APIs.
- Integrated Meituan Leaf for distributed ID generation, providing both segment and Snowflake ID modes via Feign.
- JMeter tested: 22,000+ QPS per node, 4ms average latency, 1.9+ billion IDs/day.
- Built object storage for images/videos using factory + strategy patterns for extensibility (Aliyun OSS, Minio).
- Used Nacos for distributed config and dynamic registration of file strategy beans.
- Built KV storage service on Apache Cassandra (RocksDB engine) for short text (notes, comments).
- Designed and implemented user service for profile management, registration, query, password update (BCrypt with random salt).
- Built Redis + Caffeine two-level cache for high-concurrency reads, preventing cache avalanche, penetration, and breakdown.
- Designed and implemented note service for publishing, editing, querying, pinning, permission changes.
- Used CompletableFuture for concurrent downstream calls to reduce response time.
- Two-level cache for high-concurrency note detail queries.
- Used RocketMQ for cache invalidation on update/delete to ensure data consistency.
- High-concurrency like: Redis Bloom filter (later Roaring Bitmap), Redis ZSET + MQ async persistence, RateLimiter for traffic shaping.
- Designed and implemented user relationship service for follow/unfollow with high-concurrency writes.
- Used Redis cache + ordered MQ for async DB writes, Lua scripts for atomicity, and unique index for idempotency.
- Follower/following list queries use Redis ZSET with different caching strategies for high-concurrency paginated reads.
- Designed and implemented high-concurrency counter service for user follows, fans, likes, etc.
- Consumed MQ user actions, used BufferTrigger for batch aggregation (e.g., 1000 per batch, 1s interval), wrote to Redis then async to DB.
- Designed and implemented compensation-based data alignment for eventual consistency.
- Used XXL-Job for scheduled tasks, pre-created daily increment tables, consumed MQ for daily changes, and sharded broadcast tasks for efficient large-scale processing.
- Designed note/user indexes, built full indexes with logstash.
- Used Canal to listen to MySQL, custom Binlog event handling for real-time Elasticsearch index updates.
- Developed search APIs with Chinese word segmentation, keyword highlighting, dimension filtering, and custom scoring.
- Table design with redundant fields for efficient queries (e.g., earliest reply ID, sub-comment count).
- MQ producer reliability: Spring Retry, fallback DB write, async failed message handling.
- Efficient async comment publishing, triggers for hotness and count updates.
- Batch MQ consumption and DB writes for high-throughput.
- Paginated comment queries with Redis/local two-level cache.
- Like/unlike/delete for comments: MQ async, batch consumption/writes, in-memory merge for high-performance.
These instructions will help you set up and run the InstaVerse project on your local machine for development and testing purposes.
- Java 17+
- Maven 3.6+
- Git
- Docker (optional, for running services like MySQL, Redis)
-
Clone the repository:
git clone https://github.com/your-org/insta.git cd Insta -
Build all modules:
mvn clean install -DskipTests
Each microservice is located in its respective module folder. To start a service:
cd <module-folder>/src/main
mvn spring-boot:runFor a full development environment, consider using Docker Compose to start dependencies (e.g., MySQL, Redis, Elasticsearch).
Run unit and integration tests with:
mvn testContributions are welcome! Please follow these steps:
- Fork the repository.
- Create a feature branch:
git checkout -b feature/your-feature - Commit your changes:
git commit -m 'feat: add your feature' - Push to the branch:
git push origin feature/your-feature - Open a Pull Request.
This project is licensed under the MIT License - see the LICENSE file for details.