🌌 ANIDost — Advanced Anime Vault & Deep Personalization Hub

⚡ The Ultimate Full-Stack Ecosystem for Trackers & Otakus ⚡

---

📖 Project Overview & Engineering Introduction

Note

ARCHITECTURAL MISSION Modern media tracking platforms frequently suffer from discovery fatigue and sluggish data synchronization. Users are often forced to wade through generic, unpersonalized homepages, while tracking modifications (like moving an entry from a watch list to a completed archive) are bottlenecked by sequential database round-trips that introduce noticeable user-interface latency.

ANIDost solves these architectural flaws by introducing a high-performance, asynchronous anime tracking and discovery ecosystem. The core philosophy of the platform centers around providing an immediate, highly personalized user experience from the very first interaction. By decoupling the presentation layer from heavy cloud database transactions, ANIDost balances real-time performance with deep user personalization.

The client application features a responsive, dark glassmorphic interface built to optimize screen real estate. It features global analytics trackers, interactive sliders, custom carousels, and independent state-driven dashboards tailored to both anonymous visitors and authenticated power-users.

---

🧠 Architectural Theory & System Deep-Dive

To support thousands of simultaneous actions without degrading platform speed, ANIDost implements a Dual-Layer Caching Topology paired with a scalable data persistence layer:

🔷 1. Client-Side State & Cache Hydration (TanStack Query)

To eradicate redundant network overhead and eliminate jarring loading spinners, the frontend utilizes TanStack Query (React Query) as its asynchronous state-management engine.

• Stale-While-Revalidate (SWR): TanStack Query intelligently caches API responses (such as the watchlist or recommendation arrays) locally. When a user navigates between views, cached data is rendered instantly, while a background fetch ensures the UI silently updates if server data has changed.
• Optimistic UI Mutation Machine: When a user clicks Mark as Completed, TanStack Query immediately updates the client cache directly, instantly incrementing the Completed dashboard card counter and updating the tracking button state (O(1) perceived latency).

Important

TRANSACTIONAL FAULT ROLLBACKS If the underlying Axios HTTP request to the FastAPI server fails, TanStack Query catches the exception, fires a targeted visual toast notification (Network error. Reverted changes.), and automatically rolls the client cache back to its exact previous source-of-truth state.

🔷 2. High-Speed Custom Search Engine & Memory Caching (Upstash Serverless Redis)

Traditional database text queries rely on heavy relational operations like SQL LIKE statements, which force a sequential table scan across thousands of database rows, leading to severe latency bottlenecks. To resolve this, ANIDost implements a proprietary, high-speed custom search engine paired with Redis-backed caching.

• In-Memory Prefix Search Indexing: On application startup, the FastAPI backend processes raw catalog data, mapping 15,850+ unique anime titles into 54,953 high-speed string search prefixes for instant autocomplete resolution.
• Custom Multi-Tier Search Engine:
  • Exact Matching: Highest weight assigned to absolute title matches (match_score = 1000).
  • Prefix Matches: Matches strings starting with the query using the precomputed prefix index (match_score = 800).
  • Substring Contains: Matches titles that contain the query anywhere within the string (match_score = 500).
  • Fuzzy Fallback Search: When standard lookups yield fewer than 30 matches, the system triggers a fuzzy fallback using the rapidfuzz library's Levenshtein-based partial_ratio algorithm (thresholding at $\ge 75%$ similarity). This catches typos, phonetic spellings, and minor omissions, mapping the similarity percentage directly to the match score.
  • Ranking Strategy: Results are dynamically sorted by their match_score descending, then by global popularity (members) descending, and finally by numerical review score descending, presenting the most relevant and high-quality results first.
• Serverless Redis: All cached search data and autocomplete arrays are stored in a managed Upstash Serverless Redis instance, providing sub-millisecond read/write performance.
• Optimized Caching Pipelines: Repeated search terms and autocomplete queries are served directly from Redis, shifting lookup latencies from database-bound operations to single-digit milliseconds.
• Global Feed Caching: Heavy aggregations, such as the Trending Now dashboard slider, are processed intermittently and cached in Redis, preventing repetitive calculation loops on every single user request.

🔷 3. Persistent Core Storage & Auth Engine (Supabase Cloud PostgreSQL)

While TanStack Query and Redis govern transient speed, Supabase serves as the immutable relational foundation of the platform.

• Relational Database Integrity: Running a cloud-hosted PostgreSQL instance, Supabase safely stores persistent user schema records, active watchlist tables, multi-dimensional genre preference vectors, and verified history tables.
• Frictionless Social Provisioning: The backend maps native frontend Google OAuth tokens directly into the Supabase security layer, allowing instant user profile creation and session verification without writing custom database bridges.

🔷 4. Hybrid Recommendation Engine & ML Pipelines

The Forged for You Engine combines two distinct recommendation strategies (Content-Based and Collaborative Filtering) to produce personalized recommendation arrays, using a fallback topology to handle cold starts or out-of-model items.

👥 A. Collaborative Filtering Recommender

• Methodology: Item-to-Item Collaborative Filtering using user rating behaviors to calculate item similarities.
• Matrix Construction: Pivots user ratings into a sparse User-Item matrix, transposing it to calculate pairwise Cosine Similarity among popular titles. This allows the system to identify similar items based on global co-rating behavior (item_similarity.npy).
• Data Source & Preprocessing (rating.csv):
  • Invalid Filter: Removes missing/non-rated entries (rating = -1).
  • High-Activity Thresholding: Filters out anime with fewer than 12,000 global ratings to handle sparsity and focus computational overhead on statistically significant entries, producing cleaned_ratings.csv.

📝 B. Content-Based Recommender

• Methodology: TF-IDF Semantic Text Mining coupled with Multi-Attribute Cosine Similarity.
• Feature Vectors: Combines:
  • Genre Vectors: Binary CountVectorizer on genre attributes.
  • Synopsis Vectors: TfidfVectorizer (with English stop-words, $N$-grams from 1 to 3, and a 20,000 vocabulary limit) on cleaned plots/synopses.
• Weighted Similarity Fusion: Computes Cosine Similarity matrices for both genre and synopsis features separately. The final recommendation score is compiled as a weighted fusion: $70%$ Genre Similarity + $30%$ Synopsis Similarity.
• Dynamic Boosting: Applies a $1.3\times$ boost for action/shounen genres and a $1.5\times$ boost if candidates share $4+$ genre intersections with the query anime.
• Data Source & Preprocessing (animes.csv):
  • Null Imputation: Fills blank genres/synopses; drops entries missing a numerical rating score.
  • Franchise Cleanup: Strips title suffixes (movie, special, ova, season, etc.) and non-alphanumeric characters to deduplicate franchise derivatives.
  • Ad-Filtering: Drops placeholder listings or lines containing generic text ("h for genre", "h for synopsis"), saving the final pipeline input to cleaned_animes.csv.

🔀 C. Hybrid Fusion & Fallback Topology

• Recommendation Blend: Given a target anime uid, the engine fetches recommendations from both engines. If both are available, a weighted hybrid score is computed: [\text{Final Score} = 0.55 \times \text{Content Score} + 0.45 \times \text{Collaborative Score}]
• Robust Fallback Handling: If the item does not exist in the Collaborative Filtering subset (due to the 12,000 ratings density threshold), the engine smoothly falls back to $100%$ Content-Based recommendations, ensuring a seamless user experience without query failures.
• Onboarding Seed Pipeline: Guest profiles and new users undergo a 3-genre affinity selection to seed immediate recommendations before a rich tracking history is established.

🔷 5. Session Isolation & Dual-Token Security

• Short-Lived Access Tokens: Authenticated member session states are isolated using short-lived Access Tokens (30-minute expiration boundaries) to minimize hijack windows.
• Secure Refresh Loops: Long-lived Refresh Tokens are processed securely via backend loops to silently rotate credentials without user disruption.
• SMTP-Driven Register Verification: Traditional registration pathways use an automatic backend SMTP configuration layout to securely deliver random OTP (One-Time Password) confirmation codes via email, validating user accounts instantly.

---

📁 Project Directory Structure

ML_pro_clean/
├── .gitignore
├── README.md
├── backend
│   ├── .dockerignore
│   ├── .env
│   ├── .env.example
│   ├── Dockerfile
│   ├── app
│   │   ├── __init__.py
│   │   ├── auth
│   │   │   ├── __init__.py
│   │   │   ├── google_oauth.py
│   │   │   ├── jwt_handler.py
│   │   │   └── password.py
│   │   ├── config.py
│   │   ├── core
│   │   │   └── redis_client.py
│   │   ├── database
│   │   │   ├── __init__.py
│   │   │   ├── connection.py
│   │   │   └── models.py
│   │   ├── dependencies.py
│   │   ├── limiter.py
│   │   ├── main.py
│   │   ├── models
│   │   │   ├── Cleaning.py
│   │   │   ├── processed
│   │   │   │   ├── cleaned_animes.csv
│   │   │   │   └── cleaned_ratings.csv
│   │   │   ├── raw
│   │   │   │   ├── animes.csv
│   │   │   │   └── rating.csv
│   │   │   ├── recommenders
│   │   │   │   ├── collaborative_recommender.py
│   │   │   │   ├── content_recommender.py
│   │   │   │   ├── hybrid_recommender.py
│   │   │   │   ├── shared_data.py
│   │   │   │   ├── test_hybrid_run.py
│   │   │   │   └── verify_hybrid.py
│   │   │   ├── saved_models
│   │   │   │   ├── anime_data.pkl
│   │   │   │   ├── cf_mappings.pkl
│   │   │   │   ├── genre_matrix.pkl
│   │   │   │   ├── genre_tfidf.pkl
│   │   │   │   ├── genre_vectorizer.pkl
│   │   │   │   ├── item_similarity.npy
│   │   │   │   ├── synopsis_matrix.pkl
│   │   │   │   └── synopsis_tfidf.pkl
│   │   │   ├── training
│   │   │   │   ├── CollaborativeFTrain.py
│   │   │   │   └── ContentFTrain.py
│   │   │   └── watchlist.py
│   │   ├── routes
│   │   │   ├── __init__.py
│   │   │   ├── anime.py
│   │   │   ├── auth_routes.py
│   │   │   ├── cold_start_routes.py
│   │   │   ├── public_routes.py
│   │   │   ├── recommender_routes.py
│   │   │   ├── search_routes.py
│   │   │   └── watchlist_routes.py
│   │   ├── schemas
│   │   │   ├── __init__.py
│   │   │   ├── cold_start_schemas.py
│   │   │   ├── user_schemas.py
│   │   │   └── watchlist_schema.py
│   │   ├── services
│   │   │   ├── __init__.py
│   │   │   ├── auth_service.py
│   │   │   ├── cache_service.py
│   │   │   ├── email_service.py
│   │   │   ├── jikan_service.py
│   │   │   ├── recommender_service.py
│   │   │   ├── search_service.py
│   │   │   └── watchlist_service.py
│   │   ├── tests
│   │   │   ├── __init__.py
│   │   │   └── redis_test.py
│   │   └── utils
│   │       ├── __init__.py
│   │       ├── search_index.py
│   │       └── search_utils.py
│   ├── requirements.txt
│   └── upgrade.py
├── docker-compose.yml
├── frontend
│   ├── .dockerignore
│   ├── .env
│   ├── .env.example
│   ├── .env.production
│   ├── Dockerfile
│   ├── index.html
│   ├── nginx.conf
│   ├── package-lock.json
│   ├── package.json
│   ├── postcss.config.js
│   ├── public
│   │   ├── epic-bg.jpg
│   │   └── favicon.svg
│   ├── src
│   │   ├── App.jsx
│   │   ├── assets
│   │   ├── components
│   │   │   ├── AnimeCard.jsx
│   │   │   ├── AnimeGrid.jsx
│   │   │   ├── AnimeHeader.jsx
│   │   │   ├── AnimeInfo.jsx
│   │   │   ├── AnimePosterCard.jsx
│   │   │   ├── Footer.jsx
│   │   │   ├── GenreBadge.jsx
│   │   │   ├── Navbar.jsx
│   │   │   ├── OnboardingRoute.jsx
│   │   │   ├── PageShell.jsx
│   │   │   ├── ProtectedRoute.jsx
│   │   │   ├── Skeleton.jsx
│   │   │   └── VideoPlayer.jsx
│   │   ├── context
│   │   │   └── AuthContext.jsx
│   │   ├── hooks
│   │   │   ├── useColdStartMutations.js
│   │   │   ├── useColdStartPreferences.js
│   │   │   ├── useCombinedRecommendations.js
│   │   │   ├── useDebounce.js
│   │   │   ├── useForgedList.js
│   │   │   ├── useRecommendations.js
│   │   │   ├── useTopRated.js
│   │   │   ├── useTrending.js
│   │   │   ├── useWatchlist.js
│   │   │   └── useWatchlistMutation.js
│   │   ├── index.css
│   │   ├── main.jsx
│   │   ├── pages
│   │   │   ├── AboutUs.jsx
│   │   │   ├── AlchemistPage.jsx
│   │   │   ├── AnimeDetailsPage.jsx
│   │   │   ├── Dashboard.jsx
│   │   │   ├── ForgedPage.jsx
│   │   │   ├── GuestAnimeSelection.jsx
│   │   │   ├── GuestOnboarding.jsx
│   │   │   ├── InteractiveHome.jsx
│   │   │   ├── Login.jsx
│   │   │   ├── RecommendationsPage.jsx
│   │   │   ├── SearchPage.jsx
│   │   │   ├── Signup.jsx
│   │   │   └── WatchlistPage.jsx
│   │   ├── services
│   │   │   ├── animeService.js
│   │   │   ├── api.js
│   │   │   ├── auth.js
│   │   │   ├── coldStartService.js
│   │   │   ├── recommendationService.js
│   │   │   ├── searchService.js
│   │   │   └── watchlistService.js
│   │   └── styles
│   │       └── theme.css
│   ├── tailwind.config.js
│   ├── vercel.json
│   └── vite.config.js
└── render.yaml

🔌 Unified API Routing Interface

🔐 Authentication, Validation, & Guest Control
🟢 POST /auth/signup
Description: Provisions profile records and dispatches transaction verification OTP.
Security Context: Public (None).

🟢 POST /auth/otp/request
Description: Requests a new verification OTP.
Security Context: Public (None).

🟢 POST /auth/otp/verify
Description: Validates target register code token to activate account boundaries.
Security Context: Public (None).

🟢 POST /auth/login
Description: Traditional credential handshake; issues standard secure dual-token pairings.
Security Context: Access + Refresh Tokens.

🟢 POST /auth/google
Description: Validates incoming Google social authentication payloads and provisions credentials.
Security Context: Access + Refresh Tokens.

🟢 POST /auth/refresh
Description: Seamlessly refreshes expired access tokens using valid persistent refresh tokens.
Security Context: Requires Refresh Token.

🟢 POST /auth/guest
Description: Issues transient guest session tokens for unauthenticated exploratory users.
Security Context: Public (None).

🔵 GET /auth/me
Description: Hydrates frontend application layouts with verified active profile records.
Security Context: Requires valid Access Token.

🟢 POST /auth/logout
Description: Terminates user session and completely invalidates active access/refresh tokens.
Security Context: Requires valid Access Token.

🔵 GET /auth/status
Description: Lightweight check for current authentication state.
Security Context: Requires valid Access Token.

🍿 Discovery, Onboarding, & Engine Channels
🟢 POST /api/v1/cold-start/preferences
Description: Maps 3 structural genres to build an initial interaction weights user profile.
Security Context: Guest Context or Access Token.

🔵 GET /api/v1/cold-start/preferences
Description: Retrieves user's existing cold-start genre preference schema.
Security Context: Guest Context or Access Token.

🟡 PUT /api/v1/cold-start/preferences
Description: Modifies and updates the structural onboarding parameters.
Security Context: Guest Context or Access Token.

🔴 DELETE /api/v1/cold-start/preferences
Description: Clears user's active cold-start matrix configuration.
Security Context: Guest Context or Access Token.

🔵 GET /api/v1/public/trending
Description: Pulls global index layouts optimized via general system tracking activity weights.
Security Context: Public / Guest Context.

🔵 GET /api/v1/public/top-rated
Description: Retrieves the highest-rated global anime titles ranked by averaged historical user scores.
Security Context: Public / Guest Context.

🔵 GET /api/v1/public/forged-for-you
Description: Generates personalized content streams matching dynamically assigned algorithmic weights.
Security Context: Requires valid Access Token or Guest Context.

🔵 GET /recommend/
Description: Streams hybrid spotlight content compiled using verified global reviewer metadata.
Security Context: Open / Public.

🗄️ Vault Table Control
🔵 GET /api/watchlist/{user_id}
Description: Gathers the active user dataset matching personal vault tracking profiles.
Security Context: Requires valid Access Token.

🔵 GET /api/watchlist/{user_id}/latest
Description: Retrieves the most recently active modifications in a specific tracking matrix.
Security Context: Requires valid Access Token.

🟢 POST /api/watchlist/add
Description: Modifies and toggles active item states between Plan to Watch and Completed schemas.
Security Context: Requires valid Access Token.

🔴 DELETE /api/watchlist/{user_id}/{anime_id}
Description: Discards a target entry completely from the user's relational tracking matrix rows.
Security Context: Requires valid Access Token.

🔍 Search & Metadata Execution
🔵 GET /search/
Description: Fast text-index search against the unified anime catalog structure.
Security Context: Public (None).

🔵 GET /search/autocomplete
Description: Prefix-based high-speed autocomplete resolution for navigation dropdowns.
Security Context: Public (None).

🔵 GET /anime/{mal_id}
Description: Fetches robust multi-node metadata configurations for a specific database ID.
Security Context: Public (None).

🛠️ Infrastructure Overview
[!CAUTION]
PRODUCTION ENGINE MAP

Framework Core: FastAPI (Asynchronous Python ASGI Web Hub)

Compilation Engine: React 18 bundled via ultra-low latency Vite infrastructure

Client-Side Cache Engine: TanStack Query (Query client runtime & mutation cache manager)

Relational Backend Database: Supabase Cloud PostgreSQL Cluster

High-Speed Cache Layer: Upstash Serverless Redis (Prefix index execution framework)

Messaging Node: Automated SMTP Gmail App Integration pipeline (OTP infrastructure)