SkinSight

SkinSight is an advanced, AI-powered multimodal clinical diagnostic platform engineered to support proactive dermatological screening. Developed exclusively during MacHacks 2026, SkinSight addresses a critical bottleneck in global healthcare: democratizing preliminary skin lesion analysis through high-speed, secure, and mathematically interpretable artificial intelligence.

MacHacks 2026 Submission

Target Tracks:

• Best Overall Hack
• Best Use of Google Gemini API

The Clinical Objective

Skin cancer remains one of the most prevalent and lethal cancers globally, yet early detection dramatically increases survival rates. However, access to specialized dermatologists is fundamentally constrained by slow global triage networks. Furthermore, traditional diagnostic AI models operate as impenetrable "black boxes"—providing predictions without clinical rationale or spatial interpretation. SkinSight bridges this gap by unifying state-of-the-art vision models with complex LLM reasoning to produce transparent, highly interpretable diagnostic reports.

System Capabilities

SkinSight empowers users and healthcare providers to securely upload dermoscopic images and instantly receive a comprehensive clinical diagnostic profile:

1. Primary Detection & Confidence Matrices: Analyzes the lesion against 7 major clinical dermatological classes utilizing a highly optimized, fine-tuned Vision Transformer (ViT).
2. Interpretability via Grad-CAM Geometry: Automatically generates a dynamic, mathematically rigorous heatmap mapping the exact pixel boundaries and topological textures the AI analyzed to finalize its prediction—completely eliminating the clinical "black box" effect.
3. Automated Clinical Insights: Integrates tightly with the Google Gemini 1.5 Flash API to generate deterministic, secure, and highly contextualized clinical explanations. By parsing the raw classification tensors and confidence arrays, Gemini dynamically synthesizes plain-English physiological insights and exact urgency recommendations.

Technical Architecture & Implementation

SkinSight's architecture represents a true synthesis of modern web engineering and bleeding-edge machine learning integration.

Advanced Machine Learning Pipeline

• Base Architecture: Engineered upon the google/vit-base-patch16-224 Vision Transformer (ViT) architecture.
• Dynamic Fine-Tuning (PEFT/LoRA): The model was fine-tuned across thousands of complex skin lesion occurrences utilizing Low-Rank Adaptation (LoRA). This ensured extremely high biological pattern recognition while strategically compressing the computational payload for rapid serverless-style execution.
• Custom Grad-CAM Mathematical Adapters: Standard spatial convolutions do not exist in Vision Transformers. To achieve true diagnostic transparency, we engineered a custom pytorch-grad-cam intersection adapter to mathematically reshape the 196 flat sequence tokens back into a 2D grid. By explicitly targeting the deepest layernorm_before block, the system successfully traces topological disease boundaries natively from the ViT attention layers.

High-Performance Backend Infrastructure

• FastAPI: Scaled via a highly concurrent, asynchronous Python backend architecture.
• Gemini API Translation Layer: We heavily leveraged the Google Gemini 1.5 Flash LLM as an intelligent translation matrix. By intercepting the Vision Transformer's raw output arrays and piping them into an engineered Gemini prompt, the LLM acts as an autonomous clinical consultant.
• Zero-Trust Security: Architected under a "Private by Design" protocol. Patient images are executed securely inside volatile memory limits (io.BytesIO) without ever touching persistent read-write server storage.

Enterprise-Grade Frontend Engineering

• React & Vite: Constructed as a blazing-fast Single-Page Application (SPA) leveraging React strict-mode constraints and modular state mapping.
• Responsive Clinical UI/UX: Styled via utility-first Tailwind CSS. We explicitly engineered modern responsive design architectures, CSS Grid fluid layouts, and embedded custom typography (Newsreader & Plus Jakarta Sans) to establish a rigorous, highly professional medical platform aesthetic.

Local Execution Guide

Clone the Repository:

git clone https://github.com/Anasthecode/SkinSight.git
cd SkinSight

1. Initialize the FastAPI Inference Backend:

cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Create a .env file mimicking .env.example with your authentic GEMINI_API_KEY
uvicorn app.main:app --reload --port 8000

2. Initialize the React SPA Frontend:

cd frontend/skinsight
npm install
npm run dev

Navigate to http://localhost:9119 to interface immediately with the intelligence platform.

Lead Engineers

• Aryan Verma
• Shan Truong
• Jerry Wu
• Anas Abdul aal

Engineered exclusively for MacHacks 2026.