feat: core/intelligence/llm_client.py is a single-provider wrapper with no automatic failover — OpenAI outages or rate limits halt Execra sessions despite Gemini and Llama 3 being supported.

[divyanshim27]

💡 Problem Statement

The tech stack table lists three LLM options:

OpenAI GPT-4o / Gemini 1.5 Pro / Llama 3

The README's .env.example has fields for OPENAI_API_KEY. However, core/intelligence/llm_client.py appears to be a single-provider wrapper that calls only the configured provider. When that provider returns a RateLimitError or APIStatusError (OpenAI incidents are not rare — they had 14 outages in 2024), the entire guidance pipeline fails and Execra stops providing assistance mid-session — exactly when users need it most.

There is no:

• Fallback to a secondary provider
• Retry with exponential backoff
• Graceful degradation to a rule-only mode

Proposed Fix

Implement a provider cascade with automatic fallback:

# core/intelligence/llm_client.py
import asyncio
import time
from enum import Enum
from typing import Optional
import openai
import google.generativeai as genai

class LLMProvider(Enum):
    OPENAI = "openai"
    GEMINI = "gemini"
    OLLAMA = "ollama"  # Local Llama 3 via Ollama

class LLMClient:
    def __init__(self, primary: LLMProvider, fallback_chain: list[LLMProvider]):
        self.primary = primary
        self.fallback_chain = fallback_chain
        self._provider_cooldowns: dict[LLMProvider, float] = {}
        self.COOLDOWN_SECONDS = 120  # Back off a provider for 2 min after failure

    async def analyze(self, prompt: str, image_b64: Optional[str] = None) -> str:
        providers = [self.primary] + self.fallback_chain

        for provider in providers:
            # Skip providers in cooldown
            cooldown_until = self._provider_cooldowns.get(provider, 0)
            if time.monotonic() < cooldown_until:
                continue

            try:
                result = await self._call_provider(provider, prompt, image_b64)
                return result
            except (openai.RateLimitError, openai.APIStatusError) as e:
                print(f"[LLMClient] {provider.value} failed: {e}. Trying next provider.")
                self._provider_cooldowns[provider] = time.monotonic() + self.COOLDOWN_SECONDS
            except Exception as e:
                print(f"[LLMClient] {provider.value} unexpected error: {e}. Trying next provider.")
                self._provider_cooldowns[provider] = time.monotonic() + self.COOLDOWN_SECONDS

        # All providers failed — fall back to rule-based guidance only
        return self._rule_based_fallback(prompt)

    async def _call_provider(self, provider: LLMProvider, prompt: str, image_b64: Optional[str]) -> str:
        if provider == LLMProvider.OPENAI:
            return await self._call_openai(prompt, image_b64)
        elif provider == LLMProvider.GEMINI:
            return await self._call_gemini(prompt, image_b64)
        elif provider == LLMProvider.OLLAMA:
            return await self._call_ollama(prompt)  # No vision support in local mode
        raise ValueError(f"Unknown provider: {provider}")

    def _rule_based_fallback(self, prompt: str) -> str:
        return (
            "⚠️ AI guidance temporarily unavailable (all providers rate-limited or down). "
            "Rule-based mode active: please proceed carefully and refer to documentation."
        )

Configure the cascade via .env:

# .env
EXECRA_PRIMARY_LLM=openai
EXECRA_FALLBACK_LLMS=gemini,ollama
OPENAI_API_KEY=...
GOOGLE_API_KEY=...
OLLAMA_BASE_URL=http://localhost:11434

Files to Modify

File	Change
core/intelligence/llm_client.py	Rewrite as provider cascade with cooldowns and rule fallback
.env.example	Add EXECRA_PRIMARY_LLM, EXECRA_FALLBACK_LLMS, GOOGLE_API_KEY, OLLAMA_BASE_URL
requirements.txt	Ensure google-generativeai is listed alongside openai
docs/architecture.md	Document multi-provider fallback behavior

Suggested labels: enhancement, reliability, backend, ai

I would like to work on this. Could you please assign it to me?

[divyanshim27]

Hey @sahoo-tech ,
Please assign it to me under GSSoC 2026.
/claim /assign /claim-assignment