Add travel-concierge example (cloud catalog + live session)

moss-live-labs/examples/travel-concierge/.env.example

@@ -0,0 +1,14 @@
+# LiveKit (local dev defaults — keep as-is)
+LIVEKIT_URL=ws://localhost:7880
+LIVEKIT_API_KEY=devkey
+LIVEKIT_API_SECRET=secret
+
+# Moss
+MOSS_PROJECT_ID=Your moss project id here
+MOSS_PROJECT_KEY=Your moss project key here
+TRAVEL_CATALOG_INDEX=demo-travel-catalog
+
+# Providers (OpenAI = LLM, Deepgram = STT, Cartesia = TTS)
+OPENAI_API_KEY=Your openai api key here
+DEEPGRAM_API_KEY=Your deepgram api key here
+CARTESIA_API_KEY=Your cartesia api key here

moss-live-labs/examples/travel-concierge/.python-version

@@ -0,0 +1 @@
+3.14

moss-live-labs/examples/travel-concierge/DEMO_SCRIPT.md

@@ -0,0 +1,55 @@
+# Demo Script — Travel Concierge (cloud catalog + live session)
+
+~90s. Shows Moss answering from **two indexes in one call**: a pre-loaded catalog and a
+live session that remembers what you say. Watch both panels on the right.
+
+## Before you record
+- [ ] `python seed_index.py` (seeds the catalog) · `python agent.py dev` · `livekit-server --dev` · web at localhost:3000
+
+## 1 · Frame it (0:00–0:12)
+> "This concierge knows a catalog of trips — that's loaded ahead of time. But it also
+> remembers everything I say on the call. Two Moss indexes, live, side by side. Watch."
+
+**[Click Start planning. The agent greets you.]**
+
+## 2 · Tell it about the trip (0:12–0:40)
+> "We're a family of four, our budget's around two thousand five hundred dollars a person,
+> we love beaches, and we want to travel the first week of December."
+
+**[Point at the "This call · live session" panel filling up as you talk.]**
+> "Watch the live session. It's pulling the facts out of what I say — a family of four,
+> the budget, the dates — and remembering each one. In memory, in milliseconds."
+
+## 3 · Recall (0:40–0:58)
+> "Wait, what did I say my budget was?"
+
+**[The session panel lights up; the agent answers from what you said.]**
+> "It pulled that straight from this conversation — not the catalog. And notice the question
+> itself doesn't get stored, only the facts do."
+
+## 4 · Recommend (0:58–1:20)
+> "So where should we go?"
+
+**[Both panels light: catalog hits + your session prefs.]**
+> "Now it's using both — my preferences from the session *and* the catalog it already had —
+> to recommend somewhere that actually fits. Beach, family, December, in budget."
+
+## 5 · Why it matters (1:20–1:35)
+> "That's long-term knowledge and short-term memory in the same call — one pre-loaded index,
+> one live session, both queried in milliseconds, on-device. Open source at
+> github.com/usemoss/moss."
+
+---
+
+## Say-these preferences (each is distilled to a fact in the session)
+- "Family of four, budget about $2,500 per person."
+- "We love beaches and warm weather."
+- "Traveling the first week of December."
+
+> Only facts land in the session. Questions like "what did I say my budget was?" and
+> "where should we go?" are recalled against, but never stored.
+
+## Recall / recommend prompts
+- "What did I say my budget was?" → session
+- "Which destinations do you have?" → catalog
+- "Where should we go?" → both (expect Tulum or Costa Rica for beach + family + Dec)

moss-live-labs/examples/travel-concierge/README.md

@@ -0,0 +1,55 @@
+# Travel Concierge — pre-loaded catalog + live session
+
+A voice travel concierge that answers from **two Moss indexes at once**:
+
+- a **pre-loaded catalog** (long-term, shared across every call)
+- a **live session** that captures what you say on *this* call (short-term, in-memory)
+
+Each turn it recalls your stated preferences (session) and recommends trips (catalog). The
+web UI shows both indexes side by side, lighting up per turn.
+
+```
+Browser (web/)  ⟷  LiveKit room  ⟷  agent.py (STT → LLM → TTS)  ⟷  Moss (catalog + session)
+```
+
+## What you need
+- A [Moss](https://moss.dev) account · [LiveKit](https://livekit.io) (local) · OpenAI (LLM),
+  Deepgram (STT), Cartesia (TTS) keys · Python 3.14+ (`uv`) and Node 18.18+.
+
+## Setup
+```bash
+uv sync
+cp .env.example .env          # fill in Moss + provider keys
+python agent.py download-files
+```
+
+## 1. Seed the catalog (the cloud index)
+```bash
+python seed_index.py
+```
+The live session is built at runtime by the agent — nothing to seed there.
+
+## 2. Run (three terminals)
+```bash
+livekit-server --dev
+python agent.py dev
+cd web && npm install && cp .env.local.example .env.local && npm run dev   # localhost:3000
+```
+
+Click **Start planning**, then talk: tell it your budget, dates, and who's coming, ask it
+to recall them, and ask for a recommendation.
+
+## How it works
+Per turn, `agent.py`:
+1. queries the **live session** (recall what you've said),
+2. queries the **pre-loaded catalog** (matching trips),
+3. injects both into the model, then
+4. **distills your turn into facts and stores only those** in the session, so later turns
+   recall clean preferences — not questions or filler.
+
+Both result sets are published on the `moss.retrieval` data channel; the UI renders
+**Catalog (cloud)** and **This call (session)**. See [`DEMO_SCRIPT.md`](./DEMO_SCRIPT.md).
+
+## Resources
+- [Docs — Sessions](https://docs.moss.dev/docs/integrate/sessions)
+- [GitHub](https://github.com/usemoss/moss)

moss-live-labs/examples/travel-concierge/agent.py

@@ -0,0 +1,216 @@
+import asyncio
+import json
+import logging
+import os
+import time
+import uuid
+from datetime import datetime
+
+from dotenv import load_dotenv
+from openai import AsyncOpenAI
+from livekit import rtc
+from livekit.plugins import openai, deepgram, silero, cartesia
+from livekit.plugins.turn_detector.multilingual import MultilingualModel
+from livekit.agents import (
+    JobContext,
+    WorkerOptions,
+    cli,
+    ChatContext,
+    ChatMessage,
+    Agent,
+    AgentSession,
+)
+
+from moss import MossClient, DocumentInfo, QueryOptions
+
+load_dotenv()
+
+MOSS_PROJECT_ID = os.getenv("MOSS_PROJECT_ID")
+MOSS_PROJECT_KEY = os.getenv("MOSS_PROJECT_KEY")
+# Long-term, pre-loaded knowledge shared across every call.
+CATALOG_INDEX = os.getenv("TRAVEL_CATALOG_INDEX", "demo-travel-catalog")
+
+# Turns the traveler's raw speech into clean, standalone facts before we store them.
+# Questions, recall requests, and small talk yield no facts, so they never hit the session.
+FACT_EXTRACT_PROMPT = """You pull durable traveler preferences out of one thing the traveler just said on a trip-planning call.
+
+Return JSON: {"facts": ["...", "..."]}.
+
+A fact is a short, standalone statement of something true about the traveler or their trip:
+party size, budget, dates, interests, must-haves, or destinations they like or dislike.
+
+Rules:
+- Only include preferences actually stated in this utterance.
+- Split multiple preferences into separate facts.
+- Drop filler and normalize (e.g. "our budget's around, uh, twenty five hundred a person" -> "Budget is about $2,500 per person").
+- Keep each fact under about 8 words.
+- Return {"facts": []} for questions, recall requests, or small talk (e.g. "what did I say my budget was?", "so where should we go?")."""
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger("moss-travel")
+
+
+def _docs(result):
+    return [
+        {"id": getattr(d, "id", None), "text": d.text, "score": float(getattr(d, "score", 0.0))}
+        for d in (result.docs if result and result.docs else [])
+    ]
+
+
+class TravelConciergeAgent(Agent):
+    """Answers from two Moss indexes at once: a pre-loaded catalog (long-term)
+    and a live session that captures what the traveler says on THIS call."""
+
+    def __init__(self, moss_client: MossClient, session_index, room: rtc.Room):
+        super().__init__(
+            instructions="""
+                You are a warm, upbeat travel concierge on a voice call with one traveler.
+                Each turn you're given two kinds of context:
+                  1. Trip options from our catalog.
+                  2. What the traveler has told you earlier in THIS call (their preferences).
+                Use both: remember what they've said, and recommend trips from the catalog
+                that fit. If they ask you to recall something they mentioned, answer from the
+                facts in that context. Keep replies short and natural for voice. Never mention
+                indexes, sessions, catalogs, or how you look things up.
+            """
+        )
+        self.moss = moss_client
+        self.session_index = session_index
+        self.room = room
+        self.turn = 0
+        # background task that stores the current turn's facts; awaited next turn
+        self._pending_remember = None
+        # Small, fast model used only to distill the traveler's speech into facts.
+        self._extractor = AsyncOpenAI()
+
+    async def _publish(self, query, catalog, session, catalog_ms, session_ms):
+        payload = {
+            "query": query,
+            "catalog": _docs(catalog),
+            "session": _docs(session),
+            "catalog_ms": round(catalog_ms, 2),
+            "session_ms": round(session_ms, 2),
+        }
+        try:
+            await self.room.local_participant.publish_data(
+                json.dumps(payload).encode("utf-8"), reliable=True, topic="moss.retrieval"
+            )
+        except Exception as e:
+            logger.warning(f"Failed to publish retrieval data: {e}")
+
+    async def on_user_turn_completed(self, turn_ctx: ChatContext, new_message: ChatMessage) -> None:
+        query = new_message.text_content
+        logger.info(f"Traveler: {query}")
+        try:
+            # 0. Make sure the previous turn's facts are stored before we recall,
+            #    so an immediate follow-up question sees them.
+            if self._pending_remember is not None:
+                await self._pending_remember
+                self._pending_remember = None
+
+            # 1. Recall prior turns from the live session (short-term memory).
+            t = time.perf_counter()
+            session_results = await self.session_index.query(query, QueryOptions(top_k=3))
+            session_ms = (time.perf_counter() - t) * 1000.0
+
+            # 2. Look up matching trips in the pre-loaded catalog (long-term knowledge).
+            t = time.perf_counter()
+            catalog_results = await self.moss.query(CATALOG_INDEX, query, QueryOptions(top_k=3))
+            catalog_ms = (time.perf_counter() - t) * 1000.0
+
+            # 3. Show both in the UI.
+            await self._publish(query, catalog_results, session_results, catalog_ms, session_ms)
+
+            # 4. Inject both into the model's context, clearly labeled.
+            blocks = []
+            if catalog_results.docs:
+                blocks.append("Trip options from our catalog:\n" + "\n".join(f"- {d.text}" for d in catalog_results.docs))
+            if session_results.docs:
+                blocks.append("Facts the traveler shared earlier in this call:\n" + "\n".join(f"- {d.text}" for d in session_results.docs))
+            if blocks:
+                turn_ctx.add_message(role="system", content="\n\n".join(blocks) + "\n\nUse this to help the traveler.")
+
+            # 5. Distill this turn into facts and store only those in the live session, in the
+            #    background so it never delays the reply. Awaited at the top of the next turn
+            #    (step 0) so recall always sees it. Questions/recall add nothing.
+            self._pending_remember = asyncio.create_task(self._remember_facts(query))
+        except Exception as e:
+            logger.error(f"Moss lookup failed: {e}", exc_info=True)
+
+        await super().on_user_turn_completed(turn_ctx, new_message)
+
+    async def _extract_facts(self, text: str) -> list[str]:
+        """Pull clean, standalone facts out of one traveler utterance. [] if it states none."""
+        try:
+            resp = await self._extractor.chat.completions.create(
+                model="gpt-4o-mini",
+                temperature=0,
+                response_format={"type": "json_object"},
+                messages=[
+                    {"role": "system", "content": FACT_EXTRACT_PROMPT},
+                    {"role": "user", "content": text},
+                ],
+            )
+            data = json.loads(resp.choices[0].message.content or "{}")
+            return [f.strip() for f in data.get("facts", []) if isinstance(f, str) and f.strip()]
+        except Exception as e:
+            logger.warning(f"Fact extraction failed: {e}")
+            return []
+
+    async def _remember_facts(self, text: str) -> None:
+        for fact in await self._extract_facts(text):
+            self.turn += 1
+            try:
+                await self.session_index.add_docs(
+                    [DocumentInfo(id=f"fact-{self.turn}", text=fact, metadata={"role": "traveler"})]
+                )
+                logger.info(f"Remembered: {fact}")
+            except Exception as e:
+                logger.warning(f"Failed to store fact: {e}")
+
+
+async def entrypoint(ctx: JobContext):
+    if not MOSS_PROJECT_ID or not MOSS_PROJECT_KEY:
+        raise SystemExit(
+            "Missing MOSS_PROJECT_ID / MOSS_PROJECT_KEY. Copy .env.example to .env and fill them in."
+        )
+    await ctx.connect()
+
+    client = MossClient(project_id=MOSS_PROJECT_ID, project_key=MOSS_PROJECT_KEY)
+
+    # Long-term: the pre-loaded catalog, shared across all calls. Fatal if missing
+    # so the worker doesn't keep taking calls with an empty catalog.
+    try:
+        await client.load_index(CATALOG_INDEX)
+        logger.info(f"Loaded catalog index: {CATALOG_INDEX}")
+    except Exception as e:
+        raise SystemExit(f"Catalog index '{CATALOG_INDEX}' not available ({e}). Run seed_index.py first.")
+
+    # Short-term: a fresh, empty session just for this call. The uuid suffix keeps
+    # it unique even if two calls start in the same second.
+    session_name = f"trip-session-{datetime.now():%Y%m%d-%H%M%S}-{uuid.uuid4().hex[:6]}"
+    session_index = await client.session(session_name)
+    logger.info(f"Opened live session: {session_name}")
+
+    agent = TravelConciergeAgent(client, session_index, ctx.room)
+
+    session = AgentSession(
+        stt=deepgram.STT(model="nova-2", language="en-US"),
+        llm=openai.LLM(model="gpt-4o-mini"),
+        tts=cartesia.TTS(model="sonic-turbo", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
+        vad=silero.VAD.load(min_silence_duration=0.5, activation_threshold=0.6),
+        turn_handling={
+            "turn_detection": MultilingualModel(),
+            "endpointing": {"min_delay": 0.5, "max_delay": 1.5},
+        },
+    )
+
+    await session.start(agent=agent, room=ctx.room)
+    await session.say(
+        "Hi! I'm your travel concierge. Tell me about the trip you're dreaming of and I'll find something.",
+        allow_interruptions=True,
+    )
+
+
+if __name__ == "__main__":
+    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

moss-live-labs/examples/travel-concierge/data/catalog.json

@@ -0,0 +1,16 @@
+[
+  { "id": "amalfi", "text": "Amalfi Coast, Italy — romantic cliffside villages, boat trips, and long seafood lunches. Best April to June or September. Around $3,500 per person for a week. Great for couples and honeymoons." },
+  { "id": "tulum", "text": "Tulum, Mexico — white-sand beaches, cenotes, and Mayan ruins. Warm and dry November to April. About $2,200 per person. Works for both families and couples." },
+  { "id": "kyoto", "text": "Kyoto, Japan — temples, gardens, and food. Cherry blossoms in spring, red foliage in autumn. Roughly $3,000 per person. A calm, cultural trip." },
+  { "id": "banff", "text": "Banff, Canada — turquoise lakes, hiking, and mountain gondolas. Best June to September. Around $2,600 per person. Excellent for active families." },
+  { "id": "santorini", "text": "Santorini, Greece — whitewashed cliffs, sunsets, and volcanic beaches. Ideal May to September. About $3,200 per person. A romantic favorite." },
+  { "id": "lisbon", "text": "Lisbon, Portugal — hilly old town, pastéis de nata, and day trips to Sintra. Mild most of the year. A budget-friendly city break at about $1,800 per person." },
+  { "id": "costa-rica", "text": "Manuel Antonio, Costa Rica — rainforest meets the beach, with sloths and zip-lines. Dry season December to April. Around $2,400 per person. Big hit with families." },
+  { "id": "iceland", "text": "Iceland Ring Road — waterfalls, geysers, and black-sand beaches. Northern lights in winter, midnight sun in summer. About $3,300 per person. A nature road trip." },
+  { "id": "bali", "text": "Bali, Indonesia — beaches, rice terraces, and wellness retreats. Dry season April to October. A mid-budget escape around $1,900 per person." },
+  { "id": "marrakech", "text": "Marrakech, Morocco — souks, riads, and desert excursions. Best in spring or autumn. Around $2,000 per person. Vibrant and full of color." },
+  { "id": "maui", "text": "Maui, Hawaii — beaches, the road to Hana, and snorkeling. Good year-round. A splurge at roughly $3,800 per person. Family-friendly." },
+  { "id": "prague", "text": "Prague, Czechia — medieval old town, castles, and river walks. Lovely in spring and autumn. A budget city trip around $1,700 per person." },
+  { "id": "queenstown", "text": "Queenstown, New Zealand — lakes, peaks, and adventure sports. Best November to April. About $3,400 per person. For thrill-seekers and hikers." },
+  { "id": "cape-town", "text": "Cape Town, South Africa — Table Mountain, beaches, and wine country. Best November to March. Around $2,700 per person. Scenic and varied." }
+]

moss-live-labs/examples/travel-concierge/pyproject.toml

@@ -0,0 +1,11 @@
+[project]
+name = "travel-concierge"
+version = "0.1.0"
+description = "Moss travel concierge — pre-loaded catalog + live session voice agent"
+readme = "README.md"
+requires-python = ">=3.14"
+dependencies = [
+    "python-dotenv>=1.0.0",
+    "moss>=1.1.1",
+    "livekit-agents[openai,deepgram,silero,turn-detector,cartesia]>=1.0",
+]