Add ai-file-analysis sample

README.md

@@ -103,6 +103,7 @@ $: npm run test
 - [`@examples/graph`](./examples/graph/README.md)
 - [`@examples/ai-mcp`](./examples/ai-mcp/README.md) — AI with the `openai` SDK + `@modelcontextprotocol/sdk`
 - [`@examples/a2a`](./examples/a2a/README.md) — agent-to-agent with `@a2a-js/sdk`
+- [`@examples/ai-file-analysis`](./examples/ai-file-analysis/README.md) — receiving attached files and sending them to a model
 - [`@examples/reactions`](./examples/reactions/README.md)
 - [`@examples/tab`](./examples/tab/README.md)
 - [`@examples/mcp-server`](./examples/mcp-server/README.md)

examples/ai-file-analysis/README.md

@@ -0,0 +1,87 @@
+# AI file analysis
+
+A Teams bot that reads files attached in personal (1:1) chat and sends the ones it understands to Azure OpenAI.
+
+One message handler covers both paths:
+
+- **Basic (no LLM)** replies with an Adaptive Card describing any file the sample cannot analyze, showing the metadata the file API exposes and the bytes that were downloaded.
+- **AI** converts supported text files and images into model input and streams the analysis back.
+
+### Reading the code
+
+Comments label which of two things a given block is doing:
+
+- **`FILE RECEIVE`** is the Teams SDK file API. This is the part worth copying into your own app.
+- **`SAMPLE GUARDRAIL`** is this sample deciding what it will forward to a model: which formats it accepts, how much text it sends, how many files per message, and whether anything is remembered between turns. These are arbitrary product choices, not SDK or Azure OpenAI requirements. Your app should pick its own.
+
+The distinction matters because most of the code volume here is guardrails. Receiving a file is only `ctx.files.list()` followed by `download()`.
+
+## Prerequisites
+
+- Node.js
+- A Teams bot registration
+- A Teams app manifest with `supportsFiles` set to `true` on the bot entry (see [Enable file support in the manifest](#enable-file-support-in-the-manifest))
+- An Azure OpenAI deployment (use a vision-capable model to analyze images). This is optional: without it the example still runs, receives files, and reports each one with an Adaptive Card instead of analyzing it. See [Running without a model](#running-without-a-model).
+
+## Enable file support in the manifest
+
+The bot entry in your Teams app manifest must set `supportsFiles` to `true`:
+
+```json
+"bots": [
+  {
+    "botId": "<your-bot-id>",
+    "scopes": ["personal"],
+    "supportsFiles": true
+  }
+]
+```
+
+Without it, Teams does not enable the attachment UI in the bot's chat, so there is no way to attach a file in the first place and `ctx.files.list()` has nothing to return.
+
+## Setup
+
+Add these settings to the example's `.env` alongside your bot credentials:
+
+```env
+AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com/
+AZURE_OPENAI_API_KEY=<api-key>
+AZURE_OPENAI_MODEL_DEPLOYMENT_NAME=<deployment-name>
+AZURE_OPENAI_API_VERSION=2024-10-21
+```
+
+Run:
+
+```bash
+npm run dev --workspace=@examples/ai-file-analysis
+```
+
+## Running without a model
+
+The file APIs this example demonstrates do not need a model, so the Azure OpenAI settings above are optional.
+
+Leave any of them unset and the example starts in metadata-only mode. It still receives, downloads, and reports every attached file with the Adaptive Card, showing the resolved content type, byte count, scope, and source, so the whole file round-trip is demonstrable without a model subscription. Only the analysis step is skipped, and the card says so.
+
+## What happens to an attached file
+
+1. `ctx.files.list()` returns the files on the incoming activity.
+2. Each file is downloaded once, and that in-memory copy is reused instead of refetching through the short-lived Teams download URL.
+3. `classifyFile` sorts each download into `text`, `image`, or `unsupported`.
+4. Unsupported files get the basic Adaptive Card. No model call is made for them.
+5. Supported files become OpenAI content parts and are sent in a single request, and the reply is streamed to Teams.
+
+Image bytes are sent inline as a data URI rather than as a link, so the pre-authorized `tempauth` download URL is never handed to the model.
+
+## Limits
+
+The sample accepts up to five files per message. Text input is capped at 100 KB per file and 250 KB per message, and images at 1 MB each. Supported image formats are PNG, JPEG, GIF, and WebP. Anything skipped or truncated produces a message explaining why.
+
+Because `download()` buffers the whole file first, these caps bound what reaches the model, not network transfer or process memory.
+
+## Scope
+
+The AI path is stateless: each message is analyzed on its own, with no conversation memory. That keeps a follow-up question from silently reusing files the user did not attach to it, and keeps images from being resent on every later turn.
+
+Statelessness here is a **`SAMPLE GUARDRAIL`**, not an SDK or Azure OpenAI constraint. Your app can keep conversation state and reuse previously attached files; this sample opts out so that every analysis is traceable to the files on the message that triggered it.
+
+There are no tools, citations, feedback, or follow-up suggestions here. See the [`ai-mcp`](https://github.com/microsoft/teams.ts/tree/main/examples/ai-mcp) sample for those.

examples/ai-file-analysis/eslint.config.js

@@ -0,0 +1 @@
+module.exports = require('@microsoft/teams.config/eslint.config').default;

examples/ai-file-analysis/package.json

@@ -0,0 +1,35 @@
+{
+  "name": "@examples/ai-file-analysis",
+  "version": "0.0.1",
+  "private": true,
+  "license": "MIT",
+  "main": "dist/index",
+  "types": "dist/index",
+  "files": [
+    "dist",
+    "README.md"
+  ],
+  "scripts": {
+    "clean": "npx rimraf ./dist",
+    "lint": "npx eslint",
+    "lint:fix": "npx eslint --fix",
+    "build": "npx tsc",
+    "start": "node -r dotenv/config dist/index.js",
+    "dev": "tsx watch -r dotenv/config src/index.ts"
+  },
+  "dependencies": {
+    "@microsoft/teams.api": "*",
+    "@microsoft/teams.apps": "*",
+    "@microsoft/teams.cards": "*",
+    "@microsoft/teams.common": "*",
+    "openai": "^4.104.0"
+  },
+  "devDependencies": {
+    "@microsoft/teams.config": "*",
+    "@types/node": "^22.5.4",
+    "dotenv": "^16.4.5",
+    "rimraf": "^6.0.1",
+    "tsx": "^4.23.1",
+    "typescript": "^5.4.5"
+  }
+}

examples/ai-file-analysis/src/ai/file-context.ts

@@ -0,0 +1,205 @@
+import type {
+  ChatCompletionContentPart,
+  ChatCompletionUserMessageParam,
+} from 'openai/resources/chat/completions';
+
+import type { IDownloadedFile } from '@microsoft/teams.apps';
+
+// SAMPLE GUARDRAIL: every constant below is a product choice made by this sample, not a Teams SDK or Azure OpenAI limit. They exist to keep one Teams message from turning into an unbounded model request. Pick your own values.
+//
+// `download()` buffers the whole file before any of these are checked, so they bound what reaches the model, not network transfer or process memory.
+const MAX_FILES = 5;
+const MAX_TEXT_BYTES_PER_FILE = 100 * 1024;
+const MAX_TOTAL_TEXT_BYTES = 250 * 1024;
+const MAX_IMAGE_BYTES = 1024 * 1024;
+
+// SAMPLE GUARDRAIL: the formats this sample is willing to forward. The file API itself delivers any attached file type.
+const IMAGE_CONTENT_TYPES = new Set([
+  'image/gif',
+  'image/jpeg',
+  'image/png',
+  'image/webp',
+]);
+
+const TEXT_EXTENSIONS = new Set([
+  'c',
+  'cpp',
+  'cs',
+  'css',
+  'csv',
+  'go',
+  'h',
+  'html',
+  'java',
+  'js',
+  'json',
+  'jsx',
+  'md',
+  'py',
+  'rb',
+  'rs',
+  'sh',
+  'sql',
+  'toml',
+  'ts',
+  'tsx',
+  'txt',
+  'xml',
+  'yaml',
+  'yml',
+]);
+
+/** Whether this sample can send a downloaded file to the model, and as what. */
+export type FileKind = 'image' | 'text' | 'unsupported';
+
+/** A downloaded file that {@link classifyFile} accepted, paired with its kind. */
+export type AnalyzableFile = {
+  file: IDownloadedFile;
+  kind: Exclude<FileKind, 'unsupported'>;
+};
+
+/** A model request built from the user's message and their analyzable files. */
+export type AnalysisRequest = {
+  content: ChatCompletionUserMessageParam['content'];
+  /** User-facing explanations for files that were skipped or truncated. */
+  warnings: string[];
+  /** Number of files whose content reached the model request. */
+  fileCount: number;
+};
+
+/**
+ * SAMPLE GUARDRAIL: decides whether a downloaded file can be sent to the model.
+ *
+ * The response MIME type is preferred, but the platform-supplied extension is a necessary fallback, and that part is a real file-receive detail rather than a sample preference: Teams commonly omits or misclassifies source files, reporting `.ts` as `video/vnd.dlna.mpeg-tts` for example.
+ */
+export function classifyFile(
+  file: IDownloadedFile,
+  extension?: string
+): FileKind {
+  const contentType = baseContentType(file.contentType);
+
+  if (IMAGE_CONTENT_TYPES.has(contentType)) {
+    return 'image';
+  }
+
+  if (
+    isTextContentType(contentType) ||
+    getTextExtension(extension, file.filename)
+  ) {
+    return 'text';
+  }
+
+  return 'unsupported';
+}
+
+/**
+ * Converts already-downloaded files into OpenAI content parts.
+ *
+ * The conversion itself is the AI integration. The caps it enforces along the way are SAMPLE GUARDRAILs, and each one that drops or shortens a file returns a warning so the user is never left guessing what the model saw.
+ */
+export function prepareAnalysis(
+  userText: string,
+  files: AnalyzableFile[]
+): AnalysisRequest {
+  const parts: ChatCompletionContentPart[] = [
+    {
+      type: 'text',
+      text: userText.trim() || 'Please analyze the attached file content.',
+    },
+  ];
+  const warnings: string[] = [];
+  let fileCount = 0;
+  let totalTextBytes = 0;
+
+  for (const { file, kind } of files.slice(0, MAX_FILES)) {
+    if (kind === 'image') {
+      if (file.bytes.byteLength > MAX_IMAGE_BYTES) {
+        warnings.push(
+          `${file.filename} was not sent to the model because it is larger than 1 MB.`
+        );
+        continue;
+      }
+
+      parts.push(
+        { type: 'text', text: `Attached image: ${file.filename}` },
+        {
+          type: 'image_url',
+          image_url: {
+            // FILE RECEIVE: the downloaded bytes are sent inline instead of handing the model the pre-authorized `tempauth` download URL, which is a short-lived credential.
+            url: toDataUri(file.bytes, baseContentType(file.contentType)),
+            detail: 'auto',
+          },
+        }
+      );
+      fileCount += 1;
+      continue;
+    }
+
+    const remainingBytes = MAX_TOTAL_TEXT_BYTES - totalTextBytes;
+    if (remainingBytes <= 0) {
+      warnings.push(
+        `${file.filename} was not sent to the model because the combined text-file limit was reached.`
+      );
+      continue;
+    }
+
+    const includedBytes = Math.min(
+      file.bytes.byteLength,
+      MAX_TEXT_BYTES_PER_FILE,
+      remainingBytes
+    );
+    const text = new TextDecoder().decode(file.bytes.subarray(0, includedBytes));
+    const truncated = includedBytes < file.bytes.byteLength;
+    totalTextBytes += includedBytes;
+
+    const lines = [`Attached file: ${file.filename}`, '', '<file>', text];
+    if (truncated) {
+      lines.push('[File content truncated by the sample.]');
+    }
+    lines.push('</file>');
+
+    parts.push({ type: 'text', text: lines.join('\n') });
+
+    if (truncated) {
+      warnings.push(
+        `${file.filename} was truncated before being sent to the model.`
+      );
+    }
+    fileCount += 1;
+  }
+
+  if (files.length > MAX_FILES) {
+    warnings.push(
+      `${files.length - MAX_FILES} supported file(s) were not sent to the model because this sample analyzes up to ${MAX_FILES} files per message. Unsupported files are reported separately.`
+    );
+  }
+
+  return { content: parts, warnings, fileCount };
+}
+
+function baseContentType(contentType: string): string {
+  return contentType.split(';', 1)[0].trim().toLowerCase();
+}
+
+function isTextContentType(contentType: string): boolean {
+  return (
+    contentType.startsWith('text/') ||
+    /\b(json|xml|javascript|yaml|csv|markdown)\b/.test(contentType)
+  );
+}
+
+function getTextExtension(
+  extension: string | undefined,
+  filename: string
+): string | undefined {
+  const normalizedExtension = extension
+    ? extension.replace(/^\./, '').toLowerCase()
+    : filename.split('.').pop()?.toLowerCase();
+  return normalizedExtension && TEXT_EXTENSIONS.has(normalizedExtension)
+    ? normalizedExtension
+    : undefined;
+}
+
+function toDataUri(bytes: Uint8Array, contentType: string): string {
+  return `data:${contentType};base64,${Buffer.from(bytes).toString('base64')}`;
+}