Skip to content

Add PostHog analytics integration - #17

Closed
posthog[bot] wants to merge 2 commits into
mainfrom
posthog/instrumentation-5cabb1
Closed

Add PostHog analytics integration#17
posthog[bot] wants to merge 2 commits into
mainfrom
posthog/instrumentation-5cabb1

Conversation

@posthog

@posthog posthog Bot commented Aug 2, 2026

Copy link
Copy Markdown

Summary

This PR adds anonymous, server-side PostHog analytics to the CLI using the posthog-node SDK (installed via npm), and fixes a crash the integration introduced.

  • src/posthog.js — new PostHog client (singleton), a capture() helper (personless, $process_person_profile: false, a process-scoped random distinct ID), and shutdownPostHog().
  • src/cli.js — instruments three lifecycle events and awaits SDK shutdown before exit (in a .finally() so it runs on success or failure).
  • package.json / package-lock.json — adds posthog-node@4.18.0 (pinned down from the initially-resolved 5.47.3, which requires Node ^20.20.0 || >=22.22.0 while this package declares Node >=18).
  • .env.example — documents POSTHOG_PROJECT_TOKEN and POSTHOG_HOST.
  • action.yml — sets those two env vars for the packaged GitHub Action (see Environment variables below).

Events instrumented

Event What it measures File
profile_initialized A learning profile is created or intentionally refreshed from the template. src/cli.js
explainer_generation_started A merged pull request has been retrieved and explainer generation begins. src/cli.js
explainer_generated An explainer has been generated and saved to the configured output directory. src/cli.js

Fix: CLI crashed without PostHog env vars configured

src/posthog.js threw an uncaught exception at import time whenever POSTHOG_PROJECT_TOKEN/POSTHOG_HOST were unset outside NODE_ENV=production. Since this package is installed globally via npm and run directly (no NODE_ENV set), this broke every invocation — including pr-explainer --help, which the README lists as a documented sanity check. Changed the throw to a console.error warning so the CLI still runs; behavior (loud until configured, silent no-op once set) is otherwise unchanged.

Insights and dashboards created

Type Name Link
Dashboard Analytics basics (wizard) Open
Insight Profile initialization volume (wizard) Open
Insight Profile to explainer funnel (wizard) Open
Insight Explainer generation volume (wizard) Open
Notebook PostHog setup report (wizard) Open

How to verify

  1. Locally: POSTHOG_PROJECT_TOKEN=<token> POSTHOG_HOST=https://us.i.posthog.com node src/cli.js init, or via the Action once merged.
  2. In PostHog, open Activity — you should see profile_initialized / explainer_generation_started / explainer_generated events arriving.
  3. Check the dashboard for the same.

Environment variables

This project has no hosted "production" deployment — it ships as an npm CLI (npm install -g @shilpi1958/pr-explainer) and an optional packaged GitHub Action (action.yml).

  • GitHub Action path: configured automatically. POSTHOG_PROJECT_TOKEN/POSTHOG_HOST are now set directly in action.yml's env: block, so analytics work as soon as anyone uses shilpi1958/pr-explainer@v1 — no action needed. (The PostHog project token is a write-only capture key, safe to commit — same model as a posthog-js browser key.)
  • Direct CLI install path (npm install -g + running pr-explainer locally): there's no file this reaches automatically. Analytics stay off for that path unless whoever runs it exports POSTHOG_PROJECT_TOKEN and POSTHOG_HOST themselves — this is inherent to a globally-installed CLI, not something a repo file can configure.

Before merging (per the wizard's setup report)

  • Confirm events actually arrive in PostHog after a real CLI/Action run (not yet observed in this run).
  • Decide whether to add server-side error tracking around the CLI entry point (not in scope of this PR).
  • No durable/stable user identity exists yet — captures stay anonymous (process-scoped random ID) until one is introduced.

Created with PostHog Code

posthog Bot added 2 commits August 2, 2026 18:55
Instruments anonymous, server-side usage events (profile_initialized, explainer_generation_started, explainer_generated) via posthog-node, and fixes a startup crash: the CLI threw an uncaught exception (even for --help) whenever POSTHOG_PROJECT_TOKEN/POSTHOG_HOST were unset, which is the default for virtually every install.

Generated-By: PostHog Code
Task-Id: 294b59aa-1890-42ff-8023-f6d58679a684
action.yml is the only production-analogous config this project has (an npm CLI/GitHub Action has no hosted deploy target) — it's the file every consumer's CI reads when they use shilpi1958/pr-explainer@v1. Sets POSTHOG_PROJECT_TOKEN/POSTHOG_HOST there so the instrumented events actually reach PostHog for that usage path. The project token is a write-only capture key (safe to expose publicly, same model as posthog-js browser keys).

Generated-By: PostHog Code
Task-Id: 294b59aa-1890-42ff-8023-f6d58679a684
@shilpi1958

Copy link
Copy Markdown
Owner

Closing as superseded by #18, which already landed the PostHog funnel events (profile_initialized, explainer_generation_started, explainer_generated), $ai_generation capture for AI Evals, .env loading, Action inputs for PostHog secrets, and tests — plus the Graphify-backed repo explainer.

This draft also conflicts with main and would commit the project token into action.yml; #18 passes tokens via Action inputs/secrets instead.

@shilpi1958 shilpi1958 closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant