Skip to content

Comments

chore(web): Add PostHog LLM analytics#882

Merged
brendan-kellam merged 4 commits intomainfrom
bkellam/posthog-ai
Feb 24, 2026
Merged

chore(web): Add PostHog LLM analytics#882
brendan-kellam merged 4 commits intomainfrom
bkellam/posthog-ai

Conversation

@brendan-kellam
Copy link
Contributor

@brendan-kellam brendan-kellam commented Feb 12, 2026

Summary by CodeRabbit

  • Chores
    • Added an AI/analytics integration dependency to the web package.
  • Refactor
    • Modularized and exported analytics helpers for cleaner initialization and ID retrieval.
  • Chores
    • Enhanced tracing/telemetry wiring for chat to improve observability and reliability.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 12, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3e5b31a and 5e2e827.

📒 Files selected for processing (1)
  • packages/web/src/features/chat/actions.ts

Walkthrough

Adds @posthog/ai to the web package, exports PostHog helper functions, and refactors the AISDK model-selection flow to return { model, providerOptions } while initializing a PostHog client, obtaining a distinct ID, and optionally wrapping the model with tracing.

Changes

Cohort / File(s) Summary
Dependency Addition
packages/web/package.json
Added runtime dependency @posthog/ai ^7.8.10.
PostHog Utilities
packages/web/src/lib/posthog.ts
Renamed/exported distinct-id getter as tryGetPostHogDistinctId and added createPostHogClient helper; captureEvent updated to use these helpers.
Chat Model Selection
packages/web/src/features/chat/actions.ts
Refactored _getAISDKLanguageModelAndOptions to return { model, providerOptions? }; centralized per-provider selection into one post-processing step that creates a PostHog client, retrieves distinct ID, and conditionally applies withTracing before returning.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Chat Client
    participant Action as Model Selection Action
    participant PostHog as PostHog Client
    participant AISDK as AISDK Model

    Client->>Action: Request model selection
    Action->>Action: Select model & providerOptions
    Action->>PostHog: createPostHogClient()
    Action->>PostHog: tryGetPostHogDistinctId()
    alt tracing enabled
        Action->>AISDK: withTracing(model, distinctId)
        AISDK-->>Action: tracedModel
    else tracing disabled
        Action-->>Action: use model as-is
    end
    Action->>Client: return { model, providerOptions }
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Possibly related PRs

Suggested reviewers

  • msukkari
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'chore(web): Add PostHog LLM analytics' clearly and accurately summarizes the main change—integrating PostHog analytics for LLM tracking in the web package.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch bkellam/posthog-ai

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@brendan-kellam brendan-kellam marked this pull request as ready for review February 23, 2026 21:25
@github-actions
Copy link
Contributor

@brendan-kellam your pull request is missing a changelog!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/web/src/lib/posthog.ts (1)

90-110: ⚠️ Potential issue | 🟡 Minor

distinctId may be undefined when passed to posthog.capture().

tryGetPostHogDistinctId() can return undefined, but posthog.capture() expects distinctId to be a string. When no distinct ID is resolvable (no cookie, no session, no API key), this will pass undefined to the PostHog client, which could cause silent data loss or a runtime error.

Consider guarding against this:

Suggested fix
     const distinctId = await tryGetPostHogDistinctId();
     const posthog = await createPostHogClient();
 
+    if (!distinctId) {
+        return;
+    }
+
     const headersList = await headers();
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/lib/posthog.ts` around lines 90 - 110, The call to
posthog.capture in captureEvent currently passes distinctId which can be
undefined (from tryGetPostHogDistinctId); ensure captureEvent always provides a
string by adding a fallback before calling posthog.capture (e.g., const
distinctIdSafe = distinctId ?? crypto.randomUUID() or a deterministic anon id
using env.SOURCEBOT_INSTALL_ID) and pass distinctIdSafe to posthog.capture
instead of distinctId; update references in captureEvent and keep
tryGetPostHogDistinctId usage but guard its result so posthog.capture never
receives undefined.
🧹 Nitpick comments (2)
packages/web/src/features/chat/actions.ts (1)

34-36: Consolidate duplicate imports from @/lib/posthog.

Lines 34 and 36 both import from @/lib/posthog. Merge them into a single import statement.

Suggested fix
-import { captureEvent } from "@/lib/posthog";
-import { withTracing } from "@posthog/ai";
-import { createPostHogClient, tryGetPostHogDistinctId } from "@/lib/posthog";
+import { captureEvent, createPostHogClient, tryGetPostHogDistinctId } from "@/lib/posthog";
+import { withTracing } from "@posthog/ai";
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/chat/actions.ts` around lines 34 - 36, There are
duplicate import lines from "@/lib/posthog"; consolidate them by replacing the
two separate imports with a single import that includes captureEvent,
createPostHogClient, and tryGetPostHogDistinctId together (so update the import
statements that reference captureEvent and the import that references
createPostHogClient/tryGetPostHogDistinctId into one unified import).
packages/web/package.json (1)

66-66: Consider updating @posthog/ai to the latest available version.

Version 7.8.10 is valid and exists on npm, but it is not current. The latest version is 7.9.1, with several newer patch releases also available (7.8.11, 7.8.12, 7.8.13). Since the dependency specifies ^7.8.10, it would automatically accept these compatible updates. Consider upgrading to 7.9.1 to benefit from the latest features and fixes.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/package.json` at line 66, Update the `@posthog/ai` dependency in
package.json by replacing the current version specifier "^7.8.10" with the newer
release (e.g. "7.9.1") so the project uses the latest patch/feature fixes;
locate the dependency entry for "@posthog/ai" and change the version string
accordingly, then run your package manager (npm/yarn/pnpm) to install and verify
no breakages.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/web/src/lib/posthog.ts`:
- Around line 80-88: The code currently creates a new PostHog client on every
call of createPostHogClient (and callers like captureEvent) causing resource
leaks; change createPostHogClient to lazily initialize and return a module-level
singleton PostHog instance (e.g., keep a private let posthogInstance and
instantiate it once inside createPostHogClient if undefined), update callers
(captureEvent and any usages in actions.ts) to reuse createPostHogClient instead
of creating new clients per invocation, and add/export a shutdown function
(e.g., shutdownPostHog) that calls posthogInstance.shutdown() for graceful
teardown (also call it on process exit in initialization code or tests as
needed).

---

Outside diff comments:
In `@packages/web/src/lib/posthog.ts`:
- Around line 90-110: The call to posthog.capture in captureEvent currently
passes distinctId which can be undefined (from tryGetPostHogDistinctId); ensure
captureEvent always provides a string by adding a fallback before calling
posthog.capture (e.g., const distinctIdSafe = distinctId ?? crypto.randomUUID()
or a deterministic anon id using env.SOURCEBOT_INSTALL_ID) and pass
distinctIdSafe to posthog.capture instead of distinctId; update references in
captureEvent and keep tryGetPostHogDistinctId usage but guard its result so
posthog.capture never receives undefined.

---

Nitpick comments:
In `@packages/web/package.json`:
- Line 66: Update the `@posthog/ai` dependency in package.json by replacing the
current version specifier "^7.8.10" with the newer release (e.g. "7.9.1") so the
project uses the latest patch/feature fixes; locate the dependency entry for
"@posthog/ai" and change the version string accordingly, then run your package
manager (npm/yarn/pnpm) to install and verify no breakages.

In `@packages/web/src/features/chat/actions.ts`:
- Around line 34-36: There are duplicate import lines from "@/lib/posthog";
consolidate them by replacing the two separate imports with a single import that
includes captureEvent, createPostHogClient, and tryGetPostHogDistinctId together
(so update the import statements that reference captureEvent and the import that
references createPostHogClient/tryGetPostHogDistinctId into one unified import).

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5be4667 and 3e5b31a.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (3)
  • packages/web/package.json
  • packages/web/src/features/chat/actions.ts
  • packages/web/src/lib/posthog.ts

@brendan-kellam brendan-kellam merged commit 5a00847 into main Feb 24, 2026
8 checks passed
@brendan-kellam brendan-kellam deleted the bkellam/posthog-ai branch February 24, 2026 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant