2 Commits

Author	SHA1	Message	Date
Pamela Chia	98cfe3307e	feat(telemetry): fix creation-funnel tracking gaps (#47386) ... ## Summary The creation-funnel instrumentation that shipped Jun 25 (#47291, #47293) had real gaps, surfaced by the weekly telemetry audit and confirmed against production PostHog data before I touched code. The two automated reports also contradicted each other on `errorReason`; I checked production (every value is a controlled slug) and the emit path (only `useTrackFunnelError` sets it, and it only accepts classified slugs), so I left the type as-is rather than add a cross-package abstraction for a risk that cannot occur today. ## Changes - Classify HTTP 401/403/404 API errors as `unauthorized` / `forbidden` / `not_found` instead of the catch-all `other`. In production the `org_creation` `other` bucket was ~96% 401s (~1,300 real over 4 days), invisible in reason breakdowns. The status-code fallback runs after the message-pattern match, so specific reasons still win and it only rescues errors that would otherwise be `other`. - Add a single `tier` property (`tier_free` / `tier_pro` / `tier_payg` / `tier_team`) to `organization_creation_completed`, which previously carried no properties. One canonical billing slug (matching `SubscriptionTier`) instead of two overlapping plan/tier fields, so the org-creation funnel segments cleanly by tier and joins against subscription data. `tier_payg` is uncapped PRO. - Freeze the submitted tier at submit time (snapshot in `createOrg`) rather than reading live form state in the success callback, so the event records the tier that was actually created even if the user edits the form during the async payment flow. - Emit `project_creation_form_exposed` with `surface: 'vercel'` on the integration deploy-button project-creation page (the enum value existed but was never fired). Gated on the URL `slug` so the impression is captured as soon as the form renders, matching the sibling exposure hook on that page. I also checked the confirm-modal error path flagged in the insights post: it already classifies via the shared `useProjectCreateMutation.onError`, so adding instrumentation there would double-count. No change made. ## Testing These are analytics events with no UI change, so correctness is in what lands in PostHog. Post-deploy validation I will run against production (project 34344): - `dashboard_error_created` where `origin='org_creation'` and `errorReason='other'` drops ~96%, with `unauthorized` / `not_found` appearing. - `organization_creation_completed.tier` populated on 100% of new events with one of the four tier slugs. - `project_creation_form_exposed` with `surface='vercel'` goes from 0 to greater than 0. ## Linear - fixes GROWTH-948 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added telemetry for organization creation completion that includes the selected billing tier. * Added one-time telemetry when the Vercel project creation form is exposed. * **Bug Fixes** * Improved API error classification to more accurately distinguish unauthorized, forbidden, and not found responses. * **Documentation** * Updated telemetry event definitions to require tier metadata for organization creation events. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-06-30 23:11:53 +08:00
Pamela Chia	2d0bcd4714	feat(telemetry): classify funnel creation errors (#47293) ... ## Summary The KPI-3 friction dashboard needs to know *why* users hit errors on the signup, project-creation, and org-creation funnels, not just that they did. The existing `dashboard_error_created` event already fires for these paths (10% sampled, with `$pathname`), but carries no reason: ~98.5% of events have no `errorType` and no property carries an error message. This adds PII-safe classification computed client-side from a controlled vocabulary, so raw error text never leaves the browser. Validation errors (previously invisible, since they are inline form errors that never raise a toast) are now captured on invalid submit. ## Changes - Extend `dashboard_error_created` with `origin`, `errorCategory`, `errorReason`, `errorCode`, and a `form` source value - Add a pure, unit-tested classifier (`funnel-errors.ts`) and a 10%-sampled tracking hook (`use-track-funnel-error.ts`); the classifier maps errors to stable slugs and emits only slugs + HTTP status, never raw message text - Classify signup errors (API failures + validation) in `SignUpForm` - Classify project-creation errors (API failures, OrioleDB guard, validation) in the new-project wizard - Classify org-creation errors (API failures, payment/card declines, confirm-subscription, validation) in `NewOrgForm` ## Testing 13 unit tests cover every classifier branch (validation / api / network / payment, status-code handling, message-pattern matching, and fallbacks). To verify on the Vercel preview (events are 10% sampled; set the sample rate to 1 locally to observe each fire): - Signup with a weak but non-empty password: `origin=signup, source=form, errorCategory=validation, errorReason=password_invalid` - Signup with an already-registered email: `origin=signup, source=toast, errorCategory=api, errorReason=email_already_registered` - New project with an empty name: `origin=project_creation, source=form, errorReason=project_name_invalid` - New org with an empty name: `origin=org_creation, source=form, errorReason=org_name_missing` - New org with a declined test card: `origin=org_creation, errorCategory=payment` PII: raw `error.message` is never sent; only controlled slugs and HTTP status. Dashboard consumers must filter `origin IS NOT NULL` so these do not collide with the generic toast events the global tracker still emits. ## Linear - fixes FE-3691 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added improved, categorized telemetry for signup, project creation, and organization creation errors, including payment, subscription-change, and validation failures. * Extended dashboard error events with optional structured diagnostics (origin, category, reason, and optional error code) and support for form-origin reporting. * **Bug Fixes** * Improved project-creation handling to record a validation telemetry event when an Oriole image is unavailable. * Ensured payment-related and subscription-change failures are captured consistently alongside existing user toasts. * **Tests** * Added unit tests covering API/network/validation/Stripe error classification and reason mapping. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-06-25 20:56:23 +08:00