PublicArchive/supabase
2
0
Fork 0
mirror of https://github.com/supabase/supabase.git synced 2026-07-16 04:26:19 -04:00
Code Issues Packages Projects Releases Wiki Activity
37,415 Commits 1,453 Branches 56 Tags
create-pull-request/patch
Commit Graph

7 Commits

Author SHA1 Message Date
Pamela Chia 9858562b8b fix(telemetry): dedupe funnel toast error events (#47802) 
## Summary
Since #47293, an API failure on a signup / org-creation /
project-creation form emitted `dashboard_error_created` twice:
`useTrackFunnelError` fired the origin-tagged event and the global
`ToastErrorTracker` independently fired the legacy untagged
`source:'toast'` event for the same toast, each behind its own 10%
sampling draw. I verified the twin rate empirically at 8-11% of
origin-tagged funnel toasts, exactly the floor for two independent 10%
draws, meaning the twin co-fires for effectively every funnel error
([Hex
thread](https://app.hex.tech/supabase/thread/019f3bc1-3a5c-7200-9122-8e3439bfbe8c)).
Any consumer counting funnel errors without an `origin IS NOT NULL`
filter saw ~2x inflation.

The fix makes `ToastErrorTracker` the sole emitter of `source:'toast'`
events, so the duplicate is unrepresentable rather than suppressed.
Funnel call sites pass the id returned by `toast.error()` into
`trackFunnelError`, which registers the funnel properties against that
toast id instead of firing its own event – the tracker then emits a
single `dashboard_error_created` enriched with `origin` /
`errorCategory` / `errorReason` / `errorCode` for registered toasts, and
the plain untagged event otherwise. The `'toast'` overload of
`trackFunnelError` requires the toast id, so a missed pairing is a
compile error rather than a silent double count. Registration is
unconditional and there's only one sampling draw, so suppression can't
lose a sampling race. `'form'`-sourced funnel events are unchanged.

## Changes
- `lib/toast-errors.tsx`: toast-id → funnel-properties registry
(`registerFunnelErrorToast`); `ToastErrorTracker` emits one (optionally
enriched) event per error toast under a single 10% draw, deleting
entries once consumed
- `lib/telemetry/use-track-funnel-error.ts`: overloaded signature –
`'toast'` requires the id returned by `toast.error()` (type-enforced),
`'form'` keeps direct emission with its own sampling
- Update the 7 funnel `toast.error` call sites in `NewOrgForm`,
`SignUpForm`, and `pages/new/[slug]` to pass the toast id
- Component tests for the tracker (previously uncovered), including an
end-to-end test through `useTrackFunnelError`
- Code hygiene (also flagged by CodeRabbit): all four
`dashboard_error_created` emitters (toast, form, `AlertError`,
`ErrorMatcher`) independently encoded the 10% draw – downstream analysis
assumes a uniform sampling multiplier across sources, so one site
drifting would silently skew comparisons. The rate and the draw now live
in one place (`isDashboardErrorSampled()` in
`lib/telemetry/error-sampling.ts`). No behavior change.
- Mount `ToastErrorTracker` in the TanStack root (`routes/__root.tsx`),
mirroring `pages/_app.tsx`. The TanStack tree mounted `Toaster` but
never the tracker, so untagged toast error telemetry has never fired in
that flavour – and with the tracker now the sole emitter, the missing
mount would have silently dropped funnel toast events there too. Side
effect once the TanStack flavour ships: untagged `source:'toast'` volume
from it goes from zero to normal.

## Testing
Component-tested (`apps/studio/lib/toast-errors.test.tsx`):
- [x] Unregistered error toast fires exactly one untagged
`dashboard_error_created {source:'toast'}`
- [x] Registered funnel toast fires exactly one event, enriched with
`origin`/`errorCategory`/`errorReason`/`errorCode`
- [x] `useTrackFunnelError` with a toast id routes through the tracker
as a single enriched event
- [x] Non-error toasts ignored; the 10% sampling gate still applies

Full Studio unit suite passes (392 files / 4371 tests), plus typecheck
and lint.

Also verified end-to-end in a local browser (TanStack flavour, sample
rate temporarily forced to 1): a failed signup produced exactly one
`dashboard_error_created` with `{source:'toast', origin:'signup',
errorCategory:'api', errorReason:'email_already_registered',
errorCode:403}` and no untagged twin (two independent trials); an
unregistered error toast produced exactly one plain `{source:'toast'}`;
a client-side validation failure produced exactly one `{source:'form',
origin:'signup', errorCategory:'validation',
errorReason:'email_invalid'}`; success toasts produced nothing.

Post-deploy I'll re-run the twin-rate query from the Hex thread; the
untagged-twin rate on funnel pages should decay to ~0 as stale bundles
reload over 2-3 days.

## Notes
- Origin-tagged funnel toast events now ride the tracker's single 10%
draw instead of their own independent draw – statistically identical
volume, but the event fires on the tracker's next effect rather than
synchronously at the call site (irrelevant for PostHog)
- Registration must happen in the same synchronous block as
`toast.error()` (documented on the `TrackFunnelError` type) – all
current call sites comply
- The invalid Postgres version toast in `pages/new/[slug].tsx` (~line
416) needs no special-casing: unregistered toasts keep the plain
untagged event, so its telemetry is preserved
- Heads-up for `dashboard_error_created` consumers: overall untagged
`source:'toast'` volume will dip slightly after this deploys, since
funnel-page twins disappear. A volume monitor seeing that drop is this
fix landing, not a tracking regression (same class as the intended
GROWTH-893 sampling-unification drop).

## Linear
- fixes GROWTH-965


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced error telemetry for organization creation, sign-up, payment,
and project-creation flows by associating failures with toast
identifiers and enriched funnel context.
* Standardized dashboard error sampling logic across error handling
components for consistency.

* **Tests**
* Added comprehensive test coverage for toast error tracking, including
funnel registration, deduplication, filtering, and sampling behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
 2026-07-10 17:35:27 +08:00
Pamela Chia 98cfe3307e feat(telemetry): fix creation-funnel tracking gaps (#47386) 
## Summary

The creation-funnel instrumentation that shipped Jun 25 (#47291, #47293)
had real gaps, surfaced by the weekly telemetry audit and confirmed
against production PostHog data before I touched code. The two automated
reports also contradicted each other on `errorReason`; I checked
production (every value is a controlled slug) and the emit path (only
`useTrackFunnelError` sets it, and it only accepts classified slugs), so
I left the type as-is rather than add a cross-package abstraction for a
risk that cannot occur today.

## Changes

- Classify HTTP 401/403/404 API errors as `unauthorized` / `forbidden` /
`not_found` instead of the catch-all `other`. In production the
`org_creation` `other` bucket was ~96% 401s (~1,300 real over 4 days),
invisible in reason breakdowns. The status-code fallback runs after the
message-pattern match, so specific reasons still win and it only rescues
errors that would otherwise be `other`.
- Add a single `tier` property (`tier_free` / `tier_pro` / `tier_payg` /
`tier_team`) to `organization_creation_completed`, which previously
carried no properties. One canonical billing slug (matching
`SubscriptionTier`) instead of two overlapping plan/tier fields, so the
org-creation funnel segments cleanly by tier and joins against
subscription data. `tier_payg` is uncapped PRO.
- Freeze the submitted tier at submit time (snapshot in `createOrg`)
rather than reading live form state in the success callback, so the
event records the tier that was actually created even if the user edits
the form during the async payment flow.
- Emit `project_creation_form_exposed` with `surface: 'vercel'` on the
integration deploy-button project-creation page (the enum value existed
but was never fired). Gated on the URL `slug` so the impression is
captured as soon as the form renders, matching the sibling exposure hook
on that page.

I also checked the confirm-modal error path flagged in the insights
post: it already classifies via the shared
`useProjectCreateMutation.onError`, so adding instrumentation there
would double-count. No change made.

## Testing

These are analytics events with no UI change, so correctness is in what
lands in PostHog. Post-deploy validation I will run against production
(project 34344):

- `dashboard_error_created` where `origin='org_creation'` and
`errorReason='other'` drops ~96%, with `unauthorized` / `not_found`
appearing.
- `organization_creation_completed.tier` populated on 100% of new events
with one of the four tier slugs.
- `project_creation_form_exposed` with `surface='vercel'` goes from 0 to
greater than 0.

## Linear

- fixes GROWTH-948


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added telemetry for organization creation completion that includes the
selected billing tier.
* Added one-time telemetry when the Vercel project creation form is
exposed.
* **Bug Fixes**
* Improved API error classification to more accurately distinguish
unauthorized, forbidden, and not found responses.
* **Documentation**
* Updated telemetry event definitions to require tier metadata for
organization creation events.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-06-30 23:11:53 +08:00
Gildas Garcia c6fc456910 chore: cleanup duplicate exports studio (#47387) 
## Problem

Knip reports many duplicate exports (both named and default). Besides,
we're moving away from default exports and even have an eslint rule to
enforce it on new code.

## Solution

- Cleanup those exports
- Update imports when necessary

No functional changes. If it builds, it's fine
 2026-06-29 15:46:16 +02:00
Pamela Chia 2d0bcd4714 feat(telemetry): classify funnel creation errors (#47293) 
## Summary

The KPI-3 friction dashboard needs to know *why* users hit errors on the
signup, project-creation, and org-creation funnels, not just that they
did. The existing `dashboard_error_created` event already fires for
these paths (10% sampled, with `$pathname`), but carries no reason:
~98.5% of events have no `errorType` and no property carries an error
message. This adds PII-safe classification computed client-side from a
controlled vocabulary, so raw error text never leaves the browser.
Validation errors (previously invisible, since they are inline form
errors that never raise a toast) are now captured on invalid submit.

## Changes

- Extend `dashboard_error_created` with `origin`, `errorCategory`,
`errorReason`, `errorCode`, and a `form` source value
- Add a pure, unit-tested classifier (`funnel-errors.ts`) and a
10%-sampled tracking hook (`use-track-funnel-error.ts`); the classifier
maps errors to stable slugs and emits only slugs + HTTP status, never
raw message text
- Classify signup errors (API failures + validation) in `SignUpForm`
- Classify project-creation errors (API failures, OrioleDB guard,
validation) in the new-project wizard
- Classify org-creation errors (API failures, payment/card declines,
confirm-subscription, validation) in `NewOrgForm`

## Testing

13 unit tests cover every classifier branch (validation / api / network
/ payment, status-code handling, message-pattern matching, and
fallbacks).

To verify on the Vercel preview (events are 10% sampled; set the sample
rate to 1 locally to observe each fire):
- Signup with a weak but non-empty password: `origin=signup,
source=form, errorCategory=validation, errorReason=password_invalid`
- Signup with an already-registered email: `origin=signup, source=toast,
errorCategory=api, errorReason=email_already_registered`
- New project with an empty name: `origin=project_creation, source=form,
errorReason=project_name_invalid`
- New org with an empty name: `origin=org_creation, source=form,
errorReason=org_name_missing`
- New org with a declined test card: `origin=org_creation,
errorCategory=payment`

PII: raw `error.message` is never sent; only controlled slugs and HTTP
status. Dashboard consumers must filter `origin IS NOT NULL` so these do
not collide with the generic toast events the global tracker still
emits.

## Linear

- fixes FE-3691


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added improved, categorized telemetry for signup, project creation,
and organization creation errors, including payment,
subscription-change, and validation failures.
* Extended dashboard error events with optional structured diagnostics
(origin, category, reason, and optional error code) and support for
form-origin reporting.

* **Bug Fixes**
* Improved project-creation handling to record a validation telemetry
event when an Oriole image is unavailable.
* Ensured payment-related and subscription-change failures are captured
consistently alongside existing user toasts.

* **Tests**
* Added unit tests covering API/network/validation/Stripe error
classification and reason mapping.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-06-25 20:56:23 +08:00
Alaister Young 7959948005 fix(studio): make useTrack stable across renders (#46412) 
Follow-up to #46140 — the returned `track` function was re-created on
every router change or selected project/org refetch, which made it
unstable for consumers that depend on referential equality (e.g. effect
deps, memoized children).

**Changed:**
- Read `project?.ref`, `org?.slug`, and `router.pathname` through
`useLatest` so the values inside `track` stay current without being deps
of the `useCallback`
- Drop the deps from the `useCallback` — `track` is now stable for the
lifetime of the component

## To test

- Verify telemetry events still send with correct `project` /
`organization` groups and `pathname`
- Confirm any consumers that put `track` in `useEffect` deps no longer
re-run unnecessarily on route or project changes

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Improved telemetry event tracking to capture more accurate context
information at the time events are sent, ensuring data reflects current
application state.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/46412?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
 2026-05-27 16:20:43 +08:00
Charis 205cbe7d26 chore(studio}: enforce import order, remove bare import specifiers (#44585) 2026-04-07 20:34:10 -04:00
Sean Oliver 73e3143b0c Add type-safe event tracking utility (#39745) 
Adds a clean, type-safe wrapper for telemetry event tracking that automatically injects project and organization context.

- Export TelemetryGroups type from telemetry-constants
- Add useTrack() hook with full TypeScript event validation
- Refactor project creation events to use new API
- Reduces boilerplate from ~10 lines to ~2 lines per event
 2025-10-24 18:25:26 +00:00