mirror of
https://github.com/supabase/supabase.git
synced 2026-07-13 02:57:00 -04:00
Pamela Chia
9858562b8b
## Summary Since #47293, an API failure on a signup / org-creation / project-creation form emitted `dashboard_error_created` twice: `useTrackFunnelError` fired the origin-tagged event and the global `ToastErrorTracker` independently fired the legacy untagged `source:'toast'` event for the same toast, each behind its own 10% sampling draw. I verified the twin rate empirically at 8-11% of origin-tagged funnel toasts, exactly the floor for two independent 10% draws, meaning the twin co-fires for effectively every funnel error ([Hex thread](https://app.hex.tech/supabase/thread/019f3bc1-3a5c-7200-9122-8e3439bfbe8c)). Any consumer counting funnel errors without an `origin IS NOT NULL` filter saw ~2x inflation. The fix makes `ToastErrorTracker` the sole emitter of `source:'toast'` events, so the duplicate is unrepresentable rather than suppressed. Funnel call sites pass the id returned by `toast.error()` into `trackFunnelError`, which registers the funnel properties against that toast id instead of firing its own event – the tracker then emits a single `dashboard_error_created` enriched with `origin` / `errorCategory` / `errorReason` / `errorCode` for registered toasts, and the plain untagged event otherwise. The `'toast'` overload of `trackFunnelError` requires the toast id, so a missed pairing is a compile error rather than a silent double count. Registration is unconditional and there's only one sampling draw, so suppression can't lose a sampling race. `'form'`-sourced funnel events are unchanged. ## Changes - `lib/toast-errors.tsx`: toast-id → funnel-properties registry (`registerFunnelErrorToast`); `ToastErrorTracker` emits one (optionally enriched) event per error toast under a single 10% draw, deleting entries once consumed - `lib/telemetry/use-track-funnel-error.ts`: overloaded signature – `'toast'` requires the id returned by `toast.error()` (type-enforced), `'form'` keeps direct emission with its own sampling - Update the 7 funnel `toast.error` call sites in `NewOrgForm`, `SignUpForm`, and `pages/new/[slug]` to pass the toast id - Component tests for the tracker (previously uncovered), including an end-to-end test through `useTrackFunnelError` - Code hygiene (also flagged by CodeRabbit): all four `dashboard_error_created` emitters (toast, form, `AlertError`, `ErrorMatcher`) independently encoded the 10% draw – downstream analysis assumes a uniform sampling multiplier across sources, so one site drifting would silently skew comparisons. The rate and the draw now live in one place (`isDashboardErrorSampled()` in `lib/telemetry/error-sampling.ts`). No behavior change. - Mount `ToastErrorTracker` in the TanStack root (`routes/__root.tsx`), mirroring `pages/_app.tsx`. The TanStack tree mounted `Toaster` but never the tracker, so untagged toast error telemetry has never fired in that flavour – and with the tracker now the sole emitter, the missing mount would have silently dropped funnel toast events there too. Side effect once the TanStack flavour ships: untagged `source:'toast'` volume from it goes from zero to normal. ## Testing Component-tested (`apps/studio/lib/toast-errors.test.tsx`): - [x] Unregistered error toast fires exactly one untagged `dashboard_error_created {source:'toast'}` - [x] Registered funnel toast fires exactly one event, enriched with `origin`/`errorCategory`/`errorReason`/`errorCode` - [x] `useTrackFunnelError` with a toast id routes through the tracker as a single enriched event - [x] Non-error toasts ignored; the 10% sampling gate still applies Full Studio unit suite passes (392 files / 4371 tests), plus typecheck and lint. Also verified end-to-end in a local browser (TanStack flavour, sample rate temporarily forced to 1): a failed signup produced exactly one `dashboard_error_created` with `{source:'toast', origin:'signup', errorCategory:'api', errorReason:'email_already_registered', errorCode:403}` and no untagged twin (two independent trials); an unregistered error toast produced exactly one plain `{source:'toast'}`; a client-side validation failure produced exactly one `{source:'form', origin:'signup', errorCategory:'validation', errorReason:'email_invalid'}`; success toasts produced nothing. Post-deploy I'll re-run the twin-rate query from the Hex thread; the untagged-twin rate on funnel pages should decay to ~0 as stale bundles reload over 2-3 days. ## Notes - Origin-tagged funnel toast events now ride the tracker's single 10% draw instead of their own independent draw – statistically identical volume, but the event fires on the tracker's next effect rather than synchronously at the call site (irrelevant for PostHog) - Registration must happen in the same synchronous block as `toast.error()` (documented on the `TrackFunnelError` type) – all current call sites comply - The invalid Postgres version toast in `pages/new/[slug].tsx` (~line 416) needs no special-casing: unregistered toasts keep the plain untagged event, so its telemetry is preserved - Heads-up for `dashboard_error_created` consumers: overall untagged `source:'toast'` volume will dip slightly after this deploys, since funnel-page twins disappear. A volume monitor seeing that drop is this fix landing, not a tracking regression (same class as the intended GROWTH-893 sampling-unification drop). ## Linear - fixes GROWTH-965 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Enhanced error telemetry for organization creation, sign-up, payment, and project-creation flows by associating failures with toast identifiers and enriched funnel context. * Standardized dashboard error sampling logic across error handling components for consistency. * **Tests** * Added comprehensive test coverage for toast error tracking, including funnel registration, deduplication, filtering, and sampling behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>