mirror of
https://github.com/supabase/supabase.git
synced 2026-05-07 17:30:25 -04:00
create-pull-request/patch
406 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Greg Richardson
|
538f9e3e82 |
fix: prevent AI assistant from soliciting sensitive creds (#45692)
Adds prompt guardrails and evals to prevent the AI assistant from asking users to share sensitive data (API keys, `.env` contents, etc.) and to warn when credentials are shared. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Stronger safety behavior: assistant now refuses requests to share full environment files, asks for variable names only, and directs users to secure secret-management tooling. * Immediate warning and guidance if credentials or other sensitive values are pasted in chat, without repeating exposed secrets. * **Behavior** * Clarified evaluation rules so responses more consistently follow the new safety guidance. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Greg Richardson
|
5f8906a20e |
fix: add destructive operation guardrails to AI assistant (#45194)
Prevents the AI assistant from helping with local git/filesystem operations, and adds explicit warnings before irreversible database operations (DROP TABLE, DELETE without WHERE, etc.). Adds a `safetyScorer` and eval cases to cover these behaviours. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a Safety metric to evaluations so assistant responses are scored for safe handling of destructive or risky requests * Assistant guidance updated to refuse destructive local VCS/filesystem actions and require clear warnings for irreversible database operations * **Tests** * Added evaluation cases covering safe refusals, clear warnings, and correct handling of destructive or risky prompts * **Chores** * Enabled Safety metric in online evaluation manifests/handlers <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jordi Enric
|
d8bb0ade65 |
feat(studio): add timezone picker to user dropdown (#45517)
## Problem The dashboard renders all timestamps in the browser's local timezone. When debugging app issues, users often want to see logs and timestamps in a different timezone (e.g. their app's deployment region) without changing their OS clock. ## Fix - New Timezone submenu in the user-avatar dropdown, sitting next to the existing Theme picker. Search-as-you-type combobox over the full IANA catalog plus an Auto detect option. - Selection persists in localStorage (`supabase-ui-timezone`) and survives `clearLocalStorage()`. No backend schema change. - New `lib/datetime.tsx` exposes pure timezone-aware formatters (`formatDateTime`, `formatDate`, `formatTime`, `formatFromNow`, `toTimezone`) plus a `TimezoneProvider` and matching React hooks (`useTimezone`, `useFormatDateTime`, ...). The pure functions take `tz` explicitly so they're easy to unit test (17 vitest cases covering DST transitions, multi-tz formatting, unix-micro/Date inputs, invalid-tz fallback). - The selected timezone propagates to every existing `<TimestampInfo>` in Studio via a new `TimestampInfoProvider` context exported from `ui-patterns`. No per-callsite changes needed for those ~20+ surfaces. - The `UnifiedLogs` date column migrates off `date-fns` to the new `useFormatDateTime` hook (the rest of the date-fns callers stay as-is, since they're either internal range math or non-display). - `ALL_TIMEZONES` (~600 entries) moves out of `PITR.constants.ts` into a shared `lib/constants/timezones.ts`. PITR keeps a re-export shim so its callers don't move. New `TIMEZONES_BY_IANA` dedupes the catalog by primary IANA name (the original list contains both PDT and PST rows for `America/Los_Angeles`, etc.) and `findTimezoneByIana` provides reverse lookup. - Telemetry: `timezone_picker_clicked` PostHog event with `previousTimezone`, `nextTimezone`, `isAutoDetected` properties. Notes for reviewers: - Bare `dayjs(x).format(...)` calls (~157 files) intentionally still render in browser-local time. Surfaces opt in by switching to the new wrappers, so this PR is the abstraction plus logs adoption; broader migration is a follow-up. - Two `// prettier-ignore` lines (`apps/studio/pages/_app.tsx`, `apps/studio/components/interfaces/UnifiedLogs/UnifiedLogs.fields.tsx`) work around a pre-existing local-tooling issue where `prettier-plugin-sql-cst` strips angle-bracket type arguments under certain conditions. Project's pinned prettier (3.8.1) does not strip; the issue surfaces with a globally-installed prettier. Worth tracking separately. - Hydration: `guessLocalTimezone()` and `useLocalStorageQuery` are client-only. Studio is mostly CSR via the Pages Router, but any SSR'd `<TimestampInfo>` may briefly render in the server's tz before client hydration. Existing behavior already had this mismatch with `.local()`; this PR does not regress it. - Backend timestamps round-tripped through query params and mutations stay UTC. The picker is display-only. ## How to test - Run `pnpm dev:studio`, sign in. - Open the user avatar dropdown (top right). Hover Timezone. - Search for "tokyo", pick `(UTC+09:00) Osaka, Sapporo, Tokyo`. - Open any project, navigate to Logs (e.g. `Project > Logs > Edge Functions`). Hover a log row's timestamp; the popover should show UTC, the chosen tz (`Asia/Tokyo`), and the relative time. Visible cell text should be in JST. - Visit any page that uses `<TimestampInfo>` (Database > Backups, Project Pause state, Edge Function details). Same tooltip should reflect Asia/Tokyo. - Refresh the page; timezone is still Asia/Tokyo. - Reopen the picker, choose Auto detect; timestamps revert to browser local. - Run `pnpm --filter studio test lib/datetime.test.ts`. 17 tests should pass. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Timezone selector added to the user menu with auto-detect and manual override * App-wide timezone provider and hooks plus a shared timezone catalog for consistent timezone-aware display * Timestamp components accept an optional timezone prop and respect user preference (persisted) * **Bug Fixes / Improvements** * Logs and timestamp displays now use the new timezone formatting hooks * **Tests** * Added comprehensive datetime and timezone catalog tests * **Telemetry** * Telemetry event added for timezone picker interactions <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Charis
|
0433eeb5f5 |
feat(studio): mark sql provenance for safety (#45336)
Mark provenance of SQL via the branded types SafeSqlFragment and UntrustedSqlFragment. Only SafeSqlFragment should be executed; UntrustedSqlFragments require some kind of implicit user approval (show on screen + user has to click something) before they are promoted to SafeSqlFragment. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Editor and RLS tester show loading states for inferred/generated SQL and include a dedicated user SQL editor for safer edits. * **Refactor** * Platform-wide SQL handling tightened: snippets and AI-generated SQL are treated as untrusted/display-only until promoted, improving safety and consistency. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Ali Waseem
|
618c902e1a |
feat(studio): add keyboard shortcuts to Database listing pages (#45467)
## Summary Adds the `O→S` / `Shift+F` / `Shift+N` / `F→C` shortcut set across every Database listing page, on top of the schema-visualizer pattern from #45386. Fixes [FE-3131](https://linear.app/supabase/issue/FE-3131). A shared `list-page.*` registry replaces what would have been a per-page registry file for each route, and `useShortcut`/`Shortcut` gain a `label` override so a single ID renders contextually in Cmd+K and hover tooltips. ## Pages and shortcuts | Page | `O→S` | `Shift+F` | `Shift+N` | `F→C` | | --- | --- | --- | --- | --- | | Tables | Open schema selector | Search tables | Create new table | Clears search + entity-type filter | | Functions | Open schema selector | Search functions | Create new function | Clears search + Return Type + Security | | Triggers — data | Open schema selector | Search triggers | Create new trigger | Clears search + Table filter | | Triggers — event | — | Search event triggers | Create new event trigger | Clears search + Owner filter | | Enumerated Types | Open schema selector | Search enumerated types | Create new enumerated type | Clears search | | Indexes | Open schema selector | Search indexes | Create new index | Clears search | | Roles | — | Search roles | Add new role | Clears search + filter type → 'all' | | Publications | — | Search publications | — | Clears search | | Extensions | — | Search extensions | — | Clears search | `Shift+N` only fires when the page-specific gate allows it (permission + unlocked schema + any other prerequisite like `hasTables` for triggers). ## Test plan - [ ] On each of the 9 pages, all four shortcuts behave as listed in the table above - [ ] Hover the wrapped controls — tooltip shows the page-specific label and the right keybind badge - [ ] Open the schema selector via `O→S` — the hover tooltip is suppressed while the popover is open - [ ] `Shift+N` is a no-op on locked schemas / when permission is missing / when a prereq fails (e.g. no tables on Triggers data) - [ ] `Shift+F` focuses the search input; while focused, Escape clears (with text) → Escape blurs (when empty) - [ ] `Cmd+K` shows the page-specific shortcut name while on each page; the entry goes away on navigation - [ ] `Mod+/` reference sheet shows one "List pages" group with 4 generic entries - [ ] No regression to existing schema-visualizer / table-editor / SQL-editor shortcut behavior <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a dedicated list-page shortcut group in the shortcuts reference. * **Improvements** * Keyboard shortcuts across database list pages: focus & select search, reset filters, and create-new-item shortcuts. * Escape now clears/searches or blurs inputs to avoid accidental popover closes. * Create/new buttons respond to shortcuts when allowed; disabled actions keep permission tooltips for clarity. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Ivan Vasilov
|
0dec08c96f |
chore: Bump vulnerable dependencies (#45513)
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Upgraded the UUID library to a newer major version across apps and removed a now-unneeded dev dependency. * Pinned PostCSS to a workspace-specific version to stabilize builds. * **Refactor** * Improved internal identifier generation for more consistent behavior without changing outward functionality. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Alaister Young
|
3036bb103b |
[FE-3134] fix(studio): handle ALTER TABLE IF EXISTS in RLS detection (#45493)
The SQL Editor was warning about missing RLS even when the same query enabled it, if the user wrote `ALTER TABLE IF EXISTS ...`. The parser regex didn't recognise `IF EXISTS` and was capturing `IF` as the table name, so the RLS event never matched the `CREATE TABLE`. **Changed:** - `ALTER TABLE` regex in `sql-event-parser.ts` now accepts the optional `IF EXISTS` and `ONLY` modifiers, matching Postgres's `ALTER TABLE [ IF EXISTS ] [ ONLY ] name` grammar. **Added:** - Unit tests for `IF EXISTS`, `ONLY`, and both combined. - Regression test in `SQLEditor.utils.test.ts` using the customer's exact SQL. ## To test 1. Open the SQL Editor and paste: ```sql CREATE TABLE IF NOT EXISTS public."Conversations" (id int8 primary key); ALTER TABLE IF EXISTS public."Conversations" ENABLE ROW LEVEL SECURITY; ``` 2. Hit Run – the "table will not have RLS" warning should **not** appear. 3. Sanity check: a `CREATE TABLE` without any matching `ENABLE ROW LEVEL SECURITY` still triggers the warning. Addresses [FE-3134](https://linear.app/supabase/issue/FE-3134/sql-editor-warns-about-missing-rls-policy-incorrectly). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added comprehensive test coverage for Row Level Security detection across different SQL syntax patterns and clause combinations * **Bug Fixes** * Enhanced Row Level Security detection capabilities in the SQL editor by extending support for additional ALTER TABLE statement syntax variations, improving the accuracy and completeness of security configuration recognition <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com> |
||
|
Matt Rossman
|
94d7c0d463 |
chore(studio): remove @supabase/mcp-utils dependency (#45438)
**Changes** Replaces our custom `StreamTransport` with [InMemoryTransport](https://github.com/modelcontextprotocol/typescript-sdk/blob/4fbcfcd176b6b189970263c4625eb6e60db043d2/packages/core/src/util/inMemory.ts#) from the official MCP SDK, removing the need for the `@supabase/mcp-utils` dependency. **Verification steps** I verified Studio's AI Assistant still works as expected. Closes AI-694 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated the Model Context Protocol SDK dependency to version 1.29.0. * Removed unused AI utilities dependency. * Optimized the internal AI service communication layer for improved efficiency. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Matt Rossman
|
36ae9beb0c |
chore(ai): remove DPA signer killswitch for assistant tracing (#45134)
Removes the temporary killswitch added when Braintrust was onboarded as
a subprocessor, to satisfy the 30-day DPA notice obligation. The window
has elapsed and legal has cleared removal.
Drops the `orgIsDpaSigned` check from `isTracingAllowed`, removes the
extra `/platform/organizations/{slug}/documents/dpa-signed` network hop
from `getOrgAIDetails`, and cleans up all call sites and tests.
Closes AI-596
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Simplified AI tracing eligibility logic by removing DPA signing status
checks. Tracing authorization decisions now depend solely on region,
HIPAA addon status, and project sensitivity settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Joshen Lim
|
7f5865872a |
Enforce noUnusedLocals and noUnusedParameters in tsconfig.json + fix all related issues (#45264)
## Context Enforce `noUnusedLocals` and `noUnusedParameters` in tsconfig.json + fix all related issues |
||
|
Tom Gallacher
|
31689b1a71 |
chore: default staging to AWS provider (#45180)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? chore ## What is the current behavior? New project dialog defaults to AWS Revamped provider on staging. ## What is the new behavior? New project dialog defaults to AWS provider on staging. ## Additional context N/A <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Standardized the default cloud provider to AWS across all environments (staging/preview included), ensuring consistent infrastructure selection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com> |
||
|
Danny White
|
1ab7251060 |
chore(studio): clarify partnered orgs and projects (#44328)
## What kind of change does this PR introduce? UI improvements. - Resolves DEPR-401 - Resolves DEPR-424 - Resolves DEPR-425 ## What is the current behaviour? Studio currently blurs two different concepts together: - `billing_partner` / `billing_via_partner`, which represent real billing ownership for marketplace-managed organisations such as AWS and Vercel - Stripe connection state, which is not actually partner billing in the same sense, but was previously being mocked through the same UI paths That made the Stripe work harder to reason about and left some local behaviour dependent on temporary overrides rather than the API shape we want to ship. ## What is the new behaviour? This PR separates those concerns while keeping the existing AWS and Vercel marketplace experience intact. - AWS and Vercel continue to use `billing_partner` / `billing_via_partner` for billing ownership, lockouts, and manage CTAs - Stripe display state now comes from `integration_source`, which lets Studio show Stripe-specific badges and alerts without treating Stripe as a billing partner - organisation-level partner UI is unified across AWS, Vercel, and Stripe, including the org banner, navbar icon treatment, and organisation cards - project-level Stripe UI now appears only when the project itself is marked as Stripe-connected, including the project switcher, project list surfaces, and a project-level banner - Stripe-connected organisations are no longer incorrectly blocked behind the AWS/Vercel-style billing management alerts for invoices, billing address, payment methods, or plan changes - banner dismissal is scoped to the relevant org/project and partner/integration state ## Review order Most of the diff size here is regression tests and generated types. The behavioural changes are concentrated in a smaller set of files. Recommended review order: 1. `integration_source` vs `billing_partner` data-model split and org/project query mapping 2. org-level UI: partner icon, org banner, org dropdown/card treatment 3. billing gating updates for Stripe vs AWS/Vercel 4. project-level Stripe UI: dropdown, list surfaces, banner 5. tests and generated types | Stripe Org(s) | | --- | | <img width="1024" height="759" alt="Organizations Supabase" src="https://github.com/user-attachments/assets/d0ef338c-3b41-4c6d-b3bd-f21a2c182840" /> | | Vercel Org(s) | | --- | | <img width="1024" height="759" alt="Organizations Supabase" src="https://github.com/user-attachments/assets/1dc57770-3f24-45ac-840f-34680555cde8" /> | | AWS Org(s) | | --- | | <img width="1024" height="759" alt="Organizations Supabase" src="https://github.com/user-attachments/assets/7847dad0-ee30-4a65-ab0b-b3b16af0d34f" /> | | Stripe Org, Non-Stripe Project | | --- | | <img width="1152" height="885" alt="Mallet Toolshed Supabase-1673E019-792C-462C-B6F8-C5DDB810B331" src="https://github.com/user-attachments/assets/556fbea3-b5ae-4f2f-96b9-6f66c6654e4a" /> | | Stripe Org, Stripe Project | | --- | | <img width="1152" height="885" alt="Hammer Toolshed Supabase-7E86C17C-561F-4221-BD16-EAFF7D41AAE0" src="https://github.com/user-attachments/assets/94f8daf6-0320-413e-8d56-59f9acaaea15" /> | | Vercel Org | | --- | | <img width="1024" height="759" alt="Projects Toolshed Supabase-A7891653-9366-4B99-89DD-789D70CD52E3" src="https://github.com/user-attachments/assets/c87ee6e8-4451-4866-a905-23a38b2593e3" /> | | AWS Org | | --- | | <img width="1024" height="759" alt="Projects Toolshed Supabase-58A43ECE-569E-4541-9463-346A90B02CFF" src="https://github.com/user-attachments/assets/9350a180-4d58-42a1-ad1a-95893c2e8b12" /> | This also removes the old Stripe mock override path in Studio so the frontend matches the intended API model more closely. ## ~~Dependencies~~ (merged!) This work depends on the private platform change that exposes `integration_source` on the relevant organisation and project payloads: - https://github.com/supabase/platform/pull/31874 _Update: now merged._ ## Local testing ### Stripe If you have the private `platform` repo checked out locally, make sure your local API returns `integration_source: 'stripe_projects'` consistently for the Stripe-linked org/project you are testing. Important responses: - `/platform/organizations` - `/platform/organizations/:slug/projects` - `/platform/projects/:ref` Verify: - org banner and org icon show the Stripe connected state - unopened and opened project switcher both show Stripe only for Stripe-linked projects - project cards / table rows show the Stripe chip only for Stripe-linked projects - the project-level Stripe banner appears across project surfaces - billing address, tax ID, invoices, payment methods, and plan changes remain editable in Studio for Stripe orgs ### Vercel Use a Vercel Marketplace org with real `billing_partner` / `billing_via_partner` values. Important org-level endpoints for local mocking in `platform`: - `/platform/organizations` - `/platform/organizations/:slug` - `/platform/organizations/:slug/billing/subscription` Project-level Vercel indicators still come from `/platform/integrations/:slug`, not `integration_source`. ### AWS Use an AWS Marketplace org with real `billing_partner` / `billing_via_partner` values. Important org-level endpoints for local mocking in `platform`: - `/platform/organizations` - `/platform/organizations/:slug` - `/platform/organizations/:slug/billing/subscription` AWS does not currently have a Stripe-like project-level indicator in these org/project payloads. ## Notes - `billing_partner` is no longer the right abstraction for the Stripe-connected case in this PR. It remains the source of truth for marketplace billing ownership, while Stripe currently uses `integration_source` as a connection/display signal. - I re-ran `pnpm api:codegen` while tightening this PR and kept only the generated type changes this branch actually depends on, to avoid unrelated API drift in the review. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Stripe Projects integration added for billing and project flows * Partner icons/badges shown across org and project lists, dropdowns, and rows * Dismissible, partner-specific marketplace/integration banners with contextual CTA behavior * Improved partner-billing detection to drive billing UI and invoice/plan availability * **Tests** * Extensive new test coverage for billing UI, partner-managed fallbacks, banners, icons, and related flows <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com> |
||
|
Greg Richardson
|
e38ba624bc |
feat(ai): update rls knowledge for 'secure by default' (#45072)
Updates the RLS knowledge loaded by the dashboard AI assistant to explain the new secure-by-default functionality. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified PostgreSQL/RLS guidance in Studio: tables are now "secure by default"—SQL-created tables aren’t exposed via the Data API unless explicit grants are given to anon/authenticated/service_role and RLS is enabled; added an “Exposing a Table to the Data API” workflow, strengthened RLS prerequisites in best practices, and improved troubleshooting/error-recovery guidance. * **Tests / Evaluations** * Added an evaluation case validating guidance for non-RLS tables requiring explicit grants and RLS policies. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Ali Waseem <waseema393@gmail.com> |
||
|
Manuel Rubio
|
667be50982 |
fix(self-hosted): local AI assistant unable to read schemas and interpret/execute SQL (#44757)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Bug fix for #42574 ## What is the current behavior? If you setup a self-hosted Supabase environment and ask local AI assistant it is unable to interpret the results for executed SQL queries. ## What is the new behavior? Local AI assistant can now read schemas and interpret the results for executed SQL queries. ## Additional context I tested it using ollama with a smaller model which does not support reasoning mode so It would be useful if someone can test it using an actual OpenAI API KEY. The main issue was that `executeSql` returns a wrapped `{ result: {...} }` on `fallback-tools.ts` making the model think there's no result. The solution is just to unwrap it like it's done at: `apps/studio/data/database-functions/database-functions-query.ts`: ``` export async function getDatabaseFunctions(...) { ... const { result } = await executeSql(...) ... return result as DatabaseFunction[] } ``` Also added a `false` default value in `execute_sql` tool schema so zod validation does not reject a valid LLM tool call if the model omits the field (which happened with the smaller model) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Refactor** * Optimized internal schema metadata query handling in AI tools. * **Chores** * Enhanced SQL execution tool robustness by providing a sensible default for query type specification when not explicitly provided. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Andrey A. <56412611+aantti@users.noreply.github.com> |
||
|
Matt Rossman
|
8f69a10cc9 |
fix(studio): reliable schema-aware SQL editor AI completions (#44730)
A variety of fixes and improvements to the Cmd+K AI completions endpoint in the [SQL Editor](https://supabase.com/dashboard/project/_/sql/new): - Pre-load table definitions for the public schema and any other schemas referenced in the editor, so the model has real column names without needing to fetch them dynamically - Replace the generic tool suite with a single streamlined `getSchemaDefinitions` tool the model can still call to look up additional schemas on demand without behavior differences across platform & self-hosted - Swap generic chat system prompt for a purpose-built `COMPLETION_PROMPT`; fix role (`assistant` → `user`) for consistency with other endpoints - Validate and type the request body with `zod`, which was previously untyped (`any`) - Improve Cmd+K behavior when nothing is selected — use the full editor content as context, return the complete query rather than just the changed fragment, and switch to a generation mode when the editor is blank - Escape single quotes in schema names when fetching entity definitions in `pg-meta` to prevent schema names from breaking out of the SQL string and injecting arbitrary content into the prompt ## Before Before, the SQL Editor would often hallucinate tables / columns that don't exist in the user's database making it less helpful if you don't know the exact table/column names. Even with maximum Assistant opt-in level on the org, it would often fail to call the necessary tools to gather database context. <img width="5062" height="1522" alt="image" src="https://github.com/user-attachments/assets/fbe1130f-6b5a-41a8-99d7-7268880af188" /> <img width="2540" height="658" alt="image" src="https://github.com/user-attachments/assets/a31c2967-7751-4fce-a9b7-60bd77660b1a" /> Sometimes it also silently fails and generates empty queries: <img width="1352" height="398" alt="CleanShot 2026-04-09 at 17 46 06@2x" src="https://github.com/user-attachments/assets/e17c103a-d47d-47e6-8c2e-101f0fae5651" /> Or echos back the user's prompt: <img width="1368" height="282" alt="CleanShot 2026-04-09 at 23 04 56@2x" src="https://github.com/user-attachments/assets/7dff6e64-f54e-45b5-8e86-5399e5a2fe41" /> ## After In this example, the completion correctly interpreted my request for "completed" todos as a query on the `completed_foo` column in my `public` schema, instead of assuming existence of a `completed` column. <img width="1452" height="838" alt="CleanShot 2026-04-09 at 17 43 13@2x" src="https://github.com/user-attachments/assets/7a575589-78b4-448d-810a-0330ff08ef8b" /> In this example, the completion was correctly aware of an `other` schema because it was detected in my existing query. I didn't have to select the text, it included the full query in context when unselected. Notice how it correctly used the `is_done` column when I asked for "completed" cakes: <img width="1372" height="534" alt="CleanShot 2026-04-09 at 17 39 07@2x" src="https://github.com/user-attachments/assets/e6b7eb6f-f3e8-4fa1-90a3-b5e34ddc14e4" /> Supersedes #44151 Closes AI-544 |
||
|
Alaister Young
|
2555e81dde |
fix(studio): don't scan dollar-quoted bodies for DDL in SQL editor (#45050)
Fixes a false positive in the CREATE-TABLE-without-RLS warning modal added in #45008. The warning was firing on `CREATE FUNCTION` statements because the `SELECT..INTO` detector was matching plpgsql variable assignments inside `$$…$$` function bodies. Reported example that triggered the modal with no table actually being created: ```sql create or replace function schema_checks() returns jsonb language plpgsql as $$ declare ret jsonb; begin select jsonb_build_object('value', 'ok') into ret; return ret; end; $$; ``` **Changed:** - `SQLEventParser.match()` now strips the body of `$tag$…$tag$` blocks before running detectors. Tags are kept as markers; content is blanked out so function bodies, DO blocks, and dollar-quoted string literals are never scanned as DDL. - Updated a pre-existing parser test that asserted the buggy behaviour (it expected `CREATE TABLE fake` inside a `$$…$$` string literal to be detected — `$$…$$` is a string literal in Postgres, not DDL). **Added:** - Regression tests in `SQLEditor.utils.test.ts` covering: the exact reported function, DO blocks with `select into`, `create table` text inside a function body, mixed top-level `CREATE TABLE` + function with `INTO` assignments, and custom `$body$…$body$` tags. - Parser-level regression test in `sql-event-parser.test.ts`. ## To test - In the SQL editor, paste the function from the Slack report and run it — the RLS warning modal should not appear. - Run `create table foo (id int8 primary key);` on its own — modal still appears as before. - Run `create table foo (id int8); create or replace function bar() returns int language plpgsql as $$ declare v int; begin select 1 into v; return v; end; $$;` — modal should flag only `foo`, not `v`. - Run an existing destructive query (`drop table x`) — unaffected, modal still works. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Parser no longer treats DDL/DML-like text inside PL/pgSQL functions, DO blocks, or dollar-quoted bodies (including nested/custom tags) as top-level CREATE TABLE/SELECT INTO, preventing false detections and UI warnings. * **Tests** * Added unit and e2e regression tests covering dollar-quoted blocks, nested dollar tags, DO blocks, SELECT INTO inside functions, and positive controls with a real top-level CREATE TABLE. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com> |
||
|
Alaister Young
|
3aed9a9a2a |
feat(studio): warn before CREATE TABLE without RLS in SQL editor (#45008)
Adds a pre-execution warning in the SQL editor when a `CREATE TABLE` statement is run without enabling Row Level Security on the new table. Responds to the press call-out around SQL editor security. <img width="708" height="498" alt="Screenshot 2026-04-18 at 4 31 07 PM" src="https://github.com/user-attachments/assets/4f23ed5e-f32c-46f0-b0da-ac6d4c661c7c" /> **Added:** - Pre-execution check in `executeQuery` that detects `CREATE TABLE` statements without a matching `ALTER TABLE ... ENABLE ROW LEVEL SECURITY` in the same submitted SQL. - New "Run and enable RLS" action in the warning modal that rewrites the SQL to append `ALTER TABLE [schema.]<table> ENABLE ROW LEVEL SECURITY;` for each detected table before running. - Link in the modal to the RLS docs. **Changed:** - `RunQueryWarningModal` now renders `Dialog` directly (instead of `ConfirmationModal`) so it can show three buttons: Cancel / Run without RLS / Run and enable RLS. - `sqlEventParser` table-name regex now supports quoted identifiers containing spaces (e.g. `"My Table"`) and escaped quotes (e.g. `"user""table"`). The check runs against the SQL that's actually submitted, so partial-selection works correctly — selecting only the `CREATE TABLE` portion will trigger the warning even if there's a matching `ENABLE RLS` lower in the editor. ## To test - Open the SQL editor and run `create table foo (id int8 primary key);` → modal should appear with the RLS warning bullet and three buttons. - Click **Run and enable RLS** → query runs, table is created with RLS enabled. - Click **Run without RLS** → query runs as written, no RLS. - Run `create table foo (id int8); alter table foo enable row level security;` → no modal (RLS already enabled in same submission). - Run `create table public.bar (id int8); create table baz (id int8); alter table baz enable rls;` → modal flags only `public.bar`. - Select only the `create table` portion of a snippet that also enables RLS lower down and run the selection → modal should still fire. - Run an existing destructive query (`drop table x`) → modal still works as before with two buttons (Cancel / Run this query). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * SQL editor now detects CREATE TABLE statements missing Row Level Security (RLS) and shows counts and dynamic table/schema details in a redesigned warning dialog with updated pluralization and a “Learn more” link. * New actions: “Run without RLS” and, when available, “Run and enable RLS” which applies RLS and runs the query; editor can execute an overridden SQL payload when applying RLS changes. * **Tests** * Added comprehensive unit and e2e tests covering RLS detection, SQL augmentation, trigger handling, identifier parsing, and the “Run and enable RLS” flow. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com> |
||
|
Alaister Young
|
1b1d05ff96 |
chore: upgrade vite to v8 and vitest to v4 (#44833)
Upgrade vite and vitest to their latest major versions across the
monorepo, along with related packages.
**Changed:**
- `vite` catalog: `^7.3.2` → `^8.0.8` (Rolldown replaces esbuild/Rollup)
- `vitest` catalog: `^3.2.0` → `^4.1.4`
- `@vitejs/plugin-react`: `^4.3.4` → `^6.0.1`
- `@vitest/coverage-v8`: `^3.2.0` → `^4.1.4`
- `@vitest/ui`: `^3.2.0` → `^4.1.4`
- `vite-tsconfig-paths`: `^4.3.2` / `^5.1.4` → `^6.1.1`
**Pinned to vite 7:**
- `apps/lite-studio` — `@react-router/dev` hasn't declared vite 8
support yet
- `blocks/vue` — Nuxt plugins (`vite-plugin-inspect`, `vite-dev-rpc`,
`vite-hot-client`, `vite-plugin-vue-tracer`) haven't declared vite 8
support yet
**Test fixes for vitest 4 breaking changes:**
- **`apps/studio/lib/api/snippets.utils.test.ts`** — Replaced
`vi.mock('fs/promises')` automock with an explicit factory. Vitest 4's
automocking doesn't create mock functions for getter-based exports on
Node built-ins, so `mockedFS.access.mockResolvedValue` etc. were
`undefined`.
- **`apps/studio/lib/api/self-hosted/functions/index.test.ts`** —
Changed `mockReturnValue` to `mockImplementation(function() { ... })`
for a constructor mock. Vitest 4 no longer allows `mockReturnValue` when
the mock is called with `new`.
- **`apps/studio/tests/pages/api/mcp/index.test.ts`** — Changed arrow
function to regular `function` in `mockImplementation` for
`StreamableHTTPServerTransport`. Arrow functions can't be constructors,
and vitest 4 now enforces this.
- **`packages/ui-patterns/vitest.setup.ts`** — Changed `ResizeObserver`
mock from arrow function to regular `function` for the same constructor
enforcement reason. This was crashing Radix popover rendering in jsdom.
## To test
- `pnpm test:studio` — all 226 test files should pass
- `pnpm --filter ui-patterns vitest run` — all 183 tests should pass
- `pnpm --filter www test -- --run` — all 19 tests should pass
- `pnpm --filter ui vitest run` — all tests should pass
- `pnpm --filter dev-tools vitest run` — all tests should pass
- `pnpm --filter ai-commands vitest run` — all tests should pass
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Standardized and updated development tooling versions and version
sources for consistent installs across the repo (Vite, Vitest,
vite-tsconfig-paths and related plugins/catalog entries).
* **Tests**
* Improved test mocks and typings (updated mock
factories/implementations and tightened spy/type assertions) to increase
test reliability and compatibility with updated tooling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
|
||
|
Matt Rossman
|
4f90833ed9 |
feat(assistant): enable Braintrust tracing in production (#44644)
Removes the temporary killswitch that was blocking tracing in production environments. Note this still excludes from tracing projects that are any of the following: - HIPAA sensitive - Hosted in EU regions - Within orgs where some owner previously signed our DPA See [approval message](https://supabase.slack.com/archives/C051L8U2EJF/p1775526578561789?thread_ts=1775144006.270549&cid=C051L8U2EJF) for context. Closes AI-450 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Tracing enablement now depends solely on the presence of required API credentials instead of environment-based restrictions, enabling diagnostic tracing functionality across all environments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jordi Enric
|
d2017189e1 |
fix(migrations): parse migration version timestamps as UTC SU-353303 (#44655)
## Problem
The "INSERTED AT (UTC)" column in the Database Migrations UI showed
local time instead of UTC. For a user in Buenos Aires (UTC-3), a
migration timestamped at UTC 08:33:31 would show 05:33:31 in the table.
The tooltip's relative time also showed "in 3 hours" (future) for a
migration that had already run, because the UTC offset was applied in
the wrong direction.
Root cause: `parseMigrationVersion` parsed the version string (format
`YYYYMMDDHHmmss`, which the Supabase CLI generates in UTC) using
`dayjs()` without the UTC flag, so dayjs interpreted the digits as local
time.
## Fix
- Changed `parseMigrationVersion` to use `dayjs.utc()` so the version
string is correctly interpreted as a UTC timestamp.
- Updated the label formatter in `Migrations.tsx` to use
`.utc().format()`, so the displayed time matches the column header
("INSERTED AT (UTC)").
- Added the dayjs UTC plugin setup to the test file and added a
regression test that asserts `toISOString()` returns the correct UTC
time.
## Before
<img width="1102" height="658" alt="CleanShot 2026-04-08 at 12 41 18@2x"
src="https://github.com/user-attachments/assets/5eccdfb1-757c-4794-b24f-6a2c71f483dc"
/>
## After
<img width="1126" height="612" alt="CleanShot 2026-04-08 at 12 42 03@2x"
src="https://github.com/user-attachments/assets/6f3da69f-ace5-4758-b025-b49d8b325034"
/>
## How to test
- Set your browser/OS timezone to something other than UTC (e.g.
America/Buenos_Aires, UTC-3).
- Open the Database Migrations page for a project that has migrations.
- The "INSERTED AT (UTC)" column should show the UTC time matching the
version number digits (e.g. version `20260406083331` should show `06 Apr
2026, 08:33:31`).
- Hover over the timestamp. The tooltip should show the same value for
"UTC", a correctly offset value for your local timezone, and a relative
time that reflects the past (e.g. "3 hours ago", not "in 3 hours").
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Migration timestamps now parsed and displayed in UTC for consistent,
accurate labels; unparsable versions show "Unknown".
* **New Features**
* Improved migration version labeling for clearer, uniformly formatted
date/time shown in the UI.
* **Tests**
* Expanded tests for migration parsing and label formatting; test setup
updated for UTC handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
Charis
|
205cbe7d26 | chore(studio}: enforce import order, remove bare import specifiers (#44585) | ||
|
Matt Rossman
|
8aeacc6152 |
feat(assistant): disable Braintrust tracing for EU regions and DPA signers (#44504)
**Changes** - Extracted tracing conditional to an `isTracingAllowed` helper with unit tests (the function is simple but sensitive hence the extra testing precaution) - Disables Braintrust tracing for projects in EU database regions (region prefix `eu-`) to address GDPR data residency concerns - Disables Braintrust tracing for orgs whose owners have signed the previous DPA, as a stopgap during the 30-day notice period for the updated DPA that adds Braintrust as a subprocessor - Refactored `org-ai-details.ts` → `ai-details.ts`, splitting `getOrgAIDetails` into separate org and project helpers to cleanly scope the EU-region check at the project level DPA check uses the newly added `/documents/dpa-signed` endpoint from https://github.com/supabase/platform/pull/31060. This PR includes regenerated `api.d.ts` and `platform.d.ts` from running `pnpm codegen` in `packages/api-types` to get type safety on this new endpoint. Note tracing is still yet to be activated in production, this is a preparatory step. **To verify** Send a chat message and check for the `x-braintrust-span-id` response header on `POST /api/ai/sql/generate-v4` — it should be absent for DPA-signed orgs or EU-region projects, and present otherwise. <img width="3594" height="1992" alt="CleanShot 2026-04-03 at 14 28 58@2x" src="https://github.com/user-attachments/assets/4c91d7ad-2604-4531-a78e-dedf41632fa5" /> If you have access to the Braintrust dashboard, you can also verify whether logs are produced or not in the Assistant project there. Closes AI-570 Closes AI-569 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tracks organization DPA signing and detects EU-region projects * Assistant tracing now follows a combined compliance policy (HIPAA addon, DPA, project sensitivity, region) * Added helpers to fetch org and project AI details * **Documentation** * Expanded API docs with additional examples and clarified parameter descriptions * Added response schemas for subscription preview and document status * **Tests** * Added/updated tests covering DPA/region behavior and tracing policy enforcement <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Ali Waseem
|
6be596ea34 |
feat: add user preference to enable queue operations (#44366)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? - Remove queue operations from feature preview into settings - Refactor dashboard settings - Resolves DEPR-434 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Dashboard settings panel in Account preferences with toggles for Inline Editor and Queue Operations; “Dashboard” added to project Configuration. * **Removed** * Old Inline Editor settings UI and the Queue Operations feature-preview UI removed. * **Refactor** * Consolidated dashboard preferences into a single settings surface; banners and actions now navigate to preferences; account/preferences layouts and back-navigation behavior adjusted for platform vs self-hosted. * **Tests** * Added tests for settings UI, menu generation, redirects, and local-storage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Joshen Lim <joshenlimek@gmail.com> Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com> |
||
|
Matt Rossman
|
82deff37de |
feat(assistant): lazy load topic knowledge via load_knowledge tool (#44296)
Moves knowledge (RLS, Edge Functions, PostgreSQL best practices, Realtime) out of the static system prompt and into a `load_knowledge` tool the model calls on demand, reducing prompt bloat. This is a temporary stopgap until the [standard Supabase agent-skills](https://github.com/supabase/agent-skills) are ready for integration in Assistant. - New always-available `load_knowledge` tool added to `rendering-tools.ts` - Updated `Message.Parts.tsx` so the "Ran load_knowledge" chip renders in chat - System prompt replaces the four knowledge blobs with an `## Available Knowledge` block and is hardened to load knowledge for given topics - New "Knowledge Usage" scorer and `requiredKnowledge` assertions check that knowledge loads as expected in test scenarios - Filters GraphQL error responses out of `output.docs` before faithfulness scoring to reduce noise See "Knowledge Usage" scoring 100% in evals with no major regressions: https://github.com/supabase/supabase/pull/44296#issuecomment-4145760236 Sample trace showing the tool in action ([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=351a11c8-9cb7-4945-93ad-d11e8cc2e3e1&s=351a11c8-9cb7-4945-93ad-d11e8cc2e3e1)) <img width="2192" height="1730" alt="CleanShot 2026-03-30 at 13 53 59@2x" src="https://github.com/user-attachments/assets/f483767c-34e0-401c-8089-5b9834fe696a" /> **References** - https://ai-sdk.dev/cookbook/guides/agent-skills Closes AI-508 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added dynamic knowledge loading capability enabling the AI assistant to retrieve on-demand information about PostgreSQL best practices, Row Level Security, Edge Functions, and Realtime. * **Bug Fixes** * Improved search results filtering to exclude error responses in tool outputs. * **Tests** * Enhanced evaluation metrics with knowledge usage scoring. * Expanded test dataset cases to validate knowledge requirement handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Danny White
|
73692b0a4d |
feat(studio): add stuck pausing and restoring escalations (#43368)
## What kind of change does this PR introduce? Bug fix / UX improvement for long-running project transitions. Resolves DEPR-362. ## What is the current behaviour? - `PausingState` does not preserve elapsed time across refreshes, so the stuck escalation can disappear for the same user. - `RestoringState` relies on weaker frontend heuristics and always showed a support CTA in the footer even before the restore was clearly long-running. ## What is the new behaviour? - `PausingState` - Persists a per-project pause start time in local storage so the stuck CTA survives refreshes in the same browser. - Escalates after 10 minutes. - Clears the stored timer when pausing succeeds or fails. - `RestoringState` - Persists a per-project restore start time in local storage so the stuck CTA survives refreshes in the same browser. - Removes the always-visible footer CTA and only escalates once restoration is genuinely long-running. - Computes the long-running threshold from volume size using a shared restore estimate: `max(10, ceil(estimateRestoreTime(sizeGb) * 1.5))`. - Clears the stored timer when restoration succeeds or fails. - Shared changes - Extracts reusable transition timing helpers and restore estimate helpers with unit tests. - Reuses the same restore estimate formula for branch restore timing and restore escalation, so the two do not drift. | `PausingState` | `RestoringState` | | --- | --- | | <img width="1570" height="906" alt="Krosno Toolshed Supabase-C6D7E29F-C38D-43E1-8AF9-C612B6A2FD8D" src="https://github.com/user-attachments/assets/e0bd9434-09b6-4cf6-bffa-07a0ddcdf5db" /> | <img width="1570" height="906" alt="Krosno Toolshed Supabase-51F4763D-B798-4B41-A92D-43B3CF8ECDAF" src="https://github.com/user-attachments/assets/d0e47356-dcc3-42aa-b602-802a35249a16" /> | ## Additional context - This PR intentionally stays frontend-only. - We are not exposing backend lifecycle timestamps here; local storage is the stopgap to improve the same-browser experience now. - If you need to test the frontend blocker states locally, use [`dnywh/chore/depr-362-blocker-preview-mocks`](https://github.com/supabase/supabase/tree/dnywh/chore/depr-362-blocker-preview-mocks) and append one of the following query params to a project URL: - `?mockProjectBlockingState=pausing` - `?mockProjectBlockingState=pausing-long-running` - `?mockProjectBlockingState=restoring` - `?mockProjectBlockingState=restoring-long-running` - I know these two views are quite differently stylistically, and will consolidate later - References DEPR-434 |
||
|
Usama Nadeem
|
55e0b34a18 |
fix(studio): add option to treat empty CSV cells as NULL on import (#43281)
When importing a CSV file, empty cells were always imported as empty strings with no way to import NULL values instead. This adds a "Treat empty cells as NULL" checkbox to the import configuration panel. When enabled, PapaParse's transform option converts empty strings to null before the data is parsed and previewed, so the imported rows correctly contain NULL rather than empty strings. Fixes #43258. ## What kind of change does this PR introduce? Bug fix ## What is the current behavior? Empty cells in a CSV file are always imported as empty strings with no way to import NULL values instead. Fixes #43258. ## What is the new behavior? "Treat empty cells as NULL" checkbox is added to the import configuration panel. When enabled, empty cells are imported as NULL instead of empty strings. Toggling the checkbox instantly re-parses the preview. ## Additional context The fix uses PapaParse's transform option to convert empty strings to null before parsing. Applies to both file uploads and pasted text. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com> |
||
|
Ivan Vasilov
|
ee8eae7309 |
chore: Clean the ui package from next imports (#44278)
This PR moves several components which rely on `next` out of the `ui` package to the `ui-patterns` package. `ui-patterns` package is intented to be imported with specific imports so it's ok if there are components reliant on `next` in there. The `SonnerToaster` component has removed its dependency by requiring a prop for `theme`. |
||
|
Matt Rossman
|
0c5f64fcba |
feat(assistant): upgrade default models to gpt-5.4-nano and gpt-5.3-codex (#44107)
Replaces `gpt-5-mini` and `gpt-5` with `gpt-5.4-nano` and `gpt-5.3-codex` respectively. Clients with stale model IDs in IndexedDB will gracefully reset to the new defaults. While we can technically keep the existing models around, we've [opted](https://supabase.slack.com/archives/C051L8U2EJF/p1774283070517609?thread_ts=1773771991.871669&cid=C051L8U2EJF) to replace them w/ the newer models for simplicity. Basic completion endpoints use `'none'` reasoning level for optimal speed. Rationale for these models is they provide they best balance of intelligence/speed and cost. GPT-5.4-nano is less expensive (0.8x price), faster, and smarter than GPT-5-mini. GPT-5.4-mini would be even smarter but is 3x the price. GPT-5.3-Codex is ~1.4x the price of GPT-5, while GPT-5.4 would be 2x price, but 5.3-Codex is still a big intelligence boost from GPT-5. See [eval comparison](https://www.braintrust.dev/app/supabase.io/p/Assistant/experiments/mattrossman%2Fai-509-v2-upgrade-assistant-models-beyond-gpt-5-family-1774468619?c=master-1774458837&diff=between_experiments), scores are relatively stable and conciseness naturally improves on gpt-5.4-nano. Other change: - Fixed an eval test case to clarify that https://supabase.help is also a correct URL for submitting support ticket, which was unfairly scored as incorrect [here](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=5244cccd-23b2-4f79-9dd2-287f1b40ebad&r=bac9b903-8bde-4c21-99dd-e0ed141c4f9e&s=f248fbf5-75bf-4aab-be0a-87a4298e6d11) I sanity checked the Assistant, natural language filters, and SQL Editor completions on staging preview. References: - https://openai.com/index/introducing-gpt-5-4-mini-and-nano/ - https://openai.com/index/introducing-gpt-5-3-codex/ - https://developers.openai.com/api/docs/pricing Closes AI-509 |
||
|
Matt Rossman
|
adf8b0c67c |
feat(assistant): per-endpoint reasoningEffort + model config cleanup (#43981)
We're exploring support for newer models like [gpt-5.4-nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/) in Assistant. This model doesn't support the `'minimal'` reasoning effort level we use for gpt-5-mini which leads to vague errors. <img width="595" height="263" alt="CleanShot 2026-03-18 at 17 13 05@2x" src="https://github.com/user-attachments/assets/cf7c2370-322d-4a8a-be55-23e680db0aa0" /> Also, we've [previously discussed](https://supabase.slack.com/archives/C0161K73J1J/p1771544464850199?thread_ts=1771493920.775699&cid=C0161K73J1J) that reasoning adds unnecessary latency to otherwise simple AI completion endpoints like `title-v2`. We want more control of reasoning level independent of model/endpoint. This PR aims to solve both problems by: - making reasoning effort configurable on a per-request basis - adding compile-time guardrails to prevent selecting an incompatible reasoning level for models - adding a `DEFAULT_COMPLETION_MODEL` with minimal reasoning that we can update with newer models that support disabling reasoning (independent of Assistant chat model reasoning) Other improvements to our model config logic: - Fixes bug in `onboarding/design.ts` and `assistant.eval.ts` where `providerOptions` was being dropped - `getModel()` now returns a bundled `modelParams` object (spread into AI SDK calls) so `providerOptions` can't be accidentally omitted (this [has happened before](https://supabase.slack.com/archives/C0161K73J1J/p1771518443534309?thread_ts=1771493920.775699&cid=C0161K73J1J)) - Introduces an `ASSISTANT_MODELS` registry as a single source of truth for assistant model config, eliminating hardcoded model IDs across the codebase - Aligns free/pro model conditional logic with `assistant.advance_model` entitlement naming conventions instead of the `isLimited` pattern - Adds `console.error` logging of Assistant stream errors so we can interpret reasoning effort compatibility errors in the future (instead of just opaque "Sorry, I'm having trouble responding right now" card) - Removes unnecessary type casts and generally making the model config logic stricter - Removes pre-existing dead code: `anthropic` provider variant in `GetModelParams` / `PROVIDERS` registry that was never implemented in `getModel()` Now if you try to select an unsupported reasoning level you get a type error: <img width="1306" height="320" alt="CleanShot 2026-03-20 at 14 37 24@2x" src="https://github.com/user-attachments/assets/a6ac234b-5ea5-4d81-8e01-ac4be34a0800" /> And if for some reason an invalid reasoning level slips through, you now get a server-side error surfacing the issue: <img width="1268" height="204" alt="CleanShot 2026-03-20 at 14 58 14@2x" src="https://github.com/user-attachments/assets/aadc1b7a-9495-475f-9741-39979bd27cd7" /> I've tested gpt-5 and gpt-5-mini are still working on the staging preview and verified the models were selected properly in Braintrust logs. Both models are available on my Pro test account, and my Free test account shows the Pro upgrade CTA. Closes AI-446 Closes AI-551 |
||
|
Ali Waseem
|
aa12ae790a |
fix: flatten AI generation schema for filters (#44092)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? OpenAI claims to support recursive schemas with $defs/$ref, but in practice it's unreliable. When Zod's z.lazy() is converted to JSON Schema, it produces recursive $ref entries that OpenAI's structured output frequently rejects with errors like "Recursive reference detected" or "Invalid schema for response_format". Simplify the AI generation schema since we only support AND and don't need the recursion because we don't support nesting of groups. |
||
|
Joshen Lim
|
98b1b79909 |
Chore/shift manual queries into pg meta 04 (#43956)
## Context Shifts all remaining dashboard queries into pg-meta so that we centralize all manually written queries in one place Having them in packages/pg-meta also allows us to write tests for them ## To test Just needs a smoke test on - Role Impersonation - Lints - Data API - Database - Enumerated Types - Integrations - Foreign Data Wrappers - Vault |
||
|
Ali Waseem
|
8a26132bde |
fix(studio): resolve outstanding Sentry issues (#44106)
## Summary Fixes several high-impact Sentry errors reported in production. ### Fixed Issues - **[SUPABASE-APP-EJ3](https://supabase.sentry.io/issues/7356937474/)** — `TypeError: Cannot read properties of undefined (reading 'direct')`. `connectionStringPooler` could be `undefined` when the connection source doesn't match any key in the connection strings map. Added an early return guard in `resolveConnectionString`. - **[SUPABASE-APP-B17](https://supabase.sentry.io/issues/7117468199/)** — `RangeError: Invalid time zone specified: Etc/Unknown`. `dayjs.tz.guess()` returns `"Etc/Unknown"` for some users with misconfigured browser/OS timezones. Added a shared `guessLocalTimezone()` helper that validates the guessed timezone via `Intl.DateTimeFormat` and falls back to UTC. Applied across all 4 call sites. - **[SUPABASE-APP-BCM](https://supabase.sentry.io/issues/7192934901/)** — `TypeError: Cannot convert undefined or null to object`. `Object.entries(definition.properties)` crashed when a JSON schema definition existed but had no `properties` field. Updated the guard to check `definition?.properties` instead of just `definition`. - https://supabase.sentry.io/issues/7357780302/?project=5459134 - https://supabase.sentry.io/issues/7358344652/?project=5459134 - https://supabase.sentry.io/issues/7096737077/?project=5459134 ## Test plan - [ ] Verify connect dialog renders without errors when connection data is still loading - [ ] Verify API docs Entity view handles schema definitions without properties - [ ] Verify charts/tooltips display correct timezone labels --------- Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com> |
||
|
Matt Rossman
|
d29fbf6eb7 |
feat(assistant): upgrade AI SDK v5 → v6 (#43931)
Upgrades `ai` from v5 to v6 and all related packages.
**Package bumps:**
- `ai`: `5.0.52` → `^6.0.116`
- `@ai-sdk/openai`: `2.0.32` → `^3.0.41`
- `@ai-sdk/react`: `2.0.52` → `^3.0.118`
- `@ai-sdk/provider`: `^2.0.0` → `^3.0.8`
- `@ai-sdk/provider-utils`: `^3.0.0` → `^4.0.19`
- `@ai-sdk/amazon-bedrock`: `^3.0.0` → `^4.0.81`
- `@ai-sdk/mcp`: N/A → `^1.0.25`
- `openai`: bumped to `^4.104.0`
- `braintrust`: `3.0.x` → `^3.4.0`
**Breaking change migrations:**
- `generateObject` removed in v6 — migrated 5 API routes to
`generateText` with `Output.object({ schema })`, returning
`result.output`
- `convertToModelMessages` is now async — added `await`
- MCP import path changed: `experimental_createMCPClient` from `ai` →
`createMCPClient` from `@ai-sdk/mcp`
- `openai()` defaults to Responses API — added `store: false` to
provider options for ZDR org compatibility
**Streaming fix:**
Added `Content-Encoding: none` header to `pipeUIMessageStreamToResponse`
calls. Without it, proxy middleware buffers the entire SSE response
before flushing, causing the full reply to appear at once.
**Zero Data Retention fix:**
In recent AI SDK versions, `openai()` default to Responses API instead
of the legacy chat completions API. This produces a 404 from OpenAI with
message `"Items are not persisted for Zero Data Retention organizations.
Remove this item from your input and try again."` The Responses API is
OpenAI's [recommended
endpoint](https://developers.openai.com/api/docs/guides/migrate-to-responses).
This PR adds `store: false` as mentioned in
https://github.com/vercel/ai/issues/10060 to avoid incompatible
persistence attempts.
**References:**
- https://ai-sdk.dev/docs/migration-guides/migration-guide-6-0
-
https://ai-sdk.dev/docs/troubleshooting/streaming-not-working-when-proxied
- https://github.com/vercel/ai/issues/10060
Closes AI-514
Related AI-509
|
||
|
Joshen Lim
|
be26feb9ba |
Chore/shift manual queries into pg meta 03 (#43951)
## Context Shifting more dashboard queries into pg-meta so that we centralize all manually written queries in one place Having them in packages/pg-meta also allows us to write tests for them ## To test Just needs a smoke test on - Table Editor - Fetching entities - Viewing definition - SQL Editor - View ongoing queries - Abort queries - Integrations - Queues - Database - Migrations -Triggers (Updating) |
||
|
Charis
|
fe0da16820 |
refactor: move /incident-banner to app router (#43930)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Refactor ## What is the current behavior? The `/incident-banner` endpoint is implemented using the Pages Router. ## What is the new behavior? The `/incident-banner` endpoint is moved to the App Router, enabling caching of the upstream fetch. This does not turn on the querying from the frontend yet, making that a separate PR so we can revert easily if needed. ## Additional context |
||
|
Danny White
|
fd17b246e1 |
fix(studio): tighten webhook endpoint validation (#43892)
## What kind of change does this PR introduce? Bug fix. ## What is the current behavior? Webhook endpoint validation is inconsistent across Studio forms. The webhook sheet accepts incomplete hostnames like `https://webhook`, event type validation is not surfaced clearly, and HTTP endpoint validation differs between webhooks, log drains, cron jobs, and database hooks. ## What is the new behavior? - Tightens webhook endpoint URL validation and rejects incomplete hostnames while still allowing localhost and IP-based endpoints. - Surfaces the Event types validation through the standard form error styling and highlights the existing accordion item border when invalid. - **Extracts a shared HTTP endpoint URL validator and reuses it in webhooks, log drains, cron jobs, and database hooks.** - Adds focused regression tests for the webhook sheet and the shared/consumer validation paths. | After | | --- | | <img width="1728" height="997" alt="Webhooks Settings AWS Healthy Toolshed Supabase-CB0D999C-D0BF-47AA-A10F-342A2E328DF9" src="https://github.com/user-attachments/assets/bcbe4876-f9a7-497a-b288-460087a65546" /> | ## To test Form behaviour (in particular URL validation) on: - Webhook endpoint - Log drains - Cron jobs - Database hooks |
||
|
Charis
|
a4641d0b9f |
refactor: move /incident-status to app router (#43881)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Refactor ## What is the current behavior? `/incident-status` is handled via Pages Router. ## What is the new behavior? `/incident-status` is handled via App Router, enabling use of Vercel Data Cache to cache the upstream fetch. ## Additional context Adding the first App Router route handler triggered `next typegen` (run as `pretypecheck`) to generate `.next/dev/types/validator.ts`, which imports all route files and expanded the type-checked graph. This surfaced pre-existing `null`-safety errors in: - `components/grid/SupabaseGrid.utils.ts` — `useSearchParams()` result - `components/layouts/ProjectLayout/UpgradingState/index.tsx` — `useSearchParams()` result - `pages/project/[ref]/sql/quickstarts.tsx` — `useParams()` result - `pages/project/[ref]/sql/templates.tsx` — `useParams()` result These are fixed with optional chaining. The `tsconfig.json` change (adding `.next/dev/types/**/*.ts` to `include`) is auto-generated by Next.js and committed as correct behavior. |
||
|
Matt Rossman
|
25036af80e |
fix(assistant): sanitize backslash-escaped apostrophes in SQL (#43728)
Fix for the LLM occasionally generating MySQL-style `\'` escapes in SQL, which are invalid in PostgreSQL. Example trace where this happened in the wild: ([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/review?tab=experiment&r=5fcf1b12-8584-455c-9e9a-bdc0fa3ed21c&s=5fcf1b12-8584-455c-9e9a-bdc0fa3ed21c&o=0627ada8-b567-4117-9fe8-49d847cb73a7&review=1)) **Changes** - Adds `fixSqlBackslashEscapes` to convert `\'` → `''` before SQL is executed - Unit tests + adversarial eval dataset case Compare the results of the adversarial test case: - `master`: 0% SQL Validity ([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Dev%20(mattrossman%2FAssistant)/trace?object_type=experiment&object_id=b469cbf7-4d6f-429c-9819-6c4099294123&r=dce5a29b-2fde-44c3-80f8-4e14d1f657c0&s=dce5a29b-2fde-44c3-80f8-4e14d1f657c0)) - This branch: 100% SQL Validity ([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=160e9ce0-e320-4f6d-8aa7-c5ad7e01fbd2&r=d75ef0e3-90ed-42a7-9ef3-8bf69592f193&s=0eeca492-dbe6-451e-8d81-127caff30320)) Closes AI-400 |
||
|
Etienne Stalmans
|
b79a645f4f |
fix: escape regex control character (#43806)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Bug fix ## What is the current behavior? regex control character, `.` is not escaped. ## What is the new behavior? Escapes control characters and makes regex a little stricter. Use regex literal |
||
|
Ivan Vasilov
|
9fa96977be |
chore: Minor prettier fixes (#43849)
This PR fixes some prettier issues: - Bump and unify all prettier versions to 3.7.3 across teh whole repo - Bump the SQL prettier plugin - When running `test:prettier`, check `mdx` files also - Run the new prettier format on all files --------- Co-authored-by: Joshen Lim <joshenlimek@gmail.com> |
||
|
Pedro Rodrigues
|
62426253c3 |
fix: pass exposedSchemas to getLints in MCP advisor operations (#43790)
## Summary - MCP `getSecurityAdvisors` and `getPerformanceAdvisors` now pass `exposedSchemas` to `getLints`, fixing empty advisor results in local/self-hosted environments - Extracts `DEFAULT_EXPOSED_SCHEMAS` constant shared between the MCP handler and the `run-lints` API route (cc @joshenlim related https://github.com/supabase/supabase/pull/40043) - Adds unit tests for `enrichLintsQuery` and the MCP advisor operations ## The bug The MCP advisor tools (`get_advisors`) return empty arrays (`[]`) for **all** scenarios when running locally via `supabase start`. No security or performance advisors are surfaced, even when the database has clear issues (e.g., tables with no RLS). ### Root cause In `lib/api/self-hosted/mcp.ts`, both `getSecurityAdvisors` and `getPerformanceAdvisors` call `getLints({ headers })` **without passing `exposedSchemas`**: ```typescript // Before (mcp.ts:131) const { data, error } = await getLints({ headers }) ``` When `exposedSchemas` is `undefined`, `enrichLintsQuery` in `lints.ts` skips the `SET LOCAL pgrst.db_schemas = '...'` SQL statement: ```typescript // lints.ts:23 ${!!exposedSchemas ? `set local pgrst.db_schemas = '${exposedSchemas}';` : ''} ``` Without this GUC being set, the splinter SQL queries filter results using `current_setting('pgrst.db_schemas', 't')` — which returns an empty string in local environments. Every schema-filtered lint matches no schemas and returns zero rows. ### Why this only affects local/self-hosted environments In **hosted Supabase**, PostgREST sets the `pgrst.db_schemas` GUC on its own database connections based on the project's API configuration. The Studio MCP server in production reads the same project configuration, so the GUC is already available. **Locally**, PostgREST runs in a separate Docker container and only sets this GUC on _its own_ connections. Studio connects directly to PostgreSQL (bypassing PostgREST), so `current_setting('pgrst.db_schemas', 't')` returns `''`. The HTTP API endpoint (`/api/platform/.../run-lints`) already worked because `run-lints.ts` passes `exposedSchemas: 'public, storage'` — this parameter was simply never added to the MCP code path. ## How we verified the fix ### 1. Tests written to fail against the previous code We wrote two test files that target the exact bug: **`tests/unit/lints/enrichLintsQuery.test.ts`** — validates the SQL generation: - Confirms `SET LOCAL pgrst.db_schemas` is included when `exposedSchemas` is provided - Confirms it's omitted when `undefined` or empty (documenting current behavior) **`tests/unit/lints/mcp-advisors.test.ts`** — validates the MCP operations: - Asserts `getSecurityAdvisors` passes `exposedSchemas` to `getLints` - Asserts `getPerformanceAdvisors` passes `exposedSchemas` to `getLints` - Asserts the value matches `DEFAULT_EXPOSED_SCHEMAS` - Verifies SECURITY/PERFORMANCE category filtering still works Before the fix, the two `exposedSchemas` assertions failed: ``` FAIL getSecurityAdvisors should pass exposedSchemas to getLints → expected { Object (headers) } to have property "exposedSchemas" FAIL getPerformanceAdvisors should pass exposedSchemas to getLints → expected { Object (headers) } to have property "exposedSchemas" ``` ### 2. Fix applied, all tests pass After adding `exposedSchemas: DEFAULT_EXPOSED_SCHEMAS` to both MCP operations, all 14 tests pass (9 new + 5 existing MCP tests). ## Test plan run `supabase start`, create a table without RLS, call `get_advisors` via MCP — should return `rls_disabled_in_public` lint --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Joshen Lim
|
7d1b38f804 |
Float up error code from status page into incident-status endpoint (#43737)
## Context Just a nit change to float the status code from status page API into incident-status endpoint so its clearer what the error is from the network tab --------- Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com> |
||
|
Chris Stavitsky
|
e217f747bd |
feat(studio) instrument sentry on table creation flow (#43673)
(working with @charislam @awaseem on this one) ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES Per [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md): - ✅ `pnpm --filter studio build` - ✅ `pnpm test:prettier` - ✅ `pnpm format` ## What kind of change does this PR introduce? Instruments `studio` app's table creation flow with Sentry. ## What is the current behavior? The table creation flow isn't instrumented in detail, which means we have less visibility into understanding performance problems or errors within this specific flow. ## What is the new behavior? The table creation flow is instrumented in detail. This instrumentation allows for Sentry dashboards like this one (created based on locally generated Supabase data), to help monitor anything going wrong with this critical flow: <img width="1915" height="1131" alt="image" src="https://github.com/user-attachments/assets/e2c5380e-7b00-41c1-9109-3ee4e47ff63b" /> --------- Co-authored-by: Ali Waseem <waseema393@gmail.com> |
||
|
Jordi Enric
|
ec26943390 |
feat: improve db overload debugging UX (#43564)
When the dashboard hits a DB connection timeout, users currently see a
raw error message with no
path forward. This PR adds an inline troubleshooting system that detects
known error types and
surfaces contextual next steps — restart the DB, read the docs, or debug
with AI.
## Changes
- New ErrorDisplay component (packages/ui-patterns) — styled error card
with a title, monospace error
block, optional troubleshooting slot, and a "Contact support" link that
always renders. Accepts
typed supportFormParams to pre-fill the support form.
- Error classification in handleError (data/fetchers.ts) — on every API
error, the message is tested
against ERROR_PATTERNS. If matched, handleError throws a typed subclass
(ConnectionTimeoutError
extends ResponseError) instead of a plain ResponseError. Stack traces
now show the exact error
class. All existing instanceof ResponseError checks continue to work.
- ErrorMatcher component — reads errorType from the thrown class
instance, does an O(1) lookup into
ERROR_MAPPINGS, and renders the matching troubleshooting accordion as
children of ErrorDisplay.
Falls back to plain ErrorDisplay for unclassified errors.
- Connection timeout mapping — first error type wired up, with three
troubleshooting steps: restart
the database, link to the docs, and "Debug with AI" (opens the AI
assistant sidebar with a
pre-filled prompt).
- Telemetry — three new typed events track when the troubleshooter is
shown, when accordion steps are
toggled, and which CTAs are clicked.
## Adding a new error type
1. Add a class to types/api-errors.ts
2. Add { pattern, ErrorClass } to data/error-patterns.ts
3. Create a troubleshooting component in errorMappings/
4. Add an entry to error-mappings.tsx
|
||
|
Ivan Vasilov
|
c5b6695380 |
fix: Remove auth from /incident-status and /incident-banner endpoints (#43751)
|
||
|
Charis
|
befc817f94 |
feat: version who-knows-what of incident banner (#43726)
Feature ## What is the current behavior? Incident banner logic depends on StatusPage and Supabase project for metadata. ## What is the new behavior? New incident banner logic that depends only on incident.io. Displays in non-production environments for now because I haven't wired up the rest of the workflow. This is just to allow a total end-to-end testing/playground for test incidents <-> Slack <-> preview dashboard for people to try out the UX. ## Additional context You can test using my [test incident](https://app.incident.io/supabase/incidents/405). This has severity minor, so the preview site should have a banner. Toggle to informative, hard refresh dashboard with cache off, and banner should disappear. Toggle back to minor, hard refresh without cache again, and banner should reappear. Same thing if you edit the "Banner shown" field from 1 to -1 and back. |
||
|
Alaister Young
|
9652e8c4d3 |
[SEC-385] feat(studio): Add default privileges for new entities toggle (#43583)
Adds a new toggle in: <img width="1161" height="356" alt="Screenshot 2026-03-10 at 17 17 06" src="https://github.com/user-attachments/assets/b09ac1aa-a8f5-4fb4-8771-f113b140eac8" /> Other changes: - form submissions with no table/function changes were failing because an empty string got passed to executeSql. Added an early return when there's nothing to execute To test: - Ensure the form is still in working order - Create some tables and functions with the toggle on add off and make sure your selected default applies |
||
|
Matt Rossman
|
f49ed14dfb |
fix(assistant): preserve template URL placeholders in responses (#43495)
Fixes a rendering bug where template URLs like `https://<project-ref>.supabase.co/...` were displayed incorrectly in Assistant responses. The markdown parser was treating `<project-ref>` as a tag and silently stripping it. - Adds `wrapPlaceholderUrls()` preprocessing step that auto-wraps bare URLs containing `<kebab-case>` placeholders in backticks before passing to the markdown renderer - Strengthens the prompt instruction to explicitly say "always format template URLs as inline code using backticks" Sample prompt: ``` What are the OAuth2 endpoints for Supabase projects? List the authorization, token, and JWKS URLs. ``` | Before | After | |--------|--------| | <img width="1314" height="841" alt="CleanShot 2026-03-06 at 15 32 13@2x" src="https://github.com/user-attachments/assets/55eaa0c6-39fb-48e0-91a0-31903021a4c9" /> | <img width="1421" height="827" alt="CleanShot 2026-03-06 at 15 31 37@2x" src="https://github.com/user-attachments/assets/5335c71a-19c7-44a0-b027-6f2efd76eb8c" /> | Closes AI-470 |
||
|
Francesco Sansalvadore
|
23c827bdda |
feat: nested mobile nav menu (#43333)
- consolidate `top level menu` and `contextual menu` into nested menu on mobile - remove legacy mobile submenu |
||
|
Matt Rossman
|
517171b246 |
feat(assistant): online evals support and CI workflows (#43194)
Lays groundwork for online evals on Assistant chat logs. https://www.braintrust.dev/docs/observe/score-online ### Changes - New workflows: - `braintrust-scorers-deploy.yml` keeps prod scorers in sync on push to `master` - `braintrust-preview-scorers-deploy.yml` deploys preview scorers to the staging project for PRs labeled `preview-scorers`, posting a comment with scorer links ([example](https://github.com/supabase/supabase/pull/43194#issuecomment-4000097222)) - `braintrust-preview-scorers-cleanup.yml` deletes preview scorers when the PR is closed ([example](https://github.com/supabase/supabase/pull/43194#issuecomment-4000749847)) - Adds `evals/scorer-online.ts` entry point invoked with `pnpm scorers:deploy`, registering scorers for online evals in the Braintrust "Assistant" project - Refactors scorer code to separate online-compatible scorers (`scorer-online.ts`) from WASM-dependent ones (`scorer-wasm.ts`) - "URL Validity" scorer now only checks Supabase domains to prevent requests to untrusted origins - Span `input` is now shaped `{ prompt: string }` instead of plain `string` for compatibility with offline eval scorers - Env vars `BRAINTRUST_STAGING_PROJECT_ID` and `BRAINTRUST_PROJECT_ID` configured in GitHub repo settings - `generateAssistantResponse` now uses `startSpan` + `withCurrent` instead of `traced()` to manually manage the root span lifecycle — this ensures `onFinish` logs output to the span _before_ `span.end()` is called, which is when Braintrust triggers scoring automations ### Online Scorers We share scoring logic across offline and online evals, but some of our scorers aren't transferrable to an "online" setting due to runtime challenges or ground truth requirements. **Supported** - Goal Completion - Conciseness - Completeness - Docs Faithfulness - URL Validity **Unsupported** - Correctness (requires ground truth output) - Tool Usage (requires ground truth requiredTools) - SQL Syntax (uses libpg-query WASM) - SQL Identifier Quoting (uses libpg-query WASM) ### How to use these scorers Going forward if you want to add/edit online eval scorers, add the `preview-scorers` label to a PR. This deploys scorers to the [Assistant (Staging Scorers)](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)?v=Overview) project in Braintrust with branch-specific slugs, and comments on the PR ([example](https://github.com/supabase/supabase/pull/43194#issuecomment-4000097222)). From the Braintrust dashboard you can "Test" the scorer with traces from any project. <img width="1866" height="528" alt="CleanShot 2026-03-05 at 15 15 00@2x" src="https://github.com/user-attachments/assets/4f15cebc-3f2d-4e8a-9ee2-fe8ef7bf4199" /> Once merged, scorers are deployed to the primary [Assistant](https://www.braintrust.dev/app/supabase.io/p/Assistant) project, and preview scorers are deleted from the staging project. Down the road, scorers on the Assistant project will run automatically on a sample of production traces. Closes AI-437 |