mirror of
https://github.com/supabase/supabase.git
synced 2026-07-22 15:37:14 -04:00
create-pull-request/patch
14 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joshen Lim
|
dc23320e43 |
Add sentry capture exception to apiWrapper (#47804)
## Context As per PR title - also adjusts the imports for files consuming `apiWrapper` to remove the default export for `apiWrapper` Have tested locally by throwing an error in one of the API routes - verified that the event shows up on Sentry <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * API errors are now captured in Sentry before returning server error responses, improving production visibility while keeping endpoint behavior the same. * **Tests** * Added coverage to confirm rejected handler executions are reported to Sentry and return the expected HTTP 500 JSON payload. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jordi Enric
|
2aa1b52234 |
feat(studio): add feature to rewrite queries DEBUG-145 (#47266)
## Problem Moving the Logs Explorer to ClickHouse means users' saved BigQuery queries no longer run. <img width="2430" height="1010" alt="CleanShot 2026-06-29 at 11 36 04@2x" src="https://github.com/user-attachments/assets/ae0ab155-7d3d-4ae9-81c3-22bf3a88cf8c" /> ## Fix Rewrite the query with AI instead of a SQL transpiler. AI handles the long tail of nested fields and dialect differences far better than a rule-based rewriter, and it needs no extra runtime dependency. - `rewriteLogsSqlWithAI` posts the current query to `/api/ai/code/complete` with `dialect: 'clickhouse'`. The endpoint skips the Postgres schema and best-practices for that dialect and uses logs-specific instructions and model so the output is ClickHouse logs SQL (FROM `logs` + `source` filter, no `unnest` joins, nested fields read from `log_attributes['...']`). - The query's `source` is detected and its real `log_attributes` keys are fetched and passed to the model, so it maps to exact paths instead of guessing. - The rewrite runs in the background and is proposed as a side-by-side accept/discard diff in the editor. The AI Assistant panel is not opened. - Entry points: a banner shown only for legacy-looking queries (dismissal persisted), and a "Fix Query" button next to Field Reference. - The Field Reference drawers discover `log_attributes` keys from real data so the listed fields match what the source actually emits. ## Dependencies Built on top of #47265 (Logs Explorer -> OTEL endpoint) — that is the base branch of this PR. Merge #47265 first. Behind `otelLegacyLogs` (off by default). Part of DEBUG-145 (split from #47087). ## How to test - Open the Logs Explorer with a BigQuery logs query (the templates have some), click "Fix Query", and confirm the diff shows valid ClickHouse SQL. Accept it and confirm the applied query runs. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an OTEL legacy logs workflow (behind a feature flag) with an interactive banner and a “Fix Query” ClickHouse rewrite action, including an accept/discard diff review overlay. * Introduced OTEL-aware field reference rendering with dynamic discovery of `log_attributes` keys and updated OTEL source insertion behavior. * Enabled dialect-aware SQL completion for ClickHouse logs, using logs-specific instructions and output constraints. * **Bug Fixes** * Improved rewrite flow validation and handling, including log source detection and cleanup of AI-generated SQL formatting. * **Tests** * Added Vitest coverage for rewrite prompt generation, detection/classification utilities, SQL fence stripping, OTEL field mapping, and OTEL log attribute key discovery. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Joshen Lim <joshenlimek@gmail.com> |
||
|
Joshen Lim
|
1baaded0bb |
Consolidate execute-sql-query into execute-sql-mutation (#46944)
## Context Just some clean up as I was going through stuff - `useExecuteSqlQuery` is deprecated and not used at all - As such `execute-sql-query` is technically irrelevant, the more relevant file is `execute-sql-mutation` - Hence opting to consolidate `execute-sql-query` into `execute-sql-mutation` - Also removing `ExecuteSqlError` since its just re-exporting the `ResponseError` type There's a lot of file changes but its essentially just updating the importing statements across the files |
||
|
Matt Rossman
|
d143571586 |
feat(assistant): trace-level scorers + server-side tool execution with needsApproval (#45654)
## Motivation When Assistant runs a potentially destructive tool like `execute_sql`, it stops the LLM request and prompts for client-side approval and execution of the tool. After approval, a second request kicks off under a separate trace. This has made scoring and [Topics](https://www.braintrust.dev/blog/topics) classification challenging, as the generated `output` is split across stateless requests. The [span-level scoring](https://www.braintrust.dev/docs/evaluate/custom-code#score-spans) approach we've used thusfar (after the LLM call, we massage the result into an `output` payload that's stuck onto the root span) has been cumbersome and led to invalid scores / topics where only part of the assistant response is considered. It's also inefficient, as we're duplicating potentially large info (like the `search_docs` output) that already exists within the trace. An alternative to scoring spans is to [score traces](https://www.braintrust.dev/docs/evaluate/custom-code#score-traces). Braintrust [best practices](https://www.braintrust.dev/docs/evaluate/score-online#best-practices) advise: > Use span scope for evaluating individual operations or outputs. Use trace scope for evaluating multi-turn conversations, overall workflow completion, or when your scorer needs access to the full execution context. We've also received [direct guidance](https://supabase.slack.com/archives/C05QYJBLX89/p1777925770927149?thread_ts=1777905716.911979&cid=C05QYJBLX89) from their team to use this approach. ## Changes Migrates eval scorers from custom `AssistantEvalOutput` shape to trace-level scoring via `trace.getThread()` / `trace.getSpans()`, with thread parsing that scores the full latest Assistant turn and passes prior conversation separately where relevant. Moves `execute_sql` and `deploy_edge_function` from client-side execution after approval to AI SDK `needsApproval` + server-side `execute()`. SQL results returned to the model are gated by AI opt-in level, so row data is only included with `schema_and_log_and_data`; otherwise the tool returns the no-data-permissions sentinel. Adds `metadata.isFinalStep` to disambiguate multiple LLM requests within an "assistant" turn due to tool call requests/responses. For online evals, this means we should configure automations to only score traces with `metadata.isFinalStep = true` to ensure we're judging the complete generated response. Other minor kaizen changes: - Renamed `promptProviderOptions` to `systemProviderOptions` to clarify that this is associated with the "system" message and disambiguate from the root `providerOptions` - Adds `evals/trace-utils.ts` to handle Zod validation of the `unknown` span shapes from Braintrust, to more easily access typed inputs/output on tool spans. - Bumps AI SDK floor version `^6.0.116` → `^6.0.174` - Tweaked the "Conciseness" scorer to not unfairly dock points for the new `[called tool_name]` labels in serialized assistant response ## Verification In the studio staging build, I asked Assistant to create a todos table with 3 sample todos. I manually approved the `execute_sql` call and saw Assistant generate text before & after the call. In Braintrust I verified two traces were produced (see [filtered logs](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs?v=Staging&tvt=trace&search={%22filter%22:[{%22text%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22label%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22originType%22:%22btql%22},{%22text%22:%22%2560Chat%2520ID%2560%2520%253D%2520%25221cb2ac45-e5e7-458c-9da4-3bf6863b8842%2522%22,%22label%22:%22Chat%2520ID%2520equals%25201cb2ac45-e5e7-458c-9da4-3bf6863b8842%22,%22originType%22:%22form%22}]})), the first with `metadata.isFinalStep = false` and the second with `metadata.isFinalStep = true`. In the Braintrust staging scorers, I ran the preview Completeness scorer on the second trace and verified it sees the complete Assistant response including markers for tool calls ([link to trace](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)/trace?object_type=project_logs&object_id=b5214b62-ad1e-4929-9d5b-40b1daebe948&r=0ed0a4f8-8aff-4a34-bb1d-1df1d88a5070&s=ff9015f8-6bf7-4ab3-83a9-ca4e69e27e82)) <img width="1193" height="960" alt="CleanShot 2026-05-07 at 11 27 10@2x" src="https://github.com/user-attachments/assets/509d4858-c3a1-4068-986d-3aa4d5617d1a" /> I also tested the `deploy_edge_function` workflow and verified it still prompts for permission and warns on deployment of existing functions. **References** - https://www.braintrust.dev/docs/evaluate/custom-code#score-traces - https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval Supercedes https://github.com/supabase/supabase/pull/45556 and https://github.com/supabase/supabase/pull/45339 Closes AI-473 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tool actions (SQL execution, edge-function deploy) now require explicit user Approve/Deny before proceeding. * **Improvements** * Assistant pauses for approval responses before sending follow-ups, giving clearer control over risky actions. * Deploy/replace flows show confirmation and clearer replace warnings. * Evaluation/scoring updated to use richer trace data for more accurate assistant performance signals. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Matt Rossman
|
8f69a10cc9 |
fix(studio): reliable schema-aware SQL editor AI completions (#44730)
A variety of fixes and improvements to the Cmd+K AI completions endpoint in the [SQL Editor](https://supabase.com/dashboard/project/_/sql/new): - Pre-load table definitions for the public schema and any other schemas referenced in the editor, so the model has real column names without needing to fetch them dynamically - Replace the generic tool suite with a single streamlined `getSchemaDefinitions` tool the model can still call to look up additional schemas on demand without behavior differences across platform & self-hosted - Swap generic chat system prompt for a purpose-built `COMPLETION_PROMPT`; fix role (`assistant` → `user`) for consistency with other endpoints - Validate and type the request body with `zod`, which was previously untyped (`any`) - Improve Cmd+K behavior when nothing is selected — use the full editor content as context, return the complete query rather than just the changed fragment, and switch to a generation mode when the editor is blank - Escape single quotes in schema names when fetching entity definitions in `pg-meta` to prevent schema names from breaking out of the SQL string and injecting arbitrary content into the prompt ## Before Before, the SQL Editor would often hallucinate tables / columns that don't exist in the user's database making it less helpful if you don't know the exact table/column names. Even with maximum Assistant opt-in level on the org, it would often fail to call the necessary tools to gather database context. <img width="5062" height="1522" alt="image" src="https://github.com/user-attachments/assets/fbe1130f-6b5a-41a8-99d7-7268880af188" /> <img width="2540" height="658" alt="image" src="https://github.com/user-attachments/assets/a31c2967-7751-4fce-a9b7-60bd77660b1a" /> Sometimes it also silently fails and generates empty queries: <img width="1352" height="398" alt="CleanShot 2026-04-09 at 17 46 06@2x" src="https://github.com/user-attachments/assets/e17c103a-d47d-47e6-8c2e-101f0fae5651" /> Or echos back the user's prompt: <img width="1368" height="282" alt="CleanShot 2026-04-09 at 23 04 56@2x" src="https://github.com/user-attachments/assets/7dff6e64-f54e-45b5-8e86-5399e5a2fe41" /> ## After In this example, the completion correctly interpreted my request for "completed" todos as a query on the `completed_foo` column in my `public` schema, instead of assuming existence of a `completed` column. <img width="1452" height="838" alt="CleanShot 2026-04-09 at 17 43 13@2x" src="https://github.com/user-attachments/assets/7a575589-78b4-448d-810a-0330ff08ef8b" /> In this example, the completion was correctly aware of an `other` schema because it was detected in my existing query. I didn't have to select the text, it included the full query in context when unselected. Notice how it correctly used the `is_done` column when I asked for "completed" cakes: <img width="1372" height="534" alt="CleanShot 2026-04-09 at 17 39 07@2x" src="https://github.com/user-attachments/assets/e6b7eb6f-f3e8-4fa1-90a3-b5e34ddc14e4" /> Supersedes #44151 Closes AI-544 |
||
|
Charis
|
205cbe7d26 | chore(studio}: enforce import order, remove bare import specifiers (#44585) | ||
|
Matt Rossman
|
8aeacc6152 |
feat(assistant): disable Braintrust tracing for EU regions and DPA signers (#44504)
**Changes** - Extracted tracing conditional to an `isTracingAllowed` helper with unit tests (the function is simple but sensitive hence the extra testing precaution) - Disables Braintrust tracing for projects in EU database regions (region prefix `eu-`) to address GDPR data residency concerns - Disables Braintrust tracing for orgs whose owners have signed the previous DPA, as a stopgap during the 30-day notice period for the updated DPA that adds Braintrust as a subprocessor - Refactored `org-ai-details.ts` → `ai-details.ts`, splitting `getOrgAIDetails` into separate org and project helpers to cleanly scope the EU-region check at the project level DPA check uses the newly added `/documents/dpa-signed` endpoint from https://github.com/supabase/platform/pull/31060. This PR includes regenerated `api.d.ts` and `platform.d.ts` from running `pnpm codegen` in `packages/api-types` to get type safety on this new endpoint. Note tracing is still yet to be activated in production, this is a preparatory step. **To verify** Send a chat message and check for the `x-braintrust-span-id` response header on `POST /api/ai/sql/generate-v4` — it should be absent for DPA-signed orgs or EU-region projects, and present otherwise. <img width="3594" height="1992" alt="CleanShot 2026-04-03 at 14 28 58@2x" src="https://github.com/user-attachments/assets/4c91d7ad-2604-4531-a78e-dedf41632fa5" /> If you have access to the Braintrust dashboard, you can also verify whether logs are produced or not in the Assistant project there. Closes AI-570 Closes AI-569 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tracks organization DPA signing and detects EU-region projects * Assistant tracing now follows a combined compliance policy (HIPAA addon, DPA, project sensitivity, region) * Added helpers to fetch org and project AI details * **Documentation** * Expanded API docs with additional examples and clarified parameter descriptions * Added response schemas for subscription preview and document status * **Tests** * Added/updated tests covering DPA/region behavior and tracing policy enforcement <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Matt Rossman
|
adf8b0c67c |
feat(assistant): per-endpoint reasoningEffort + model config cleanup (#43981)
We're exploring support for newer models like [gpt-5.4-nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/) in Assistant. This model doesn't support the `'minimal'` reasoning effort level we use for gpt-5-mini which leads to vague errors. <img width="595" height="263" alt="CleanShot 2026-03-18 at 17 13 05@2x" src="https://github.com/user-attachments/assets/cf7c2370-322d-4a8a-be55-23e680db0aa0" /> Also, we've [previously discussed](https://supabase.slack.com/archives/C0161K73J1J/p1771544464850199?thread_ts=1771493920.775699&cid=C0161K73J1J) that reasoning adds unnecessary latency to otherwise simple AI completion endpoints like `title-v2`. We want more control of reasoning level independent of model/endpoint. This PR aims to solve both problems by: - making reasoning effort configurable on a per-request basis - adding compile-time guardrails to prevent selecting an incompatible reasoning level for models - adding a `DEFAULT_COMPLETION_MODEL` with minimal reasoning that we can update with newer models that support disabling reasoning (independent of Assistant chat model reasoning) Other improvements to our model config logic: - Fixes bug in `onboarding/design.ts` and `assistant.eval.ts` where `providerOptions` was being dropped - `getModel()` now returns a bundled `modelParams` object (spread into AI SDK calls) so `providerOptions` can't be accidentally omitted (this [has happened before](https://supabase.slack.com/archives/C0161K73J1J/p1771518443534309?thread_ts=1771493920.775699&cid=C0161K73J1J)) - Introduces an `ASSISTANT_MODELS` registry as a single source of truth for assistant model config, eliminating hardcoded model IDs across the codebase - Aligns free/pro model conditional logic with `assistant.advance_model` entitlement naming conventions instead of the `isLimited` pattern - Adds `console.error` logging of Assistant stream errors so we can interpret reasoning effort compatibility errors in the future (instead of just opaque "Sorry, I'm having trouble responding right now" card) - Removes unnecessary type casts and generally making the model config logic stricter - Removes pre-existing dead code: `anthropic` provider variant in `GetModelParams` / `PROVIDERS` registry that was never implemented in `getModel()` Now if you try to select an unsupported reasoning level you get a type error: <img width="1306" height="320" alt="CleanShot 2026-03-20 at 14 37 24@2x" src="https://github.com/user-attachments/assets/a6ac234b-5ea5-4d81-8e01-ac4be34a0800" /> And if for some reason an invalid reasoning level slips through, you now get a server-side error surfacing the issue: <img width="1268" height="204" alt="CleanShot 2026-03-20 at 14 58 14@2x" src="https://github.com/user-attachments/assets/aadc1b7a-9495-475f-9741-39979bd27cd7" /> I've tested gpt-5 and gpt-5-mini are still working on the staging preview and verified the models were selected properly in Braintrust logs. Both models are available on my Pro test account, and my Free test account shows the Pro upgrade CTA. Closes AI-446 Closes AI-551 |
||
|
Matt Rossman
|
3fcb6cbe2c |
fix(studio): pass providerOptions to all AI SDK calls (#43031)
`reasoningEffort: 'minimal'` was [configured](https://github.com/supabase/supabase/blob/d5cc70560d/apps/studio/lib/ai/model.utils.ts#L55-L59) in the provider registry but `getModel()` returns it as a separate value that callers must destructure and forward — and 7 of 8 endpoints weren't doing so. This meant `gpt-5-mini` (a reasoning model) was running at default reasoning effort for every call. This PR destructures `providerOptions` from `getModel()` and passes it to `generateObject`/`generateText` in all affected endpoints. ## Benchmark (local, median of 5 runs) | Endpoint | Before (s) | After (s) | Speedup | |----------|-----------|----------|---------| | title-v2 | 7.0 | 1.9 | 3.7x | | cron-v2 | 2.3 | 0.9 | 2.6x | | filter-v1 | 5.8 | 2.2 | 2.6x | | feedback/classify | 3.5 | 0.9 | 3.9x | | feedback/rate | 2.9 | 0.9 | 3.2x | `code/complete` and `policy` also received the fix but aren't benchmarked here as they require a live DB connection and use multi-step tool calls (separate latency concern tracked in AI-419). To test the SQL naming, visit the SQL Editor in sidebar, add some SQL like: ```sql create table todos ( id serial primary key, task text not null, completed boolean default false ); ``` Right click on the snippet, "Rename" and "Rename with Supabase AI" Closes AI-443 |
||
|
Greg Richardson
|
9820707d71 |
feat: local mcp server (#38797)
* feat: local mcp server * feat(local-mcp): implement migrations * fix: remove unsupported mcp args * feat(local-mcp): tests * fix(local-mcp): packages to adhere to minimumReleaseAge * fix(mcp): import path for createSupabaseApiPlatform * fix(local-mcp): move tests out of pages/api dir * refactor: self-hosted execute sql logic * fix: deps |
||
|
Saxon Fletcher
|
bf004b8a1d |
Choose provider and model (#38267)
* choose provider and model * fix test * Fix lint * Nit clean up * add provider options and remove anthropic * remove package * use chat model and update prompt * prompt rendering * default to pro * fix test * low * remove log * remove redundant part.text --------- Co-authored-by: Joshen Lim <joshenlimek@gmail.com> |
||
|
Saxon Fletcher
|
cddc472705 |
Test GPT-OSS (#37710)
* try a really long context window to maximize caching
* update examples
* attempt to update packages and useChat
* update endpoints
* update zod
* zod
* update to v5
* message update
* Revert "zod"
This reverts commit
|
||
|
Saxon Fletcher
|
7b941a0fd7 |
Attempt to fix completion (#37787)
* attempt to fix completion * use experimental output |
||
|
Saxon Fletcher
|
d60aceb562 |
Prompt and tool refactoring (#37500)
* try a really long context window to maximize caching
* update examples
* attempt to update packages and useChat
* update endpoints
* update zod
* zod
* update to v5
* message update
* Revert "zod"
This reverts commit
|