Commit Graph

26 Commits

Author SHA1 Message Date
Jordi Enric 2aa1b52234 feat(studio): add feature to rewrite queries DEBUG-145 (#47266)
## Problem

Moving the Logs Explorer to ClickHouse means users' saved BigQuery
queries no longer run.
<img width="2430" height="1010" alt="CleanShot 2026-06-29 at 11 36
04@2x"
src="https://github.com/user-attachments/assets/ae0ab155-7d3d-4ae9-81c3-22bf3a88cf8c"
/>

## Fix

Rewrite the query with AI instead of a SQL transpiler. AI handles the
long tail of nested fields and dialect differences far better than a
rule-based rewriter, and it needs no extra runtime dependency.

- `rewriteLogsSqlWithAI` posts the current query to
`/api/ai/code/complete` with `dialect: 'clickhouse'`. The endpoint skips
the Postgres schema and best-practices for that dialect and uses
logs-specific instructions and model so the output is ClickHouse logs
SQL (FROM `logs` + `source` filter, no `unnest` joins, nested fields
read from `log_attributes['...']`).
- The query's `source` is detected and its real `log_attributes` keys
are fetched and passed to the model, so it maps to exact paths instead
of guessing.
- The rewrite runs in the background and is proposed as a side-by-side
accept/discard diff in the editor. The AI Assistant panel is not opened.
- Entry points: a banner shown only for legacy-looking queries
(dismissal persisted), and a "Fix Query" button next to Field Reference.
- The Field Reference drawers discover `log_attributes` keys from real
data so the listed fields match what the source actually emits.

## Dependencies

Built on top of #47265 (Logs Explorer -> OTEL endpoint) — that is the
base branch of this PR. Merge #47265 first. Behind `otelLegacyLogs` (off
by default).

Part of DEBUG-145 (split from #47087).

## How to test

- Open the Logs Explorer with a BigQuery logs query (the templates have
some), click "Fix Query", and confirm the diff shows valid ClickHouse
SQL. Accept it and confirm the applied query runs.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added an OTEL legacy logs workflow (behind a feature flag) with an
interactive banner and a “Fix Query” ClickHouse rewrite action,
including an accept/discard diff review overlay.
* Introduced OTEL-aware field reference rendering with dynamic discovery
of `log_attributes` keys and updated OTEL source insertion behavior.
* Enabled dialect-aware SQL completion for ClickHouse logs, using
logs-specific instructions and output constraints.
* **Bug Fixes**
* Improved rewrite flow validation and handling, including log source
detection and cleanup of AI-generated SQL formatting.
* **Tests**
* Added Vitest coverage for rewrite prompt generation,
detection/classification utilities, SQL fence stripping, OTEL field
mapping, and OTEL log attribute key discovery.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2026-06-29 14:31:18 +02:00
Alaister Young d676c832f3 fix(studio): pre-empt React 19 regressions in tests + Support form (#45784)
Four React-19-sensitive patterns that pass on React 18 today but break
under React 19 (verified on the in-flight TanStack Start branch).
Landing on master now so the eventual React 19 upgrade is a no-op for
tests, instead of a separate cleanup pass under upgrade pressure.

Each fix is a strict superset / less-fragile equivalent of the existing
pattern, so master (React 18) stays green.

**Changed:**
- `hooks/misc/useStateTransition.ts` — fire on entry into `newTest` from
any state other than `newTest`, instead of requiring exactly `prevTest →
newTest`. React 18+ auto-batches dispatches across awaits (e.g.
`dispatch SUBMIT` in the handler, `dispatch ERROR` in `onError`),
collapsing `editing → submitting → error` into a single render where the
intermediate `submitting` tick is never observed. Strict superset of the
old check for our reducers — `success`/`error` are only reachable from
`submitting`.
- `Support/CategoryAndSeverityInfo.tsx` — guard `onValueChange` against
Radix Select's spurious `''` emission. When the controlled value
transitions from `undefined` to a defined value whose `SelectItem` isn't
mounted yet (dropdown closed → items haven't registered), Radix's hidden
`BubbleSelect` fires `onValueChange('')` and clobbers the field. No
`SelectItem` can have `value=""` (Radix throws), so any `''` is
guaranteed spurious — drop it before calling `field.onChange`.
([radix-ui/primitives#3381](https://github.com/radix-ui/primitives/issues/3381))
- `EditSecretModal.test.tsx` — `getByLabelText` → `findByLabelText`.
Under React 19's scheduling, the decrypted-value query resolves on a
separate render tick, so form fields appear one tick after the skeleton.
- `LogsPreviewer.test.tsx` — `addEventListener('click', spy)` instead of
`loadOlder.onclick = vi.fn()`. React 19 reassigns `.onclick` on managed
elements as part of its event wiring, clobbering the direct-property
spy.

## To test

### Unit tests
- `pnpm --filter studio test` — all unit tests pass on master (React 18)

### Support form URL prefill (Radix Select guard)
- `/support/new?category=Problem` → category dropdown reads "APIs and
client libraries" on first paint
- `/support/new?category=dashboard_bug` → "Dashboard bug"
(case-insensitive match)
- `/support/new?category=invalid_garbage` → falls back to "Select an
issue" placeholder, no crash
- `/support/new?subject=My%20issue&message=Details%20here` → subject and
message inputs are prefilled
- `/support/new?projectRef=<your-ref>&category=Problem` → both project
selector and category set, library selector appears
- With a prefilled URL, click the category dropdown and pick a different
option — the new value sticks (this is the path that surfaced the Radix
bug, want to confirm we didn't break user selection)
- DevTools console on first load should be clean — no React hydration
mismatch warning

### Support form submit (`useStateTransition` success + error branches)
- Submit a valid support form → green toast "Support request sent"
appears **once**, view swaps to the success screen, one `POST
/platform/feedback/send` in the network panel
- Block `POST /platform/feedback/send` in DevTools → submit → red error
toast appears **once** (not twice — if you see two toasts the relaxed
transition is firing more than it should), form stays editable with all
inputs preserved
- Unblock and submit again → success path runs cleanly

### Sidebar support form (same reducer + `useStateTransition`, separate
component)
- Open the support widget in the side nav (`SupportSidebarForm`)
- Repeat the success and error paths — should behave identically

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed category selector to prevent selected values from being
unexpectedly cleared during form interactions.

* **Tests**
* Improved test reliability for modal field rendering and event handling
assertions.

* **Chores**
  * Clarified internal comments for form initialization logic.

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/45784)

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
2026-05-11 21:56:51 +08:00
Joshen Lim 7f5865872a Enforce noUnusedLocals and noUnusedParameters in tsconfig.json + fix all related issues (#45264)
## Context

Enforce `noUnusedLocals` and `noUnusedParameters` in tsconfig.json + fix
all related issues
2026-04-27 17:42:34 +08:00
Charis 205cbe7d26 chore(studio}: enforce import order, remove bare import specifiers (#44585) 2026-04-07 20:34:10 -04:00
Ivan Vasilov 4fdab665a8 chore: Fix vulnerable packages (#43634)
Fix bunch of vulnerable dependencies:
- https://github.com/supabase/supabase/security/dependabot/3027
- https://github.com/supabase/supabase/security/dependabot/3038
- https://github.com/supabase/supabase/security/dependabot/3058
- https://github.com/supabase/supabase/security/dependabot/3064
- https://github.com/supabase/supabase/security/dependabot/3065
- https://github.com/supabase/supabase/security/dependabot/3085
- https://github.com/supabase/supabase/security/dependabot/3095
2026-03-11 14:10:59 +01:00
Jordi Enric 4b994fcb54 feat: copy multiple logs (#43218)
Allows users to select logs and copy their contents for debugging, or
pass them on to assistant with one click.

## To test
- go to logs or log explorer
- select some logs
- try copying as json, markdown, or sending it to assistant.
2026-03-06 13:24:37 +01:00
Ignacio Dobronich 6a873c67fa chore: logs entitlements badges (#43443)
### Changes
- Replace hardcoded plan-to-days mapping (`getAvailableInForDays`) with
entitlement-driven checks.
- Show a lock icon on date picker helpers that exceed the user's
entitled log retention, instead of a plan name badge.
- Remove availableIn from DatetimeHelper type — the entitlement API is
now the single source of truth.

### Testing
- Head to `/project/_/logs` on a free plan project and verify lock icons
appear on helpers beyond 1 day
- Verify clicking a locked helper still triggers the upgrade prompt
<img width="511" height="483" alt="image"
src="https://github.com/user-attachments/assets/41dce8c0-205a-4d82-9c07-12f4bb6b0f4d"
/>
2026-03-05 13:38:07 -03:00
Joshen Lim a54fd1e93c Fix resizeable panel error in LogTable (#42851)
## Context

There's an error from the `LogTable` component `Previous layout not
found` that's caused by trying to set `defaultSize` as a dynamic value
for the `ResizablePanel` component. Theres 2 ways to address this really

Option 1: Set `id` param on `ResizablePanel` to be a dynamic: this came
from a suggestion by the library's author
[here](https://github.com/bvaughn/react-resizable-panels/issues/401#issuecomment-2351010630)

Option 2: Avoid dynamic values entirely

Opting for option 2 as the cleaner way to fix this. Just note as well
that there's a couple of console warnings from the library about
`Invalid layout total size` - this one's safe to ignore and is seemingly
an issue with the current version of `react-resizable-panels` that we're
using (v2, there's currently v4 already)

## To test

Tbh I couldn't reproduce this locally actually, but I happened to come
across this same bug while debugging the `Invalid layout total size`
warning in `DefaultLayout` by trying to set a dynamic value for
`defaultSize` which made me realise the dynamic value was the one
causing that error

<img width="534" height="89" alt="image"
src="https://github.com/user-attachments/assets/3b696c16-ea66-481e-a698-d59841c7400d"
/>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **Refactor**
* Improved logs viewer layout with resizable panels for better content
management and log selection display.
* Reorganized internal component structure and module exports for
improved maintainability.

* **Chores**
  * Updated test imports to reflect internal module restructuring.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-16 07:42:14 -07:00
Joshen Lim f0fbcbd2a3 Add preflight EXPLAIN check to table editor rows (#42321)
## Context

Part of an investigation to see how we can make the dashboard more
resilient for large databases by ensuring that the dashboard never
becomes the reason for taking down the database accidentally.

Am proposing that for interfaces that rely heavily on queries to the
database for data to render, we add preflight checks to ensure that we
never run queries that exceed a certain cost threshold (and also have UI
handlers to communicate this) - this can be done by running an EXPLAIN
query before running the actual query, and if the cost from the EXPLAIN
exceeds a specified threshold, the UI throws an error then and skips
calling the actual query.

## Demo
Am piloting this with the Table Editor, and got an example here in which
my table has 500K+ rows, and I'm trying to sort on an unindexed column:


https://github.com/user-attachments/assets/ccad2ea9-d62c-4106-8295-2a6df5941474

With this UX, the pros are that
- It's relatively seamless and not too invasive, most users won't notice
this unless they run into this specific scenario
- We can incrementally apply this to other parts of the dashboard, next
will probably be Auth Users for example

However there are some considerations:
- The additional EXPLAIN query adds a bit more latency to the query
since its a separate API request to the query endpoint
- ^ On a similar note, it will hammer the API a bit more, which may
result in higher probability of 429s
- However, I reckon that the preflight checks are meant to be used
sparingly and only for certain parts of the dashboard that we believe
may cause high load.
- e.g for the Table Editor, reckon we only need this for fetching rows?
The count query is largely optimized already (although we could just add
a preflight check there too)
- It's just meant to be a safeguard to prevent running heavy queries on
the database



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Query preflight with cost checks and a user-facing high-cost dialog
showing cost details and remediation suggestions.
* Grid exposes an explicit error flag and surfaces richer error
metadata.

* **Bug Fixes**
* Standardized error handling and more consistent error displays across
the app.
* Explain analysis now reports an additional max-cost metric for
queries.

* **UI**
* Tweaked empty-state interaction/layout and slightly wider header
delete control.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Ali Waseem <waseema393@gmail.com>
2026-02-03 17:55:54 +08:00
Ali Waseem f4409d4ac5 fix: updated explain to not crash on non text formats (#42076)
updated explain to not crash on non text formats
2026-01-22 12:26:45 -07:00
Jordi Enric d786c1bdd4 add dynamic datepicker (#41924)
* add dynamic datepicker

* tests

* handle singulars
2026-01-15 13:11:40 +00:00
Jordi Enric 5b3cb3e8ba fix Log Explorer run query inconsistency (#41634) 2026-01-06 16:23:30 +01:00
Ali Waseem 89e0fe4f28 feature: Explain tab in SQL editor that shows output of explain analyze (#41569)
* wip: explain tab in results editor

* updated to add sql explain

* updated to default back to results

* updated explain function

* updated case with multiple statements

* updated to reset explain query results

* added tests for semi colon comments

* feature: add explain w/ AI on pretty-explain tab (#41588)

* wip: added explain with AI

* wip: updated header with new buttons

* updated prompt

* remove any types

* removed unused flag

* updated header

* formatted code
2025-12-24 10:14:53 -07:00
Ali Waseem 3a41846044 Feature: Pretty explain in SQL Editor (#41289)
* added cleaner explanation

* update UI

* refactored components

* updated logic

* removed steps

* updated UI

* updated read

* updated node render styles

* added tests

* updated file naming

* updated utility method

* updated description

* updated spacing

* updated to include header

* updated logic

* added flag to hide pretty explain

* updated cost width

* added aria label

* move index scan down

* updated to catch schema and qoutes

* remove node

* forgot the missed node

* addressed PR feedback

* updated bad slop

* removing tests

* updated node to be undefined, null remove it makes the logic cleaner

* updated function name

* updated row indicator to be separate components

* updated code rabbit feedback

* added divide by zero check

* updated test description
2025-12-12 20:35:18 +00:00
Jordi Enric 0d95b4f9c4 feat(reports): add request country map (#40903)
* wip

* move map to file, fix projection

* fix

* support micro countries

* cleanup code, add tests, improve theming

* remove border on hover

* fix theme detection

* fix query

* update map json to simplify code

* formatting

* make active color opacity based

* fix names

* rm flaky test

* rm comment

* Update apps/studio/components/interfaces/Reports/renderers/ApiRenderers.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update apps/studio/components/interfaces/Reports/renderers/ApiRenderers.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update apps/studio/components/interfaces/Reports/renderers/ApiRenderers.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add missing import

* rename

* validate error safely

* undo tsconfig change

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-03 20:59:02 +00:00
Joshen Lim 7facc62652 Fix Logs.Datepickers.test.tsx (#40939) 2025-12-01 14:08:15 +08:00
kemal.earth 4a9def58c1 feat(studio): rename reports to observability (#40317)
* feat: rename reports to observability on nav sidebar

* fix: observability route

* feat: observability navigation and redirects

* fix: first time load redirects

* feat: change auth overview reports link

* feat: clean up of other links to observability sections

* fix: query perf redirects

* chore: find exhaustive deps and disable for time being

* fix: charis suggested fixes

* fix: forgotten suggested fix

* chore: delete old reports views files

* fix: paths in reports tests

* fix: url sync useEffect should trigger only once

* feat: reorganise observability sidebar

* fix: data api not being selected

* fix: wrong layout for realitime

* fix: custom reports reloading

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Ali Waseem <waseema393@gmail.com>
2025-11-18 17:54:05 +00:00
Charis 04c2fd3103 fix(support link): auto-fill organization when on org page (#40108)
* fix(support link): auto-fill organization when on org page

If user clicks the Support Link from an org page (not a project page),
pass along the organization slug in the query params to auto-fill the
Support Form.

* fix(date picker test)

The new UI for range picking is a little different, and requires
double-clicking to start a new range.
2025-11-03 16:41:50 -05:00
Charis 0c501d7889 chore: consolidate calendar pickers (#40052)
* chore: update react-day-picker

* fix(design system): date picker demos

* refactor(date picker): change logs date picker to use react-day-picker

* refactor(date pickers): change remaining date pickers to use react-day-picker

* cleanup(date pickers): minor code cleanup

* fix(date picker): fix behavior for single day selection
2025-11-03 10:10:19 -05:00
Jordi Enric 1d15a53e97 Improve logic to calculate time range when log chart is clicked (#38781)
* move calc click time range to fn

* fix import

* fix import

* fix import

* fix import

* fix import

* fix import

* add tests

* make coveralls step optional

* rm coveralls change
2025-09-18 10:04:46 +02:00
Jordi Enric f65884f341 Reports consolidation 1: Edge Functions (#38129)
* wip

* add formatTimestamp for reports

* add tests for formatTimestamp

* fix type err

* rm old chart logic

* do not retry replication sources

* update report config schema

* use report config

* fix charts, simplify config

* sync charts

* rm comment

* usefilltimeseries

* move chart upsell to its own component

* cleanup v2 chart component

* fix chartheader function format bug

* make attributes dynamic

* move types to its own file

* update reportv2chart types

* update reportv2 chart type

* use new edgefn report

* rm mock

* rm old code

* fix type err

* fix type err

* rename hook

* return sql in dataprovider

* add button to open in log explorer in chart header

* fix type

* fix chart type toggle

* fix type error

* export data transformation for testing

* add tests for data transformation

* rm dupped code

* add report settings to edge fn report

* rm mocked data

* rename

* fix type err

* Apply suggestion from @joshenlim

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>

* rm unnecessary div

* rm unnecessary props from report upsell

* cleaner query keys

* move loadin/error/empty states to inside panel content to prevent layout shift

* use card instead of panel

* use buttontooltip in chartheader

* use buttontooltips in buttons with tooltips

* rm export default

* rm export defualt

* rm i from key

* fix status code transformation

* fix filltimeseries

* add tests for filltimeseries

* fix data transform

* pass xaxis key to chart

* refactor hover state to use timestamp not index

* fix tests

* fix areachart

* fix barchart

* fix chartheader

* fix composedchart

* fix stacked

* fix type missing xaxiskey

* type fix

* Revert "refactor hover state to use timestamp not index"

This reverts commit cfd7886460.

* undo charthover changes, fix data aggregation

* Revert "fix tests"

This reverts commit e5200a2035.

* move transforms to testable functions, add tests

* nit refactors and clean up

* feat: add aws nimbus cloud provider (#38312)

* feat: add aws nimbus cloud provider

* Fix return type of useCustomContent hook

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>

* add utc plugin to dayjs

* move plugins to vitest setup

* rm customparser plugin

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Alaister Young <alaister@users.noreply.github.com>
2025-09-01 11:10:30 +02:00
Jordi Enric e4837267f6 Warehouse Clean Up (#37507) 2025-07-28 17:14:57 +02:00
Drake Costa 57cd26ac77 Move non-layout Storage components to components/interfaces (#37381)
* Update studio testing setup files

Improves API mocking type safety and polyfills browser APIs necessary to run tests with framer-motion components

* fix missing listen call for msw, resolve test type error

* fix imports

* Update studio testing setup files

Improves API mocking type safety and polyfills browser APIs necessary to run tests with framer-motion components

* fix missing listen call for msw, resolve test type error

* fix imports

* Move non-layout Storage related components to `components/interfaces`

* Fix paths

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-07-22 23:48:20 +08:00
Jordi Enric 804f270c3a reports pricing (#36992)
* add database pricing config

* pricing paywalls

* fix showing correct charts

* fix test

* remove disabled helpers in favor of upsell dialog

* fix missing fn to calc time

* rm clog

* show badge next to helpers

* ui fix

* fix type errs

* fix hallucinated import fml
2025-07-14 16:45:29 +02:00
Jordi Enric 3d88e1a03d Edge Function Report (#36763) 2025-07-02 15:54:39 +02:00
Jordi Enric 3c588294a6 Improve Integration Test Setup and re-add LogPreviewer tests (#35358)
* fix logs previewer and msw

* refactor api mocking for better dx

* update readme

* comment out error handler for vitest

* rm unnecessary tests

* fix custom render nuqs type

* add logs search test

* rm unnecessary import

* update readme with customRender and customRednerHook

* rm unnecessary api handler

* Move the NODE_ENV to the studio build command in turbo.json.

* Update apps/studio/tests/README.md

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>

* add cursor rule

---------

Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-05-06 09:50:56 +02:00