Commit Graph

37 Commits

Author SHA1 Message Date
saltcod 6da7ce4f3b [create-pull-request] automated change 2026-05-07 20:28:41 +00:00
Coenen Benjamin 151a1792d9 feat(studio): add support for new ducklake destination in replication UI (#45370) 2026-04-29 21:42:05 +02:00
Han Qiao 67deabf67e fix: create etl publication as postgres (#45043)
## What kind of change does this PR introduce?

Bug fix

## What is the current behavior?

Creating a schema only branch fails because ETL publication is owned by
`supabase_etl_admin` which users have no access.

## What is the new behavior?

Since ETL supports user managed publications, create them through pgmeta
so it's owned by `postgres` role instead.

## Additional context

mirrors [upstream
etl](https://github.com/supabase/etl/blob/main/etl-api/src/db/publications.rs#L22-L51)
implementation


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Added guards to prevent creating publications when project or
connection info is missing, with clearer error logging.
* Ensure the project connection string is explicitly passed so
publications target the correct database.

* **Refactor**
* Publication creation now executes generated SQL directly against the
database, with correct handling of empty or selected table lists and
proper identifier quoting for reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
2026-04-27 19:45:26 +08:00
Joshen Lim 7f5865872a Enforce noUnusedLocals and noUnusedParameters in tsconfig.json + fix all related issues (#45264)
## Context

Enforce `noUnusedLocals` and `noUnusedParameters` in tsconfig.json + fix
all related issues
2026-04-27 17:42:34 +08:00
Riccardo Busetti 8347877957 feat(etl): Add UI elements to disable external replication (#45035) 2026-04-24 08:17:58 +02:00
Riccardo Busetti a23e673993 ref(etl): Improve UX around table resets (#44843) 2026-04-14 16:21:36 +02:00
Charis 3b7052b5a9 cleanup: fix import order and prefixes for studio/data (#44501) 2026-04-03 09:15:57 +02:00
Riccardo Busetti d41ab5b4ff ref(etl): Update configuration and add new parameters (#43126) 2026-03-02 16:01:44 +01:00
Riccardo Busetti 67ed8f18ee feat(etl): Add new setting for controlling table copy (#42853) 2026-02-24 08:14:23 +01:00
Joshen Lim 6ffc3fc94c chore(studio): Replication UI updates for new replica CTA + URL query state handling (#42378)
## Context

Related to unifying read replicas into database replication page which
is currently in internal testing

## Changes involved

- Updates the "New replica" CTA in `DatabaseSelector` (e.g within the
SQL Editor) to link to the replication page instead of settings
infrastructure
- Refactor replication UI to use URL query states for new destination +
edit destination

## To test

- [ ] Verify that the "New replica" CTA links correctly to replication
page if flag is on
- [ ] Verify that new + edit destination UI works as expected
- Note that there's some server side validation for ETL replication now
so might be tricky to test creating an actual ETL replication
- Minimally can test creating a read replica and ensure that the UI
behaviour is all as expected

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added unified replication experience with URL-based state management
for destination creation and editing

* **Refactor**
* Simplified replication panel component interfaces and consolidated
destination data fetching logic
* Enhanced edit flows to leverage URL parameters for seamless navigation

* **Chores**
  * Marked legacy read replica creation panel as deprecated
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-03 15:54:21 +08:00
Joshen Lim 1536ee1064 Joshen/fe 2287 summary nodegraph in replication page (#42042)
* init

* Init 2

* Finalize

* bit more details in the summary graph

* Moreeee details

* Clean

* 🧹

* FIIIXX

* Fix rabbit

* Simplify long polling logic to only depend on database for destinations UI

* ADdress rabbit

* Address feedback
2026-01-23 17:59:52 +08:00
Riccardo Busetti 1238a19fe4 feat(etl): Expose more configuration parameters (#41758)
* feat(etl): Expose more configuration parameters

* Nit nudges

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2026-01-07 15:36:32 +08:00
Riccardo Busetti d795240928 feat(etl): Add integration with new validation endpoint (#41731)
* feat(etl): Add integration with new validation endpoint

* Fix

* Fix

* Fix

* Fix

* UI nudges + clean up + refactors

* Fix TS

* Sort issues

* Small improvement

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2026-01-07 00:32:08 +08:00
Riccardo Busetti c62cd07b86 feat(etl): Implement batch rollback (#41552) 2026-01-06 04:13:25 +01:00
Joshen Lim 31d2e8ac5b Shift ETL destination components into a separate folder (#41457)
* Shift ETL destination components into a separate folder

* Small fix
2025-12-18 18:55:07 +08:00
Ivan Vasilov 0d5be306ef chore: Bump React Query to v5 (#40174)
* Bump the deps, refactor deprecated code.

* Migrate keepPreviousData usage.

* Migrate all uses of InfiniteQuery.

* Fix refetchInterval in queries.

* Migrate all use of isLoading to isPending in mutations.

* Fix accessing location in claim-project.

* Fix a bug in duplicate query keys.

* Migrate all queries to use isPending.

* Revert "Fix accessing location in claim-project."

This reverts commit 2a07df64b5.

* Revert the rss.xml file to master.
2025-12-10 10:10:29 +01:00
Riccardo Busetti 1a3ba9f15a ref(etl): Rename ETL Replication to Replication and update docs (#40769)
* ref(etl): Rename ETL Replication to Replication and update docs

* Fix

* Fix

* Fix

* Fix

* Fix
2025-11-25 20:45:50 +08:00
Riccardo Busetti 91464b8fe8 ref(etl): Rename Replication to ETL Replication (#40202)
* ref(etl): Rename Replication to ETL Replication

* Fix

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-11-07 12:02:40 +08:00
Ivan Vasilov 4da5d019ae chore: Migrate some stray queryKeys to the RQ 5 syntax (#40086)
Fix some stray queryKeys.
2025-11-04 09:33:49 +01:00
Ivan Vasilov 8b657165b5 chore: Migrate to use custom type for ReactQuery queries and mutations (#40073)
* Add custom types for queries, mutations and infinite queries.

* Migrate all queries to use the new type.

* Migrate all infinite queries to useCustomInfiniteQueryOptions.

* Migrate all mutations to use useCustomMutationOptions.

* Add type to all imports in `types` folder.
2025-11-03 13:18:13 +01:00
Joshen Lim 64e3e047eb Final final cleaning up barrel files (#40018)
* Final final cleaning up barrel files

* Fix merge conflict
2025-10-31 14:02:59 +08:00
Joshen Lim 0910908561 Swap analytics buckets endpoint (#39610)
* Swap analytics buckets endpoint

* Fix type issues

* fix

* Update API types

* Fix tests

* nit

* Final fixes

* Fix

* chore(studio): analytics buckets contents (#39701)

* first commit

* improvements

* misc

* cleanup

* remove tappable whole

* design updates

* analytics improvements

* polish

* add clashed case

* fix pairing status

* connect tables basics

* add multiselect

* restore prod version for non-feature preview users

* fix padding inconsistency

* terminology fix

* better empty state

* fix terminology

* block table connection dialog button

* Nit refactors

* minor nits

* Add comment

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>

* Default circumstance to fresh in ImportForeignSchemaDialog

* Hook up connecting tables for analytics buckets (#39906)

* Hook up connecting tables for analytics buckets

* Address comments

* Clean up iceberg wrapper when deleting analytics bucket (#39902)

* Clean up iceberg wrapper when deleting analytics bucket

* Clean up s3 access key when deleting analytics bucket

* Make connect table disabled for now

* Most clean up and refactoring

* Refactor RQ

* Nit

* Refactor

* nit

---------

Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com>
2025-10-29 19:10:10 +08:00
Ivan Vasilov da4a40e308 chore: Migrate RQ functions to use object syntax style (#39895)
* Migrate all uses of invalidateQueries to use object syntax.

* Migrate the remainder of useInfiniteQuery.

* Migrate all setQueriesData.

* Migrate all fetchQuery uses.

* Migrate some leftover functions from RQ.

* Fix issues found by Charis.
2025-10-28 10:43:14 +01:00
Alaister Young 8855d05803 chore(studio): swap react-query to object syntax (#39842)
* chore(studio): swap react-query to object syntax

* Fix small issues found

* Fix realtime settings

* Nit

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-10-27 09:38:27 +01:00
Raminder Singh a21fa43d8f feat: add iceberg (analytics bucket) destination in UI (#39324) 2025-10-16 06:56:11 +05:30
Riccardo Busetti f07c9cad3a ref(replication): Add alpha banner and remove batch size (#38588) 2025-09-11 15:25:39 +08:00
Riccardo Busetti 48d76a1c6b feat(replication): Add UI for updating pipeline versions (#38473) 2025-09-10 11:26:37 +02:00
Riccardo Busetti 5e4cc212aa feat(replication): Add ability to copy pipeline id and filter logs by pipeline id (#38407) 2025-09-04 15:47:01 +02:00
Riccardo Busetti 77356bf946 feat(replication): Significantly improve the replication UI behavior (#38237) 2025-08-29 12:20:51 +02:00
Riccardo Busetti e9424b98de feat(replication): Delete destination and pipeline atomically (#37790) 2025-08-11 08:23:15 +02:00
Riccardo Busetti 1ea8c656db feat(replication): Add new UI for new table state handling (#37662)
* feat(replication): Add new UI for new table state handling

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Improve

* Fix

* Fix

* Fix TS

* Some UI tweaks

* Tweaaakk

* Fix semantics

* Small optimization

* Some adjustments

* Bit more fixes

* Small UI tweak to simplify UI

* Fixy fix

* gahhh tiny tiny tiny

* Last bit of cleanup

* I swear one final tweak

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-07 12:16:03 +02:00
Joshen Lim 5da5ad06c0 Shift pipeline status page to its own route (#37431)
* Shift pipeline status page to its own route

* Fix back link
2025-07-29 16:33:14 +08:00
Riccardo Busetti 0ecddf1901 feat(replication): Implement UI to render pipeline replication status (#37376) 2025-07-24 13:04:07 +02:00
Riccardo Busetti a77d40e43b feat(replication): Update the API calls for replication endpoints (#36866) 2025-07-08 16:34:03 +02:00
Raminder Singh b411ed3421 chore: rename sink to destination for the etl api (#35931)
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
2025-05-26 21:30:10 +05:30
Raminder Singh 520428680d feat: call the new api endpoints to handle partial failures in pg_replicate (#35418)
* feat: use the new tenant-source api to atomically create the tenant and source in pg_replicate

* feat: use the new sink-pipeline api to atomically create the sink and pipeline in pg_replicate

* fix: remove some unused imports

* feat: use the new sink-pipeline api to atomically update the sink and pipeline in pg_replicate

* feat: deleting the sink cascades to delete the pipeline

* chore: update api types

* fix: revert accidental update to types

* remove unused code

* add a comment explaining why deleting only sink is enough
2025-05-10 12:04:37 +05:30
Raminder Singh 23ceb9072d Add replication UI in Studio (#30090)
* add dummy sinks and pipelines pages

* update api types

* show empty sources state

* show empty replication state at /replication

* create source when enable replication button is clicked

* improve replication page when replication is enabled

* replace sources page with publications page

* publications table

* show publications in table

* create publication wip

* show toast error instead of throwing an exception

* user can now delete a publication

* show empty sinks page

* create and list sinks

* add ui to delete a sink

* show pipelines on the pipelines page

* add ui to create and delete pipelines

* get pipeline status wip

* show pipeline status wip

* show correct label on action buttons

* start and stop pipelines

* remove a couple of console.logs

* fix error when deleting a pipeline

* only consider replication enabled when a source with name = ref is present

* add source and sink names

* correct colspan for 'no pipelines' row

* hide 'supabase_realtime' publication on ui

* move filtering to fetch query

* show sink name in ui

* show source/sink names on pipelines page

* fix start/stop status shown on ui

* fix prettier formatting

* update api types

* extract pipeline action button as a separate component

* fix a crashing page

* fixed publications page crash

* update to match with changes in api

* add new replication page under database

* hide replication page behind feature flag

* update types

* update api types

* show destinations empty state

* add destinations table

* factor out components from Destinations table

* show status dot

* move pipeline fetch query to parent component

* add ability to enable/disable a pipeline

* show loader when starting or stopping a pipeline

* fix a bug in which loading & empty states were shown together

* fix a bug in which error & empty states were shown together

* wrap in default layout

* add new destination panel

* add type field

* fix a forwardRef error

* fix layout

* create destination

* delete destination

* add ability to create or delete destinations with pipelines

* create source if missing

* show only a single error

* add an enable switch

* new layout

* add subsections

* comment out unused code

* show enabled switch only in the header

* close panel when destination is created

* disable buttons when api requests in flight

* reduce panel size

* remove commented out code

* treat max size and max fill secs as numbers

* use drop down to show publications

* simpler vertical layout

* add separators

* add form validation

* remove publications drop down padding

* add new publication button

* hide advanced settings behind an accordion

* add some margin between icon and text

* show publications panel on clicking new publication button

* add header to new publication panel

* fix validation not running for publication drop down

* create publication in the new publication panel

* add table selector in new publication panel

* update api types

* remove old code

* update platform.d.ts

* update navigation bar utils

* remove a redirect from replication page to publications page

* ask user for confirmation before deleting destination

* edit destination panel

* edit destination panel values fixed

* bug fixes

* fix prettier formatting

* enable/disable pipeline after editing

* rename snake_case params to camelCase

* loading button when editing

* remove merge markers

* update api types

* add max_staleness parameter in sinks for bigquery

* add read replicas flow diagram to replication page

* remove an unused import

* Revert "add read replicas flow diagram to replication page"

This reverts commit 8852d7847b457885603dba786141a8aaf8e99350.

* add panel to warn users about additional cost before creating a destination

* hide replication page contents behind a feature flag

* fix merge conflicts

* styling changes

* revert static flag

* styling updates

* fixes

* fix switch

* copy

* fix layout

---------

Co-authored-by: Saxon Fletcher <saxonafletcher@gmail.com>
2025-04-23 10:52:46 +05:30