Commit Graph

24 Commits

Author SHA1 Message Date
Joshen Lim 3d10f2cab9 Add user flow for iceberg wrapper if api keys are rotated (#47336)
## Context

We found an issue regarding Analytics Buckets and the Iceberg wrapper -
upon creation of an analytics bucket, the wrapper is automatically
created for the users which involves using the project's API keys as the
catalog's token.

However, if the user were to rotate the API keys, this will cause the
wrapper to break and there's currently no clear user flow for the user
to self-remediate - the only indicator they'll see is just a 403 error
(e.g when trying to view the analytics bucket table via FDW on the table
editor or SQL editor)

## Changes involved

Am adding a user path for users to self-remediate a little, starting
from the Table Editor - we'll add a contextual error message as such if
we detect a 403 that's caused by an invalid token:
<img width="1110" height="320" alt="Screenshot 2026-06-26 at 17 33 11"
src="https://github.com/user-attachments/assets/28ea4ce6-5b81-4217-9952-880acb02f2bd"
/>

We'll subsequently also float this issue up in the Analytics Bucket UI
(which is linked from the contextual error above)
<img width="1114" height="466" alt="Screenshot 2026-06-26 at 17 31 52"
src="https://github.com/user-attachments/assets/8d112e5b-6ecc-458b-b4dc-7e7647da3fb2"
/>

And users can then choose to use another API key as the catalog token
<img width="585" height="246" alt="Screenshot 2026-06-26 at 17 31 56"
src="https://github.com/user-attachments/assets/3d9689a5-b18d-4f07-a5a5-d882e41c5958"
/>

The warning will thereafter go away, and users will be able to query the
FDW again via Table Editor or SQL Editor

## To test

- [ ] Create an analytics bucket, set up a table and foreign schema (via
Query via Postgres)
- [ ] Insert some data, or verify that you can view the iceberg table
from the Table Editor
- [ ] Now rotate your API secret key (delete the old, create a new)
- [ ] Verify that you'll run into that error if you view the iceberg
table from the Table Editor
- [ ] Follow the flow -> Go to the Analytics Bucket UI to update the
catalog token
- [ ] Verify that thereafter, you can view the iceberg table again from
the Table Editor

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added clearer Iceberg/analytics bucket setup prompts for missing,
outdated, or uninstalled wrappers.
* Added an “Update catalog token” dialog and a collapsible “View error”
troubleshooting UI.
* **Bug Fixes**
* Improved detection of Iceberg authorization failures and now shows a
more specific error with guidance.
* Warn users when the saved catalog token no longer matches available
API keys.
* Enhanced post-update refresh behavior so updated token values display
correctly.
* **Documentation**
* Clarified vault token description to indicate it may be a secret or
service role key.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 17:43:38 +08:00
Joshen Lim 1baaded0bb Consolidate execute-sql-query into execute-sql-mutation (#46944)
## Context

Just some clean up as I was going through stuff
- `useExecuteSqlQuery` is deprecated and not used at all
- As such `execute-sql-query` is technically irrelevant, the more
relevant file is `execute-sql-mutation`
- Hence opting to consolidate `execute-sql-query` into
`execute-sql-mutation`
- Also removing `ExecuteSqlError` since its just re-exporting the
`ResponseError` type

There's a lot of file changes but its essentially just updating the
importing statements across the files
2026-06-16 00:07:16 +08:00
Charis 0433eeb5f5 feat(studio): mark sql provenance for safety (#45336)
Mark provenance of SQL via the branded types SafeSqlFragment and
UntrustedSqlFragment. Only SafeSqlFragment should be executed;
UntrustedSqlFragments require some kind of implicit user approval (show
on screen + user has to click something) before they are promoted to
SafeSqlFragment.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Editor and RLS tester show loading states for inferred/generated SQL
and include a dedicated user SQL editor for safer edits.

* **Refactor**
* Platform-wide SQL handling tightened: snippets and AI-generated SQL
are treated as untrusted/display-only until promoted, improving safety
and consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 13:08:06 -04:00
Charis 3b7052b5a9 cleanup: fix import order and prefixes for studio/data (#44501) 2026-04-03 09:15:57 +02:00
Joshen Lim 98b1b79909 Chore/shift manual queries into pg meta 04 (#43956)
## Context

Shifts all remaining dashboard queries into pg-meta so that we
centralize all manually written queries in one place
Having them in packages/pg-meta also allows us to write tests for them

## To test

Just needs a smoke test on
- Role Impersonation
- Lints
- Data API
- Database
  - Enumerated Types
- Integrations
  - Foreign Data Wrappers
  - Vault
2026-03-24 16:23:13 +08:00
Joshen Lim e48c909b31 Chore/deprecate use query state with select part 04 (#42747)
## Context

Related to FE-2461

More refactoring to clean up usage of `useQueryStateWithSelect`, in the
following pages
- Auth OAuth Apps
- Cron Jobs
- Vaults Secrets Management
- Database Hooks
- Wrappers

## To test

In each of those pages, verify that
- [ ] Clicking the "new" cta updates the URL params, and refreshing
should re-open the sheet
- [ ] Editing an existing item should update URL params, and refreshing
should re-open the sheet with the right item
- Verify that if the URL param has the wrong id, page should not open
the sheet, show a toast, and reset the URL param
- [ ] Deleting an existing item should update URL params, and refreshing
should re-open the sheet with the right item
- Verify that if the URL param has the wrong id, page should not open
the sheet, show a toast, and reset the URL param

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Added "item not found" toasts and clearer feedback after
create/update/delete actions; prevent opening create flows when server
features are disabled.

* **Style**
* Standardized button labels, sizes and modal/dialog spacing; refined
table column widths and inline code formatting for readability.

* **Refactor**
* Simplified UI state flows across OAuth Apps, Hooks/Webhooks, Cron
Jobs, Vault Secrets, and Wrappers to use consistent URL-driven
interactions and centralized modals.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-13 16:18:28 +08:00
Ivan Vasilov f9cbf661ca fix: Fix a bug in Edit FDW sheet (#41685)
* Add types to the getDecryptedValues function.

* Refactor the edit-wrapper-sheet to fetch the secrets by id, instead of name.

* Add try/catch for the secret values fetching.

* Minor CodeRabbit nitpicks.

* Small improvement for types in EditWrapperSheet

* Small fix for WrapperRow icon space issue hiding ChevronRight

* Update comment

* Only fetch encrypted IDs if value is a valid UUID

* Nit

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2026-01-05 12:00:49 +07:00
Joshen Lim 2209d72990 Update UI to de-couple ETL replication context from Analytics Buckets (#40645)
* Update UI to de-couple ETL replication context from Analytics Buckets

* Nit

* Nit

* nit

* Address all feedback

* Address feedback

* Add enabled feature check for analytics and vector buckets

* FIXX

* Update apps/studio/components/interfaces/Storage/AnalyticsBuckets/AnalyticsBucketDetails/NamespaceWithTables/TableRowComponent.tsx

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

---------

Co-authored-by: Alaister Young <alaister@users.noreply.github.com>
Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>
2025-11-21 16:28:31 +08:00
Ivan Vasilov 8b657165b5 chore: Migrate to use custom type for ReactQuery queries and mutations (#40073)
* Add custom types for queries, mutations and infinite queries.

* Migrate all queries to use the new type.

* Migrate all infinite queries to useCustomInfiniteQueryOptions.

* Migrate all mutations to use useCustomMutationOptions.

* Add type to all imports in `types` folder.
2025-11-03 13:18:13 +01:00
Ivan Vasilov da4a40e308 chore: Migrate RQ functions to use object syntax style (#39895)
* Migrate all uses of invalidateQueries to use object syntax.

* Migrate the remainder of useInfiniteQuery.

* Migrate all setQueriesData.

* Migrate all fetchQuery uses.

* Migrate some leftover functions from RQ.

* Fix issues found by Charis.
2025-10-28 10:43:14 +01:00
Alaister Young 8855d05803 chore(studio): swap react-query to object syntax (#39842)
* chore(studio): swap react-query to object syntax

* Fix small issues found

* Fix realtime settings

* Nit

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-10-27 09:38:27 +01:00
Ivan Vasilov 32c3eeb389 feat: Create new schema when creating a FDW with the schema option (#37205)
* Change the import foreign schema to always create a schema.

* Fix the Iceberg wrapper.

* Refactor the create wrapper to always create a new schema.

* Remove unneeded props.

* Smol fixes

* Prevent double error toasts

* Smol fix

* Fix the wrapper creation to include the correct api key.

* Fix a bug with the new api keys.

* Handle both types of keys when fetching the iceberg namespaces.

* Add a field to all wrappers to hold the schema name which can import foreign schema.

* sq

* When importing a foreign schema, save the schema in a special field.

* Fix a type error.

* Fix importing foreign schema overriding wrapper server options with unencrypted values

* Add comment

* Handle duplicate and empty schemas when importing a foreign schema.

* Update the copy in the foreign schema wrappers.

* Remove unnecessary code.

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-07-25 10:54:17 +02:00
Ivan Vasilov 89c803551b feat: Analytic storage bucket (#37003)
* Migrate the CreateBucketModal to use RHF.

* Minor fixes for the create bucket modal.

* Use the bucket type from the API everywhere.

* Improve the types on some functions. Add "skip" mode to FDWCreateMutation.

* Expand the CreateBucketModal to explain what will happen when creating an iceberg bucket.

* Add a page for iceberg buckets.

* Add error states to the iceberg explorer.

* Fix the names for the FDWs.

* Add a dialog for setting up namespace.

* Restructure the new files.

* Add a download env button.

* Add warning to install the wrappers extension when creating an iceberg bucket.

* Add failover details when the FDW is not setup.

* Fix a lint error.

* Fix a type error.

* Minor fixes.

* Fix the server name.

* Add an icon for iceberg buckets.

* Fix the import foreign schema dialog.

* Make the setup wrapper button functional.

* Fix a bad useMemo dependency.

* Small changes to the iceberg bucket page.

* Minor fix for the edit wrapper sheet.

* Rename the files from kebab-case to PascalCase.

* Rename the files again to include Analytic instead of Iceberg.

* Rename Iceberg type to Analytic.

* Add a switch for creating namespace in the import foreign schema dialog.

* Fix the CreateBucketModal.

* Fix the delete modal feature.

* Fix the S3 keys in the FDW.

* Only create a namespace if the switch is true.

* Regenerate and fix the types.

* Fix the FDW create mutation to handle numbers in the FDW names.

* Make the icon smaller.

* Check whether the namespace exists, if it doesn't create it.

* Hide action from the analytic bucket which don't work.

* Invalidate namespaces when creating them.

* Add small explanation for the creation of namespaces.

* Minor fixes.

* Tons of changes to make the namespace feature work.

* Fix type errors.

* Fix bad import.

* Minor copy fixes.

* Replace the multiple cards with a table of namespaces.

* update copy icon

* tiny copy update

* Fix the empty state for foreign tables.

* Hide the analytics bucket option for self-hosted.

* Minor copy fixes.

* Expand the CTA on no namespaces state.

* More minor fixes.

* More small fixes.

---------

Co-authored-by: Alaister Young <a@alaisteryoung.com>
2025-07-15 14:18:38 +02:00
Andrew Valleteau 31aad403de fix(studio): early fail query when x-connection-encrypted is invalid (#35331)
* fix(studio): early fail query when x-connection-encrypted is invalid

* fix(studio): uniformize readDatabase and projectDetails connString handling

* chore: update api types

* chore: add connectionString null option

* fix: only enforce x-connection-encrypted on platform

* chore: refactor connString check in a single point

* chore: fix guard logic

* chore: fix pgMetaGuard

* chore: fix types
2025-05-08 12:11:03 +02:00
Bobbie Soedirgo bc6cca73bb fix: remove refs to pgsodium keys from vault & wrappers (#35168) 2025-04-22 20:49:18 +08:00
Andrew Valleteau eb7efdef7f chore(studio): move Query to pgMeta add tests (#34232)
* chore(studio): move Query to pgMeta add tests

- Move the Query builder from studio to pgMeta
- Add e2e tests over the generated sql to ensure syntax and runtime
  result over pg database
- fix bug with orde by for table with undefined column

* chore: fix query import path

* chore: set ES target for lint

* chore: add github action for pg-meta test package

* chore: add tsconfig to sparse checkout
2025-03-20 19:04:58 +00:00
Bobbie Soedirgo 02f7e75b4d fix: use vault functions for creating/updating secrets (#33337) 2025-02-05 13:24:52 +08:00
Alaister Young 6c592dec99 chore: remove useExecuteSqlQuery() part 2 (#30467)
* foreign-key-constraints

* update entity-types stale time

* schemas query

* deprecate useExecuteSqlQuery

* users count query

* database size query

* indexes query

* keywords query

* migrations query

* table columns

* database functions

* database roles query

* fdws query

* replication lag query

* ongoing queries query

* vault secrets query

* remove unneeded staleTime: 0

* max connections query

* fix entity types key in tests

* Some fixes

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2024-11-18 05:15:37 +00:00
Alaister Young 3a27070dc2 chore(perf): table editor query optimisation (#30184)
* chore: table editor query optimisation

* removed unused queries and fix invalidations

* address feedback

* fix filtering for foreign tables

* Update

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2024-10-31 15:20:40 +08:00
Ivan Vasilov df52ea7ee0 feat: Replace all toasts with sonner (#28250)
* Update the design of the sonner toasts. Add the close button by default.

* Migrate studio and www apps to use the SonnerToaster.

* Migrate all toasts from studio.

* Migrate all leftover toasts in studio.

* Add a new toast component with progress. Use it in studio.

* Migrate the design-system app.

* Refactor the consent toast to use sonner.

* Switch docs to use the new sonner toasts.

* Remove toast examples from the design-system app.

* Remove all toast-related components and old code.

* Fix the progress bar in the toast progress component. Also make the bottom components vertically centered.

* Fix the width of the toast progress.

* Use text-foreground-lighter instead of muted for ToastProgress text

* Rename ToastProgress to SonnerProgress.

* Shorten the text in sonner progress.

* Use the correct classes for the close button. Add a const var for the default toast duration. Remove the custom width class from sonner.

* Set the position for all progress toasts to bottom right. Set the duration for all toasts to the default (when reusing a toast id from loading/progress toast, the duration is set to infinity).

* Fix the playwright tests.

* Refactor imports to use ui instead of @ui.

* Change all imports of react-hot-toast with sonner. These components were merged since the last commit to this branch.

* Remove react-hot-toast lib.

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Jonathan Summers-Muir <MildTomato@users.noreply.github.com>
2024-08-31 07:50:51 +08:00
Alaister Young 8e47d317fa chore: pg-meta functions (#26881)
* start pg-meta functions

* pg meta create function

* pg meta database functions update and delete

* fix ts

* update execute sql error type

* update execute sql error type

* remove duplicate database functions query
2024-06-03 23:21:19 +08:00
Joshen Lim 0c81551110 RQ convert mutateAsync calls to mutate where possible, otherwise add try finally blocks (#26737)
* RQ convert mutateAsync calls to mutate where possible, otherwise add try finally blocks

* Skip eligibility query if project status is coming up

* Small UI update to replicas side panel

* Update apps/studio/components/interfaces/SQLEditor/RenameQueryModal.tsx

Co-authored-by: Alaister Young <alaister@users.noreply.github.com>

* Add error handling in CIDR

* Prettier lint

---------

Co-authored-by: Alaister Young <alaister@users.noreply.github.com>
2024-05-24 16:58:49 +08:00
Kevin Grüneberg f9a55935f5 chore: use type imports for types/interfaces (#21738) 2024-03-04 20:48:22 +08:00
Ivan Vasilov 693294ec85 chore: Migrate VaultStore (#20020)
* Move listEncryptedColumns method to a function.

* Remove encryptColumn method since it's unused.

* Add CRUD RQ queries/mutations for vault secrets.

* Use the new queries/mutations.

* Add RQ for secret/decrypted values of secrets.

* Add RQ queries/mutations for pg-sodium keys.

* Use the new hook for decrypted value.

* Use the new hooks for pg sodium keys.

* Remove unneeded code from the vault store.

* Delete VaultStore and all usages of it.

* Small style fix

* Fix invalidation for create, update, delete secrets

* Small UI fixes to EditSecretModal

* Fix invalidation for pg-sodium-keys rq

* Add default error handler for pg sodium keys rq

* Undo

* Add default error handler for vault secrets RQ

* Remove observer on vault pages

* Fix database wrappers with vault

* Add loading state for vault keys

* fix pg sodium keys invalidation

* Opt for immutable const declaration in SecretsManagement

* remove keyId and keyName context from ColumnEditor, no longer supports TLE

* Address PR comments

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2024-01-04 15:15:45 +07:00