Commit Graph

4 Commits

Author SHA1 Message Date
Charis 205cbe7d26 chore(studio}: enforce import order, remove bare import specifiers (#44585) 2026-04-07 20:34:10 -04:00
Matt Rossman e8ab92408f feat(assistant): enable Braintrust tracing for non-sensitive chats (#42963)
Enables Braintrust tracing for AI Assistant chats to support debugging
and future online evals.

**Code Changes**

- Wraps `generateAssistantResponse` in a Braintrust `traced()` span,
logging the user's latest message as input along with metadata
(`chatId`, `chatName`, `projectRef`, `userId`, `orgId`, `planId`, etc.)
- Threads JWT claims from `apiWrapper` → handler to log `userId` in
Braintrust without an extra API call (+ expanded `apiWrapper` tests)
- Threads `orgId` and `planId` from `getOrgAIDetails` to log in
Braintrust

**Infrastructure Changes**

- Created a "Vercel" service account in Braintrust
- Added `BRAINTRUST_API_KEY` and `BRAINTRUST_PROJECT_ID` env vars to the
studio-staging project in Vercel using a service token for the above
service account
- Added an "Overview" view to the Logs tab in the Braintrust Assistant
project to surface the new metadata

**Precautions**

- HIPAA sensitive projects are excluded from logging (see
https://github.com/supabase/supabase/pull/42787 for the detection logic)
- Production is temporarily excluded from logging until we're confident
in the setup

**Testing steps**

- Chat with the AI Assistant in the [studio-staging preview
build](https://github.com/supabase/supabase/pull/42963#issuecomment-3917178023)
below
- Visit the [Logs tab in the Braintrust Assistant
project](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs)
and inspect the trace

<img width="4680" height="962" alt="CleanShot 2026-02-18 at 17 43 55@2x"
src="https://github.com/user-attachments/assets/c3a11b21-4e7f-4e90-bdab-a25ab8ee0d1f"
/>

<img width="2632" height="1288" alt="CleanShot 2026-02-18 at 17 45
04@2x"
src="https://github.com/user-attachments/assets/6c7b6ebc-5090-4ede-8f71-859ff7e386aa"
/>

**References**
- https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel
- https://www.braintrust.dev/docs/instrument/custom-tracing

Closes AI-438
2026-02-19 11:43:47 -05:00
Kevin Grüneberg f0436716f9 perf: use getClaims for API endpoint auth (#39311)
Use getClaims instead of getUser which avoids a network call to GoTrue to validate the user in case of asymmetric keys - this shaves off a good amount of latency for every API call.
2025-10-08 09:59:27 +08:00
Jordi Enric 6e91494b16 Add coveralls integration (#35424)
* update gh action, update vitest config

* debug

* debug cov

* idk try something different

* test2

* test3

* add base path

* rm debug

* add apiAuthenticate tests

* supabaseClient tests

* apiWrappers tests

* add apiHelpers tests

* add configcat tests

* add formatSql tests

* add github tests

* add cloudprovider utils tests

* add helpers tests

* fix typeerr

* add missing readonly err

* fix typeerrrs

* fix type errors in apiWrapper tests

* fix apiHelpers test

* add packages/ui tests

* add coveralls flags

* try coveralls parallel config

* fix coveralls parallel config

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-05-08 12:23:37 +02:00