## Summary
The `/v1/database/:name_or_identity/sql` endpoint now calls the module's
`client_connected` reducer before executing SQL, and
`client_disconnected` after. This allows module authors to accept or
reject SQL connections based on the caller's identity, matching the
existing behavior of the `/call` endpoint.
## Motivation
Previously, the `/sql` endpoint bypassed the module's `onConnect` hook
entirely, meaning module authors had no way to restrict who could run
SQL queries against their database. The `/call` endpoint already runs
the connect hook, so this brings `/sql` to parity.
## Changes
- `crates/client-api/src/routes/database.rs`: The `sql` handler now:
1. Generates a random connection ID
2. Calls `module.call_identity_connected()` before executing SQL
3. Executes the SQL query
4. Calls `module.call_identity_disconnected()` after
5. If `client_connected` rejects the connection, returns 403 Forbidden
without executing the query
- `sql_direct()` is unchanged since it is also used by the pgwire
server, which has its own connection lifecycle.
## Behavior
- If the module defines a `client_connected` reducer that throws/errors
for a given identity, the SQL request returns `403 Forbidden`
- If no `client_connected` reducer is defined, behavior is unchanged
- The connection is always cleaned up via `client_disconnected` after
the query completes
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
* Fixes#3770.
# API and ABI breaking changes
Technically breaks certain PG Wire responses by fixing the incorrect
behavior.
# Expected complexity level and risk
### 2:
Should not cause any actual breaking, as it just changes the response
type from empty rows to execution metrics where there can never be a
meaningful response.
# Testing
- [x] Manual testing
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Update pgwire to 0.37
Make `DataRowEncoder` reusable as enabled in this release, make
introduce 3x performance on encoding.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
N/A
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
## Summary
Enable confirmed reads by default for all WebSocket subscriptions and
SQL queries. This is a 2.0 breaking change that improves data integrity.
### What changed
Previously, subscription updates and SQL results were sent to clients
immediately, before the transaction was confirmed durable. A server
crash could cause clients to have observed data that was lost.
Now the server defaults to `confirmed=true`. Clients receive updates
only after durability is confirmed. This adds a small latency cost but
guarantees that any data a client receives will survive a server
restart.
### Changes
**Server (2 files, 2 lines each):**
- `subscribe.rs`: `SubscribeQueryParams.confirmed` defaults to `true`
- `database.rs`: `SqlQueryParams.confirmed` defaults to `true`
**Documentation:**
- Migration guide updated with "Confirmed Reads Enabled by Default"
section
- Added to overview list and quick migration checklist
### Opt-out
Clients can opt out by explicitly passing `?confirmed=false` in the
WebSocket URL or using `.withConfirmedReads(false)` /
`.WithConfirmedReads(false)` / `.with_confirmed_reads(false)` in SDKs.
### Smoketest impact
Smoketests that don't explicitly pass `--confirmed` will now get
confirmed reads via the server default. This should not cause failures
-- confirmed reads only add a small wait for durability confirmation
before sending results. The `confirmed_reads.py` smoketest explicitly
passes `--confirmed` and continues to work as before.
### SDK impact
No SDK changes needed. SDKs only send the `confirmed` query parameter
when explicitly set by the user. When not set, the server default
applies -- which is now `true`.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
The goal of this PR is to:
- Make it easy to switch our backing implementation of identifiers, as
seen in the commit changing to `LeanString`.
- Improve type safety of our identifiers by having every identifier
start with `RawIdentifier` and then `Identifier` is just a wrapper with
validation on construction. `TableName` and `ReducerName` are then just
wrappers around `Identifier`.
- Reduce allocation in `InstanceEnv` by using the now clone-on-write
O(1) + SSO optimized `Identifier`.
- Reduce allocations in the query engine as a consequence of improving
`RawIdentifier` and `Identifier`.
- Have `&'static str` strings be further optimized by avoiding to
allocate even for long ones. This is supported by `impl From<&'static
str> for RawIdentifier` as well as switching to `LeanString`.
The PR results in a roughly 2k TPS improvement over master for WASM +
hash indices. This should also help V8, as this is VM-agnostic.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? Mostly just small changes in many places, but in a sea of small
changes, mistakes can happen.
# Testing
Covered by existing tests.
# Description of Changes
Closes#3673
*NOTE*: C++ part will be in another PR
# Expected complexity level and risk
2
Adding a new type touch everywhere
# Testing
- [x] Adding smoke and unit test
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).
Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.
# API and ABI breaking changes
We add a new type
# Expected complexity level and risk
2
it impacts all over the code
# Testing
- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Permissions for evaluating SQL/DML are not generally "actions", but more
a set of permissions that are checked during evaluation.
To make this work with the teams feature, this patch extends `AuthCtx`
to allow checking a set of permissions as mandated by the spec. This set
is a bit more fine-grained than "is owner", so as to avoid baking in the
concept of teams/collaborators, or assumptions about what a role might
entail. Both are likely to evolve in the future, so evaluation of
permissions / capabilities should be confined to the impl of the
`Authorization` trait.
Unlike "actions", the `AuthCtx` must be able to evaluate permission
checks quickly and without side-effects, nor can it enter an `async`
context. In that sense, it is precomputed (if you will), and stored as a
closure in the `AuthCtx` for external authorization.
A challenge posed is how to thread through the constructed `AuthCtx` for
subscriptions.
A tempting approach would have been to equip the `HostController` with
the ability to summon an `AuthCtx`. That, however, would have created a
gnarly circular dependency, because the `HostController` also controls
the controldb, which itself demands an `AuthCtx`.
Instead, the `AuthCtx` is obtained in the endpoint handler and passed to
each method call that requires one. That's less pretty, but more
effective.
---------
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
This is primarily to get their fixes for the tls negotiation deadloop
that we were experiencing.
[This is the
diff](https://github.com/sunng87/pgwire/compare/v0.32.1...v0.34.1) of
changes between 0.32.1 and 0.34.1.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
2? This was something we discussed wanting to do. It's a change to an
important dependency, but we know it comes with stuff we want.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
n/a - problem not in this repo.
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
Hello team, I'm the original author of pgwire library. In this patch, I
have some updates about spacetimedb's usage of pgwire library:
- Updated pgwire library to 0.34, which includes a critical fix for busy
loop on accepting new connection
- This version will return error on no-op handlers, as requested by
@mamcx
- Update `StartupHandler` for `SslRequest`. `SslRequest` is handled in
`process_socket` automatically before `StartupHandler` kicks in. So it's
safe to remove the match branch. The framework will reject TLS
negotiation automatically if no TLS support compiled
- Updated feature flags. Add `no-default-features` so we will not pull
TLS dependencies.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
Would be nice to verify if you have integration test on postgres
interface.
---------
Co-authored-by: Mario Montoya <mamcx@elmalabarista.com>
# Description of Changes
This exposes client credentials in reducer calls for rust.
# API and ABI breaking changes
API Changes:
The main API change is the addition of `AuthCtx` and the `sender_auth`
in `ReducerContext`. This also adds JwtClaims, which has some helpers
for getting commonly used claims.
ABI Changes:
This adds one new functions `get_jwt`. This uses
`st_connection_credentials` to look up the credentials associated with a
connection id.
This adds ABI version 10.2.
# Expected complexity level and risk
2. This adds new ABI functions
# Testing
I've done some manual testing with modified versions of the quickstart.
We should add some examples that use the new API.
# Description of Changes
Closes
[#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686).
Add support for listening using the [PG wire
protocol](https://www.postgresql.org/docs/current/protocol.html) so `pg`
clients could be used against the database.
# API and ABI breaking changes
The output of `duration` is changed to `rfc3339`, instead of the way is
made with `sats` because is what is done in `pg`, see note below.
# Expected complexity level and risk
2
~~There is open questions that are in the [ticket
#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686). Also
the crate used here require `RustTls`, so it could be good idea to
decide if~~:
* ~~Rewrite a big chunk of code to use `OpenSSL`~~
* ~~Move to `RustTls`
https://github.com/clockworklabs/SpacetimeDB/pull/1700~~
* ~~Pay for the extra compilation cost~~.
I open another port(`5433`) to listen for `pg` connections using `ssl`.
Need to be decided if this is the way or instead try to multi-plex the
current port for both protocols.
# Testing
Only manual testing so far. Solving the above questions allow me to
implement some unit tests. Also, not yet integrated into cloud for the
same reasons.
- [x] Adding some test for the binary encoding of special and primitive
types
- [x] Smoke test using `psql` that connect to the db instance and run
some queries
- [x] Manually inspect using a UI database explorer how infer the types,
some of this tools generate special widgets when displaying `json,
duration, etc`
---------
Co-authored-by: Noa <coolreader18@gmail.com>
clockwork-labs-bot