Commit Graph

32 Commits

Author SHA1 Message Date
Mazdak Farrokhzad d6bc325244 Define TableName and ReducerName backed by EcoString (#4137)
# Description of Changes

The first commit defines a type `TableName` that is used in e.g.,
`TxData` and where determined profitable and necessary to do this
change.
`TableName` is backed by
[`ecow::EcoString`](https://docs.rs/ecow/0.2.6/ecow/string/struct.EcoString.html)
which affords O(1) clones and 15 bytes of inline storage and
`mem::size_of::<EcoString>() == 16`.

The second commit does the same for `ReducerName`. This is also used in
reducer execution.

Together, these commits increase TPS by around 5-7k TPS.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

Covered by existing tests.
2026-01-27 23:20:30 +00:00
Shubham Mishra 2560846f22 Rust: client query builder (#4003)
# Description of Changes
Client Query builder for rust, as per proposal -
https://github.com/clockworklabs/SpacetimeDBPrivate/pull/2356.

1. Pach moves query builder to its separate crate, so that it can be
shared between module and sdk.
2. Implements `TypedSubscriptionBuilder` in `sdks/rust` as mentioned in
proposal
3. Modify codegen to extend types to support query builder as mentioned
in proposal
4. a test

# API and ABI breaking changes
NA, additive changes.

# Expected complexity level and risk
2

# Testing
Added a test.

---------

Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
2026-01-22 15:37:22 +00:00
Piotr Sarnacki 3c8836b1a3 Templates rework (#3879)
# Description of Changes

We would like to move all of the templates to a central directory

# API and ABI breaking changes

None

# Expected complexity level and risk

2

# Testing

---------

Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
2026-01-09 15:09:26 +00:00
Mazdak Farrokhzad 65c8a8a9da Fixes #3240, perf of indexing many rows with same key (#3971)
# Description of Changes

Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3240.

Non-unique indices are now backed by a type `SameKeyEntry` which holds
the `RowPointer`s for the same key.
When these `RowPointer`s exceed 4KiB (512 entries), the data structure
switches from using an array list to a hash set.

# API and ABI breaking changes

None

# Expected complexity level and risk

2?

# Testing

Covered by existing tests, though more test will come in future PRs.
2026-01-09 11:50:11 +00:00
Mario Montoya 82d5a4f6c0 Implement SpacetimeType for Result<T, E> (#3790)
# Description of Changes

Closes #3673 

*NOTE*: C++ part will be in another PR

# Expected complexity level and risk

2

Adding a new type touch everywhere

# Testing

- [x] Adding smoke and unit test

---------

Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
2026-01-08 15:50:18 +00:00
Mario Montoya 8fb0bcf922 Add UUID built-in convenience type to SpacetimeDB (#3538)
# Description of Changes

Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).

Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.

# API and ABI breaking changes

We add a new type 

# Expected complexity level and risk

2

it impacts all over the code

# Testing

- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.

---------

Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
2026-01-02 17:17:24 +00:00
Noa afe169ac4a Fix the issues with scheduling procedures (#3816)
# Description of Changes

This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.

The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue

So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.

Fixes #3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?

# Expected complexity level and risk

2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.

# Testing

- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
2025-12-05 22:27:30 +00:00
joshua-spacetime 6cf2372a3c Remove race condition from sdk test (#3804)
# Description of Changes

Moves a reducer call to inside an `on_insert` callback to avoid race
condition.

# API and ABI breaking changes

None

# Expected complexity level and risk

0

# Testing

Fixes a test
2025-12-02 05:53:42 +00:00
joshua-spacetime c49eea31da Fix view rewrite for delta tables (#3801)
# Description of Changes

Applies the same implicit filter for views to delta tables that we
already did for physical tables.

Removes a single condition `scan.delta.is_none()` from the already
existing rewrite rule.

# API and ABI breaking changes

None

# Expected complexity level and risk

0

# Testing

- [x] Rust sdk test
2025-12-01 21:15:32 +00:00
John Detter d26f3a10a9 Revert "Procedures: fix scheduling (#3704)" (#3774)
This reverts commit b2e37e8008.

# Description of Changes

<!-- Please describe your change, mention any related tickets, and so on
here. -->

Reverts #3704 which I'm pretty sure contains some sort of bug which is
causing the smoketests to hang.

# API and ABI breaking changes

None

<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->

# Expected complexity level and risk

1

<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.

This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

- [x] CI passing again
2025-11-26 17:06:11 +00:00
Mazdak Farrokhzad b2e37e8008 Procedures: fix scheduling (#3704)
# Description of Changes

Reworks how `SchedulerActor::handle_queued` works so that it first
determines the parameters of the call to a reducer or the parameters of
the call to the procedure. This also enables the removal of the special
case `call_scheduled_reducer`.

Fixes #3645.

# API and ABI breaking changes

None

# Expected complexity level and risk

2

# Testing

A test `schedule_procedure` is added.

---------

Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
2025-11-26 03:56:53 +00:00
joshua-spacetime 0a3dda7f4e Add rust sdk tests for views (#3755)
# Description of Changes

Rust SDK test suite for views

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

This patch only adds tests, it does not change functionality.
2025-11-25 17:13:03 +00:00
Mazdak Farrokhzad ed2a18cff7 Bump hashbrown, foldhash; Fix some compile errors in master (#3722)
# Description of Changes

There were mentions of `hashbrown` in the repo that did not go through
`spacetimedb_data_structures::map`.
This caused compile errors on master when running certain tests locally.
These have been replaced with the proper imports.

The PR also bump hashbrown to 0.16.1 and foldhash to 0.2.0.

# API and ABI breaking changes

None

# Expected complexity level and risk

2

# Testing

Covered by existing tests.
2025-11-25 12:17:24 +00:00
Noa fd524cf275 [TS] Anonymous transactions (#3743)
# Description of Changes

Mirrors the Rust API.

# Expected complexity level and risk

2

# Testing

- [x] Automated procedure testing now enabled for typescript (from
`sdks/rust/tests`)
2025-11-25 02:20:57 +00:00
Phoebe Goldman 64606ace26 Print internal error when a procedure call fails in sdk test client (#3725)
# Description of Changes

And also add a `log::warn` call next to a `tracing::warn` call, since
our tests don't seem to report tracing output.

In response to suspicious test flake
https://github.com/clockworklabs/SpacetimeDB/actions/runs/19577977435/job/56068366154?pr=3409
.

# API and ABI breaking changes

N/a

# Expected complexity level and risk

1

# Testing

N/a
2025-11-24 17:33:00 +00:00
Phoebe Goldman 7df8719b61 Add procedure HTTP request API for WASM modules and the Rust module bindings library (#3684)
# Description of Changes

Closes #3517 .

With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.

Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.

I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.

This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close #3644 .

Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
  Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
  rather than running it on the single-threaded database executor.

# API and ABI breaking changes

Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.

# Expected complexity level and risk

3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.

# Testing

- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.

---------

Co-authored-by: Noa <coolreader18@gmail.com>
2025-11-20 20:47:35 +00:00
Jason Larabie 87de672bb3 Fix C# module bindings codegen for namespaces with view dispatcher (#3668)
# Description of Changes

Closes: #3658 
With the work on Views we ran into trouble with the Return Type and
certain C# code layout where the types are defined inside the Module
class. This fixes the bug in our current approach but needs
consideration to possibly be changed entirely.

# API and ABI breaking changes

N/A

# Expected complexity level and risk

1 - Minor change

# Testing

- [x] Retested with the linked code with and without the namespace
- [x] Re-ran regression tests

---------

Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
2025-11-19 23:32:21 +00:00
Mazdak Farrokhzad 0a3251708a Add ProcedureContext::with_tx (#3638)
# Description of Changes

Adds `ProcedureContext::{with_tx, try_with_tx}`.

Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3515.

# API and ABI breaking changes

None

# Expected complexity level and risk

2

# Testing

An integration test `test_calling_with_tx` is added.
2025-11-19 20:33:02 +00:00
Phoebe Goldman 565e95bdf4 Add Rust client SDK bindings for calling procedures (#3532)
# Description of Changes

This commit adds support to the Rust client SDK for calling procedures.

Similar to reducers, each `DbContext` implementor has a `pub procedures:
RemoteProcedures` field, with methods provided by extension traits for
each procedure.

Unlike reducers, the provided methods are invoke and invoke-then.
Invoke-then takes a `FnOnce` callback to run
when the SDK is notified of the procedure's termination status, while
invoke ignores that notification.
No mechanism is provided for observing procedures invoked by other
clients.

Procedure callbacks are implemented by storing a map from `request_id`
to `ProcedureCallback`, with the callback closure internally knowing how
to deserialize the return value. It's mildly unfortunate to deserialize
within the callback instead of on the preprocess background task, but it
saves significant complexity.

This commit also adds a new sdk-test module, `sdk-test-procedures`, and
a new Rust test client, `procedure-client`.
Together, these are used in two tests of invoking and observing
procedures. I've left TODOs for other tests that we should write as we
implement additional procedure features.

I also had to fix a few minor bugs in the Rust codegen which were not
strictly related to procedures: we previously assumed that the sets of
reducers and of tables were non-empty, which led to wonky invalid
codegen on modules which did not define any reducers or which did not
define any tables.

I'm sneaking a change to the Nix flake into this PR as well - when
initially writing it I had included `cargoArtifacts` (the pre-built and
cached dependencies of our actual builds) in its `packages`, but that
was neither necessary or useful, and just made building the shell take a
long time after dependency changes.

# API and ABI breaking changes

Breaks the internal interface between the Rust client SDK and codegen,
so users will have to re-run `spacetime generate`.

# Expected complexity level and risk

2-ish? Pretty simple change to the Rust SDK overall.

# Testing

- [x] Added new automated integration tests exercising the new
functionality.

---------

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-11-10 19:12:23 +00:00
Phoebe Goldman 3f1de9e096 WASM host execution for procedures (#3498)
# Description of Changes

This commit builds support for executing procedures in WASM modules.
This includes an HTTP endpoint,
`/v1/database/:name_or_address/procedure/:name POST`, as well as an
extension to the WS protocol. These new APIs are not wired up to the CLI
or SDKs, but I have manually tested the HTTP endpoint via `curl`. The
new WS extensions are completely untested.

Several TODOs are scattered throughout the new code, most notably for
sensibly tracking procedure execution time in the metrics.

I also expect that we will want to remove the `procedure_sleep_until`
syscall and the `ProcedureContext::sleep_until` method prior to release.

# API and ABI breaking changes

Adds new APIs and ABIs.

# Expected complexity level and risk

3? 4? Unlikely to break existing stuff, 'cause it's mostly additive, but
adds plenty of potentially-fragile new stuff. Notably is the first time
we're doing anything actually `async`hronous on a database core Tokio
worker, and we don't yet have strong evidence of how that will affect
reducer execution.

# Testing

- [x] Manually published `modules/module-test` and executed procedures
with the following `curl` invocations:
- `curl -X POST -H "Content-Type:application/json" -d '[]'
http://localhost:3000/v1/database/module-test/procedure/sleep_one_second`
- `curl -X POST -H "Content-Type:application/json" -d '[1223]'
http://localhost:3000/v1/database/module-test/procedure/return_value`
- [ ] Need to write automated tests.

---------

Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2025-10-30 13:17:45 +00:00
Piotr Sarnacki 647be7e9c0 spacetime init rewrite (#3366)
This is a draft of the new functionality for `spacetime init`. In order
to run it with built-in templates you have to set the path to the config
file:

```
export SPACETIMEDB_CLI_TEMPLATES_FILE=crates/cli/.init-templates.json
```

In the future it will fetch the list from GH.

A few notes:

* the previous functionality of `spacetime init` does not work at the
moment
* the code needs a bit more cleanup and tests before merging
* there is a bit of a mix in how we generate empty server and client
projects. For Rust we use the existing way of generating. For TypeScript
we clone an empty project from the repo. I wanted to play with both ways
of doing things, and I'm still not sure which is better. Generation in
Rust means that the generated code will match the CLI version and not
necessarily whatever is in Git. On the other hand, for the builtin
templates we will be fetching the newest version from GH, which I guess
might also not what we want, ie. we probably want only stable templates.
More discussion is needed here
* we use `spacetimedb` directory for the server files
* I don't particularly like the inability to disable interactive mode
easily. We discussed disabling it by default if all of the required
arguments are passed, but I don't think it's feature proof. For example,
if someone relies on a non-interactive mode, and we add a new required
argument, instead of printing a message `missing --foo`, we will
automatically launch interactive mode, which is harder to debug. That's
why I think I'd prefer to implement `--non-interactive` argument
* it's kind of hard to keep the legacy behaviour. If you don't pass any
arguments, we go into interactive mode. In the legacy version, we would
print required arguments. If someone passes `--lang` or `--project-path`
explicitly, I guess we could run the legacy workflow, but not sure if
it's worth it, as the command was marked as unstable anyway
* the project path defaults to the project name, but I think we should
probably replace change whitespaces to dashes, or at least ask for the
project path with the project name being the default (or both)

---------

Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
2025-10-30 04:26:08 +00:00
Jeffrey Dallatezza bf37b2947b Add bindings for csharp modules to use JWT claims (#3414)
# Description of Changes

This exposes JWT claims for csharp modules, similar to how they are
exposed to rust modules in
https://github.com/clockworklabs/SpacetimeDB/pull/3288.

This adds the new types `AuthCtx` and `JwtClaims`, and adds an `AuthCtx`
to the `ReducerContext`.

`AuthCtx` represents the credentials associated with the request, and
`JwtClaims` represents a jwt token.

One difference from the rust version is that I didn't create helpers to
build an `AuthCtx` from a jwt payload. The reason is that we would need
to be able to compute the identity from the payload claims, which
requires a blake3 hash implementation. The first two c# libraries I
found had issues at runtime
([Blake3](https://www.nuget.org/packages/Blake3) is wrapping a rust
implementation, and
[HashifyNet](https://github.com/Deskasoft/HashifyNET/tree/main/HashifyNet/Algorithms/Blake3)
seems to be broken by our trimming because it uses reflection heavily).
I can look into taking the implementation from `HashifyNet`, since it is
MIT licensed, but I don't think we need to block merging on that.

# API and ABI breaking changes

This adds the new types `AuthCtx` and `JwtClaims`, and adds an `AuthCtx`
to the `ReducerContext`.

This also adds a csharp wrapper for the get_jwt ABI function added in
https://github.com/clockworklabs/SpacetimeDB/pull/3288.

# Expected complexity level and risk

2.

# Testing

This has a very minimal unit test of JwtClaims.

I manually tested using this locally with the csharp quickstart, and I
was able to print jwt tokens inside the module.
2025-10-21 19:17:33 +00:00
Jeffrey Dallatezza dbc49b1fd6 Add AuthCtx to ReducerContext for rust (#3288)
# Description of Changes

This exposes client credentials in reducer calls for rust.

# API and ABI breaking changes

API Changes:

The main API change is the addition of `AuthCtx` and the `sender_auth`
in `ReducerContext`. This also adds JwtClaims, which has some helpers
for getting commonly used claims.

ABI Changes:

This adds one new functions `get_jwt`. This uses
`st_connection_credentials` to look up the credentials associated with a
connection id.

This adds ABI version 10.2.

# Expected complexity level and risk

2. This adds new ABI functions

# Testing

I've done some manual testing with modified versions of the quickstart.
We should add some examples that use the new API.
2025-10-17 21:03:54 +00:00
Noa bb43213245 Typescript module API (#3327)
# Description of Changes

Currently based on #3361 

Implements most of the TS module API (not yet a function for type
aliases).

# Expected complexity level and risk

<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.

This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

- [x] Extremely basic module stuff works
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->

---------

Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-10-16 14:58:50 +00:00
Noa 619b8ce021 Bump Rust to 1.90 (#3397)
# Description of Changes

Necessary for pulling in rolldown.

# API and ABI breaking changes

None

# Expected complexity level and risk

1, with the caveat that this updates the Rust version and therefore
touches all the code.

# Testing

- [ ] Just the automated testing
2025-10-09 20:41:25 +00:00
Phoebe Goldman 351af50578 Support BytesSources other than the one for reducer args (#3294)
# Description of Changes

Each reducer gets its arguments through an `ArgSource`, a Unix-file-like
abstraction for streams of bytes. Prior to this commit, we had an ABI
designed as if it could support other args sources, but it actually
hardcoded the ID of the reducer args source, and errored elsewhere.

This commit extends the `BytesSource` infrastructure to support other
bytes sources. This will be useful for exposing JWT payloads and HTTP
responses. No other `BytesSource` uses are actually included in this
commit, only the infrastructure.

This commit also defines a new host call,
`bytes_source_remaining_length`. This is intended to allow callers to
pre-allocate a buffer correctly sized to read the entire `BytesSource`
all at once. The new host function is added to a new ABI minor version,
10.1, so that old SpacetimeDB hosts can detect and reject too-new
compiled modules. I have added uses of this new function to
`__call_reducer__` in both Rust and C#, even though it's not strictly
necessary,
and I haven't removed the loop which repeatedly calls
`bytes_source_read` and grows the buffer.

# API and ABI breaking changes

Adds a new ABI minor version, `spacetime_10.1`. This means that old
SpacetimeDB hosts will reject new compiled modules.

# Expected complexity level and risk

2-ish? WASM ABI code is always fiddly, but this is a pretty simple case.

# Testing

- [x] New behavior and new host function are both hit through existing
tests that instantiate modules and call reducers against them, so I
believe automated testing is sufficient.

---------

Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2025-09-29 15:32:58 -04:00
Kim Altintop ad01a9ead7 Rust SDK: Make confirmed parameter optional (#3283)
For consistency with [TypeScript] and [C#].

If `DbConnectionBuilder::with_confirmed_reads` is not called, don't set
the parameter on the connection URL, so that the server can choose the
default.

[TypeScript]: https://github.com/clockworklabs/SpacetimeDB/pull/3247
[C#]: https://github.com/clockworklabs/SpacetimeDB/pull/3282

# Expected complexity level and risk

1
2025-09-24 17:44:57 +00:00
Jeffrey Dallatezza 1d08167ebd Store client credentials in a new system table (#2983)
# Description of Changes

This adds a new system table to store the jwt payloads of connected
clients. I'm planning to use this system table to expose client claims
to modules in subsequent PRs.

The new table is called `st_connection_credentials`. It is a **private**
system table which stores a mapping from `connection_id` to
`jwt_payload`. Note that a jwt payload is a json representation of the
clients claims, not a fully signed token.

The times when we need to insert and delete these rows closely mirrors
that of the existing `st_client` table, with 1.5 exceptions:
1. We weren't previously inserting to `st_client` until after the
`OnConnect` reducer ran (even though it was in the same transaction). We
want `st_connection_credentials` to be populated before calling the
reducer, so that the reducer can use it get the credentials, so I made a
change to insert to `st_client` and `st_connection_credentials` before
calling the reducer.
2. This difference has not actualized, but when clients start sending
refresh tokens, we will probably need to update the credentials stored
in this table.

This also enforces uniqueness of connection ids. A duplicate connection
id will now make the on-connect reducer fail (since it will violate
uniqueness when trying to insert to `st_connection_credentials`).

# Expected complexity level and risk

2.5

Adding a system table is a bit risky. This is almost rollback safe, with
one annoying case that is worth calling out:

If a database is created with this system table, opening it with an
older version of spacetimedb will only work if there is a snapshot of
the database. If we try to load a table without a snapshot, replaying
will fail on the first row for that table. This is because we don't
write the table schema information to the commit log when creating a
database. In practice, this is unlikely to be an issue, because new
databases asynchronously trigger a snapshot immediately after creation.

Migrating existing databases will be fine. On startup this will detect
that there is a missing system table, and add it in a way that writes it
to the commit log. Since it is in the commit log, we can open the
database with an older version and still understand the data for that
table.

# Testing

There are unit tests that cover opening a database created with an older
version (which doesn't have this table).

I manually tested opening a migrated database with an older version of
spacetimedb.
2025-09-19 15:39:45 +00:00
Kim Altintop f394de32d9 Confirmed reads (#3133)
# Description of Changes

Implements [subscribing to durable
commits](https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1594).

The setting works on a per-connection level, and essentially just delays
sending transaction updates until the transaction is reported as durable
by the database.

For connectionless SQL operations, the setting works per-request. No SQL
syntax is provided by this patch to toggle the configuration.

After some deliberation, I opted to obtain the offset when a transaction
commits (as opposed to when it starts). This creates some mild
inconvenience, because we prevent the transaction from committing until
the corresponding subscription updates are enqueued.
The strategy is, however, more correct should we ever support weaker
isolation levels, and it is easier to document.

Follow-ups include:

- Provide SQL syntax (`SET synchronous_commit = ON` or something)
- C# and TypeScript SDKs
- Reference docs?
 

# API and ABI breaking changes

Not breaking, but adds a parameter to the subscribe and sql endpoints.


# Expected complexity level and risk

4

To the author's understanding, ordering of outbound messages is not
changed by this patch, even if there are messages that don't have a
transaction offset (such as error messages). I.e. while waiting for the
transaction offset of a message to become durable, no message enqueued
after that message will be delivered. This may not be desirable in some
cases.

The patch may contain concurrency bugs, e.g. awaiting futures that may
never resolve.


# Testing

- [x] Implemented a new test in the `module_subscription_actor` module
- [x] Added unit tests for the core logic in `ClientConnectionReceiver` 

It would be desirable to also have integration-level tests, but I'm
currently unsure how to write those without being able to control if and
when the database reports an offset as durable.

---------

Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
2025-09-08 17:51:04 +00:00
Tyler Cloutier 8b0e5b1ee3 Remove version number and git commit from all but one file in codegen (#3216)
# Description of Changes

Closes https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1987

# API and ABI breaking changes

This does not affect any APIs but it does affect the code generated

# Expected complexity level and risk

1

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

- [ ] Snap tests
2025-09-05 00:44:20 +00:00
Zeke Foppa b9d30991f1 Fix remaining LICENSE files (#3193)
# Description of Changes

Apparently, I missed several license files in #3002. I'm not sure what
method I was using to find them, but apparently it was insufficient.

**This replaces all empty `LICENSE` files with an explicit (symlink to)
BSL license, and all apache licenses with symlinks to the root apache
license.** This PR does not intentionally change any license terms, so
if you see one that changed, **it's a mistake**.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

```bash
$ find . -name '*LICENSE*' -type f | grep -v '\.meta$'
./crates/sqltest/standards/LICENSE # this one is an external library that we are not allowed to re-license
./LICENSE.txt # this is the root license
```

---------

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-08-22 19:36:47 +00:00
Zeke Foppa b6fdc6c3a4 Move crates/sdk to sdks/rust (#3181)
# Description of Changes

I'm moving `crates/sdk` to `sdks/rust` to be more in line with where the
rest of our SDKs are listed. I updated the corresponding paths etc. that
pointed to the previous location.

This PR is based on
https://github.com/clockworklabs/SpacetimeDB/pull/3185, because if we
merge this without that, our release scripts will be broken.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

- [x] Existing CI passes

---------

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-08-20 19:24:42 +00:00