# Description of Changes
This PR updates Rust SDK subscription unsubscribe handling to return an
error instead of panicking when the internal pending mutation channel is
closed.
## Motivation
`SubscriptionHandle::unsubscribe_then` currently calls `unwrap()` after
sending an unsubscribe mutation through the internal pending mutation
channel. If the connection or subscription manager has already shut
down, that send can fail with a disconnected channel error, causing the
application to panic.
I encountered this in [`bevy_stdb`](https://github.com/onx2/bevy_stdb),
which tracks subscriptions across reconnects and unsubscribes old
handles after applying a new subscription for the same key.
## Changes
- Replaces `unwrap()` on `pending_mutation_sender.unbounded_send(...)`
with error propagation.
- Returns `crate::Error::Internal(...)` when the unsubscribe mutation
cannot be sent.
- Updates local unsubscribe state only after the unsubscribe mutation is
successfully queued.
# API and ABI breaking changes
None expected.
This changes an internal error path from panicking to returning an
existing `crate::Error::Internal(...)` through the existing
`crate::Result<()>` return type.
# Expected complexity level and risk
Complexity: 1/5
This is a small, localized change. The method already returns
`crate::Result<()>`, so replacing the `unwrap()` with error propagation
fits the existing API shape.
Risk is low. Callers now receive an error if the internal pending
mutation channel is closed instead of panicking. Existing `AlreadyEnded`
and `AlreadyUnsubscribed` behavior is unchanged.
# Testing
- [x] Verified that `unsubscribe_then` returns
`Err(crate::Error::Internal(...))` instead of panicking when the pending
mutation channel is disconnected.
- [ ] Verified that successful unsubscribe behavior is unchanged.
- [ ] Verified that `AlreadyEnded` is still returned when the
subscription has already ended.
- [ ] Verified that `AlreadyUnsubscribed` is still returned when
unsubscribe was already requested.
---------
Signed-off-by: Jeff Rooks <onx2rj@gmail.com>
# Description of Changes
This pull request improves the handling of connection lifecycle events
in the Rust client SDK for SpacetimeDB, particularly distinguishing
between connection failures and disconnections. It introduces a new
`ConnectionLifecycle` state machine to track connection progress,
ensures that the correct callback (`on_connect_error` or
`on_disconnect`) is invoked based on the connection state.
**Changes**
* `ConnectionLifecycle` enum to track the connection state
(`Connecting`, `Connected`, `Ended`)
* Refactored error handling so that if a connection fails before
establishment, the `on_connect_error` callback is invoked; if the
connection fails after establishment, the `on_disconnect` callback is
invoked. See `end_connection`.
* Updated where disconnections are handled
(`advance_one_message_blocking`, `advance_one_message_async`, and
message processing) to use `finish_connection`
* Improved handling of user-initiated disconnects during the connection
process to avoid reporting them as connection errors and to ensure
proper cleanup.
# API and ABI breaking changes
I guess maybe if people relied on the `on_connect_error` to actually
fire the `on_disconnect` then this changes that behavior.
# Expected complexity level and risk
Maybe a 2? Seems pretty low risk but I'm still new to the codebase,
please double check.
This doesn't fix the websocket issues, that'll be for another day. I
noticed websocket.rs has some places it just drops and the error isn't
handled properly. We could technically surface that information and run
our callbacks with more specific error messages.
# Testing
I had an agent build and run loads of tests for this but didn't commit
those since it would have made the PR massive. I was planning on testing
locally though to see if I could trigger a connection failure at some
point, maybe via an invalid access token.
# Description of Changes
Tests for case conversion.
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
---------
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Adds a `sdk-test-procedure-cs` test to the other test modules in
`sdks/rust/tests/test.rs`.
Functionally `sdk-test-procedure-cs` performs all the operations as the
Rust test `sdk-test-procedure` but through C#, fulfilling the
requirements of Issue #3951
# API and ABI breaking changes
No API or ABI changes, this only adds a test.
# Expected complexity level and risk
1
# Testing
- [X] Ran `cargo test -p spacetimedb-sdk csharp_procedures` locally
without errors.
# Description of Changes
This PR adapts the Rust SDK test suite to work with the wasm version
added in https://github.com/clockworklabs/SpacetimeDB/pull/4089 (which
I've closed in favor of this PR).
Most of the changes revolve around wasm's different async semantics -
everything runs in one thread, so things that relied on background
threads didn't work directly. Several tests would lock up because
something in them blocked synchronously, which blocked any background
work from progressing.
We moved the test-clients contents into a `test_handlers.rs` so that it
could be called from both `main` (for native tests) and `lib` (for wasm
tests). To show what actually changed, use:
```bash
git diff --no-index -- <(git show origin/master:sdks/rust/tests/procedure-client/src/main.rs) sdks/rust/tests/procedure-client/src/test_handlers.rs
```
(or similar for other test-clients)
# API and ABI breaking changes
None, I think/hope.
# Expected complexity level and risk
2
# Testing
- [x] I've augmented the CI to also run the test suite with the `web`
feature
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Thales R <thlsrmsdev@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This bug was introduced when we added the v2 subscription protocol.
Before this change, when a v2 client disconnected or unsubscribed, other
clients could stop receiving updates for sender-scoped views they were
subscribed to.
This happened because `SubscriptionManager::remove_all_subscriptions()`
was cleaning up v2 subscription state correctly, but we would also run
the legacy v1 cleanup loop which mixed v1 and v2 queries. This meant the
same query hash could be processed again in the v1 cleanup loop after
the v2 removal had already happened. In the sender-view case, that
second pass could remove metadata (`indexes` / `search_args`) for active
subscribers.
Specifically, we could have the following sequence:
1. Client B disconnects.
2. The v2 loop removes B’s v2 subscription for `query_hash`.
3. `query_hash` now has no subscribers, so
`remove_query_from_tables(query_hash)` runs once.
4. That compatibility loop then visits `query_hash` again via
`subscription_ref_count.keys()`.
5. `query_hash` still has no subscribers, so
`remove_query_from_tables(query_hash)` runs a second time.
and when `remove_query_from_tables(query_hash)` runs a 2nd time, it
calls `index_ids.delete_index_ids_for_query()` which is refcounted, not
idempotent. So calling it twice for the same query decrements the shared
index count twice, and the 2nd decrement drops an index that client A’s
query still needs.
The fix was to:
1. Collect the disconnected client’s v1 query hashes from
`client_info.v1_subscriptions`
2. Run the v2 cleanup loop over `client_info.v2_subscriptions` (as we
did before this change)
3. Run the old compatibility cleanup loop only over the deduplicated v1
hashes
# API and ABI breaking changes
None
# Expected complexity level and risk
2
Small fix to a rather intricate bug. Most of the code in this patch is
testing code.
# Testing
This bug was not reproducible with the current smoketest harness,
because `spacetime subscribe` still uses the `v1` subscription protocol.
I added a Rust SDK test, which did reproduce the bug before the fix,
because the Rust SDK uses the `v2` subscription protocol.
# Description of Changes
Same change set as
https://github.com/clockworklabs/SpacetimeDB/pull/4614, just targeting
master.
The return type of a query builder view is now a special SATS product
type `{ __query__: T }`. A view with this return type now has a primary
key if `T` has a primary key. This means that client codegen will
generate an `OnUpdate` callback for such views. It will also generate
query builder index bindings for the primary key column.
# API and ABI breaking changes
None. Old modules with query builder views still work, they just don't
have primary keys.
# Expected complexity level and risk
2
# Testing
Added equivalent tests to the ones that were added in #4573 and #4572.
# Description of Changes
In the Rust client SDK, with this PR, doing
`.with_debug_to_file("path.txt")` on the connection builder will result
in the SDK logging additional verbose info to `path.txt`.
# API and ABI breaking changes
Adds a new user-facing-ish API to the Rust client SDK.
# Expected complexity level and risk
2? Some possibility of deadlock due to adding a new mutex, but this is
explicitly not for production use.
# Testing
- [x] Ran `chat-console-rs` locally with this enabled and got some debug
logs out of it.
# Description of Changes
Query builder views return a subset of rows from a physical table. If
that table has a primary key, then so should the view. What this means
concretely is that the view should expose the same api as the table,
specifically as it relates to the primary key column.
With that in mind, this patch commits the following changes:
1. Annotates `ViewDef` with a `primary_key`
2. Updates the return type of query builder views in the raw module def
to a special product type
3. Adds an index for the primary key on the view's backing table
4. Updates the query planner to use this index
5. Updates rust client codegen to generate `on_update` for such views
# API and ABI breaking changes
None
Old `impl Query` views compiled with an older version of SpacetimeDB
will continue to work as they did before - without a primary key.
# Expected complexity level and risk
3
# Testing
- [x] New rust sdk integration suite exercising `on_update` for PK views
and semijoin scenarios
- [x] Smoketests for PK views and semijoin scenarios
# Description of Changes
- Migrated the C++ module-definition assembly path to V10-first
internals:
- Added v10_builder and module_type_registration systems.
- Switched Module::__describe_module__ to serialize RawModuleDef with
V10 payload.
- Updated macro registration pipeline to register through V10
- Added explicit naming support across macro surface (*_NAMED variants
for reducer/procedure/
view and field/index macros).
- Reworked multi-column index macros (FIELD_MultiColumnIndex,
FIELD_MultiColumnIndex_NAMED) with
migration alias.
- Added SPACETIMEDB_SETTING_CASE_CONVERSION(...) to support case
conversion policy
- Error-path hardening by adding explicit constraint-registration error
tracking and preinit validation
- Codegen updates:
- Updated C++ moduledef regen to V10 builder types.
- Adjusted C++ codegen duplicate-variant wrapper generation to emit
proper product-type
wrappers.
- Test/harness updates:
- type-isolation-test runner now defaults to focused V10 regression
checks; --v9 runs broader
legacy/full suite.
- Added focused modules for positive/negative V10 checks:
- test_multicolumn_index_valid
- error_multicolumn_missing_field
- error_default_missing_field
- Re-enabled C++ paths in sdks/rust/tests/test.rs procedure/view/test
suites.
# API and ABI breaking changes
- Refactor of the underlying module definition
- New *_NAMED variant macros for explicit canonical naming
- FIELD_NamedMultiColumnIndex renamed to FIELD_MultiColumnIndex
# Expected complexity level and risk
3 - Large set of changes moving over to V10 with underlying changes to
make future updates a little easier
# Testing
- [x] Ran the type isolation test and expanded it
- [x] Ran the spacetimedb-sdk test framework to confirm no more drift
between C++ and other module languages
- [x] Ran Unreal test suite though not really applicable
- [x] New app creation with `spacetime init --template basic-cpp`
- [x] Ran describe module tests against Rust + C# matching with C++ on
the /modules/sdk-test* modules to find any possible mis-alignment
# Review
- [x] Another look at the new features with C++
- [x] Thoughts on *_NAMED macros, I couldn't come up with a better
solution with C++20
# Description of Changes
Update the Default casing policy to `snake_case` for `RawModuleDefV10`.
Messy PR contains changes at different places, so that CI can pass:
Here are the main changes as follows:
- `bindings-macro` & `bindings` crate: `name` macro in Indexes for
canonical name and supply it to `RawModuleDefV10` via `ExplicitNames`.
- `bindings-typescript`:
- Changes has been reviewed through this PR -
https://github.com/clockworklabs/SpacetimeDB/pull/4308.
- `binding-csharp`: a single line change to pass `sourceName` of index
instead of null.
- `codegen`:
- Changes has been merged from branch -
https://github.com/clockworklabs/SpacetimeDB/pull/4337.
- Except a fix in rust codegen to use canonical name in Query buillder
instead of accessor.
- `lib/db/raw_def`: Extends `RawDefModuleV10` structure to support case
conversion.
- `schema` crate:
- `validate/v9` - Nothing itself should change or changes in v9
validation logic but the file contains a `CoreValidator` which is shared
with `validate/v10`. No test have t be updated to `validate/v9` which
ensures we aren't regressing it.
- `validate/v10`: This is the main meat, look at the new tests added in
bottom to understand what it does.
- Rest of the files are either test updates or module bindings.
## Testing:
1. Extensive unit tests have been added to verify generated `ModuleDef`
is correct.
2. I have done some e2e testing to verify rust codegen with rust and
typescript modules.
3. I would have like to do more testing for other codegens , I am
continue doing .
I have removed `sql.py` smoketest, as that seems to be already migated
in new framework and was headache to update.
## Expected complexity level and risk
4, It could have side-effect which aren't easily visible.
- - -
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
Prior to this commit, if a reducer call failed during a test in the Rust
SDK test suite, the client would hang, as it generally didn't register
callbacks when invoking reducers. This could mask some bugs, causing the
tests to fail with TIMEOUT rather than a useful message. For example, we
relied on consistency between all of the test modules, which was broken
by our not having updated the C++ bindings library to use the new
case-conversion scheme. This meant that the client was sending reducer
calls with incorrect reducer names to at least one database per test
run, which would result in a `Panic` status or an `Err(Err(_))` outcome.
With this commit, the SDK test clients are updated to inspect the exit
status of reducers they invoke, and (except for intentional reducer
failures) to panic the test client when encountering a reducer error or
panic.
# API and ABI breaking changes
N/a
# Expected complexity level and risk
1
# Testing
- [x] Automated tests passed locally.
- [x] Intentionally broke reducer calls via the 2.0 WS API, causing them
to skip the call and unconditionally respond with `Status::Panic`.
Observed "loud" failures in the test suite with non-`TIMEOUT` output
that looked more debuggable to me.
# Description of Changes
Basic tests for templates, the test tries to generate each of the
template, build and publish the SpacetimeDB part, and build or type
check the client side.
This PR also includes bindings for all of the clients as it's needed for
the tests to work and given the choices to either generate during the
test run or include the bindings in git we chose the latter. This makes
it consistent across the templates as some of them already included
bindings.
# Expected complexity level and risk
1
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
This PR implements support for the `spacetime.json` configuration file
that can be used to set up common `generate` and `publish` targets. An
example of `spacetime.json` could look like this:
```
{
"dev_run": "pnpm dev",
"generate": [
{ "out-dir": "./foobar", "module-path": "region-module", "language": "c-sharp" },
{ "out-dir": "./global", "module-path": "global-module", "language": "c-sharp" },
],
"publish": {
"database": "bitcraft",
"module-path": "spacetimedb",
"server": "local",
"children": [
{ "database": "region-1", "module-path": "region-module", server: "local" },
{ "database": "region-2", "module-path": "region-module", server: "local" }
]
}
}
```
With this config, running `spacetime generate` without any arguments
would generate bindings for two targets: `region-module` and
`global-module`. `spacetime publish` without any arguments would publish
three modules, starting from the parent: `bitcraft`, `region-1`, and
`region-2`. On top of that, the command `pnpm dev` would be executed
when using `spacetime dev`.
It is also possible to pass additional command line arguments when
calling the `publish` and `generate` commands, but there are certain
limitations. There is a special case when passing either a module path
to generate or a module name to publish. Doing that will filter out
entries in the config file that do not match. For example, running:
```
spacetime generate --project-path global-module
```
would only generate bindings for the second entry in the `generate`
list.
In a similar fashion, running:
```
spacetime publish region-1
```
would only publish the child database with the name `region-1`
Passing other existing arguments is also possible, but not all of the
arguments are available for multiple configs. For example, when running
`spacetime publish --server maincloud`, the publish command would be
applied to all of the modules listed in the config file, but the
`server` value from the command line arguments would take precedence.
Running with arguments like `--bin-path` would, however, would throw an
error as `--bin-path` makes sense only in a context of a specific
module, thus this wouldn't work: `spacetime publish --bin-path
spacetimedb/target/debug/bitcraft.wasm`. I will throw an error unless
there is only one entry to process, thus `spacetime publish --bin-path
spacetimedb/target/debug/bitcraft.wasm bitcraft` would work, as it
filters the publish targets to one entry.
# API and ABI breaking changes
None
# Expected complexity level and risk
3
The config file in itself is not overly complex, but when coupled with
the CLI it is somewhat tricky to get right. There are also some changes
that I had to make to how clap arguments are validated - because the
values can now come from both the config file and the clap config, we
can't use some of the built-in validations like `required`, or at least
I haven't found a clean way to do so.
# Testing
I've added some automated tests, but more tests and manual testing is
coming.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: bradleyshep <148254416+bradleyshep@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Standardizes the query builder API across all three language SDKs (Rust,
TypeScript, C#) for consistency.
**Rust:**
- Rename `Query` struct to `RawQuery`, make `Query` a trait with `fn
into_sql(self) -> String`
- All builder types (`Table`, `FromWhere`, `LeftSemiJoin`,
`RightSemiJoin`) implement `Query<T>` trait
- Views can return `-> impl Query<T>` instead of specifying exact
builder types
- The `#[view]` macro auto-detects `impl Query<T>` and rewrites to
`RawQuery<T>`
- Add `Not` variant to `BoolExpr` with `.not()` method
**TypeScript:**
- Add `ne()` to `ColumnExpression`
- Refactor `BooleanExpr` to `BoolExpr` class with chainable `.and()`,
`.or()`, `.not()` methods
- Make builders valid queries directly (`.build()` deprecated but still
works)
- Deprecate `from()` wrapper — use `tables.person.where(...)` directly
- Merge `query` export into `tables` so table refs are also query
builders
- Add subscription callback form: `subscribe(ctx =>
ctx.from.person.where(...))`
- Unify `useTable` with query builder syntax; deprecate `filter.ts`
**C#:**
- Add `Not()` method to `BoolExpr<TRow>`
- Add `IQuery<TRow>` interface implemented by all builder types
(`Table`, `FromWhere`, `LeftSemiJoin`, `RightSemiJoin`, `Query`)
- Add `ToSql()` to all builder types so `.Build()` is no longer required
- Update `AddQuery` to accept `IQuery<TRow>` instead of `Query<TRow>`
# API and ABI breaking changes
- Rust: `Query<T>` is now a trait (was a struct). The struct is renamed
to `RawQuery<T>`. This is a breaking change for any code that used
`Query<T>` as a type directly.
- TypeScript: `BooleanExpr` is now a `BoolExpr` class (was a
discriminated union type). The `query` export is deprecated in favor of
`tables`.
- C#: `AddQuery` now accepts `Func<QueryBuilder, IQuery<TRow>>` instead
of `Func<QueryBuilder, Query<TRow>>`. Existing `.Build()` calls still
work since `Query<TRow>` implements `IQuery<TRow>`.
# Expected complexity level and risk
3 — Changes touch multiple language SDKs and codegen, but each
individual change is straightforward. The Rust macro rewrite for `impl
Query<T>` detection is the most complex piece. All existing
`.build()`/`.Build()` calls continue to work.
# Testing
- [x] `cargo test -p spacetimedb-query-builder` — 16/16 tests pass
- [x] `cargo check -p spacetimedb` — clean, no warnings
- [x] `cargo check` on views-query, views-sql, views-basic,
views-trapped smoketest modules — all clean
- [x] `cargo test -p spacetimedb-codegen codegen_csharp` — snapshot
updated, passes
- [x] `npm test` (TypeScript) — 101/101 tests pass
- [x] C# QueryBuilder tests — new tests for `Not()`, `IQuery<T>`
interface
- [ ] CI passes
# Description of Changes
Update the Rust client SDK to use the new V2 WebSocket format, and
present the V2 user-facing API.
## Reducer events
### Remove on-reducer callbacks
It's no longer possible to observe reducers called by other clients by
registering callbacks with `ctx.reducers.on_{my_reducer}`. We no longer
code-generate those methods, or the associated
`ctx.reducers.remove_on_{my_reducer}`. Internal plumbing for storing and
invoking those callbacks is also removed.
### Add specific reducer invocation callbacks
In addition to the previous way to invoke reducers,
`ctx.reducers.{my_reducer}(args...)`, we add a method that registers a
callback to run after the reducer is finished. This method has the
suffix `_then`, as in `ctx.reducers.{my_reducer}_then(args...,
callback)`.
The callback will accept two arguments:
- `ctx: &ReducerEventContext`, the same context as was previously passed
to on-reducer callbacks.
- `status: Result<Result<(), String>, InternalError>`, denoting the
outcome of the reducer.
- `Ok(Ok(())` means the reducer committed. This corresponds to
`ReducerOutcome::Ok` or `ReducerOutcome::Okmpty` in the new WS format.
- `Ok(Err(message))` means the reducer returned an "expected" or "user"
error. This corresponds to `ReducerOutcome::Err` in the new WS format.
- `Err(internal_error)` means something went wrong with host execution.
This corresponds to `ReducerOutcome::InternalError` in the new WS
format.
Internally, the SDK stores the callbacks in its `ReducerCallbacks` map.
This is keyed on `request_id: u32`, a number that is generated for each
reducer call (from an `AtomicU32` that we increment each time), and
included in the `ClientMessage::CallReducer` request. The
`ServerMessage::ReducerResult` includes the same `request_id`, so the
SDK pops out of the `ReducerCallbacks` and invokes the appropriate
callback when processing that message.
These new callbacks are very similar to the existing procedure
callbacks.
### The `Event` exposed to row callbacks
Row callbacks caused by a reducer invoked by this client will see
`Event::Reducer`, the same as they would prior to this PR. These
callbacks will be the result of a `ServerMessage::ReducerResult` with
`ReducerOutcome::Ok`. In order to expose the reducer name and arguments
to this event, the client stores them in its `ReducerCallbacks` map,
alongside the callback for when the reducer is complete.
Row callbacks caused by any other reducer, or any non-reducer
transaction, are now indistinguishable to the client. These will see
`Event::Transaction`, which is renamed from the old
`Event::UnknownTransaction`.
### Less metadata in `ReducerEvent`
Some metadata is removed from `ReducerEvent`, as the V2 WebSocket format
no longer publishes it, even to the caller.
## `CallReducerFlags` are removed
All machinery for setting, storing and applying call reducer flags is
removed from the SDK, as the new WS format does not have any non-default
flags.
## Requesting rows in unsubscribe
When sending a `ClientMessage::Unsubscribe`, we always request that the
server include the matching rows in its response
`ServerMessage::UnsubscribeApplied`. This saves us having to update the
SDK to store query sets separately, at least for now. (We'll do that
later.)
## Handling rows
The new SDK does some additional parsing to wrangle rows in the new
WebSocket format into the same internal data structures as before,
rather than re-writing the client cache. (We'll do that later.)
Specifically, parsing of `DbUpdate` is changed so that:
- We parse raw `TransactionUpdate` into the generated `DbUpdate` type,
which requires an additional loop compared to the previous version, to
cope with the new WS format's dividing updates by query set. We define a
function `transaction_update_iter_table_updates` which encapsulates this
nested loop in an iterator.
- We have two new functions for parsing raw `QueryRows` into the
generated `DbUpdate` type, one for when they come from a
`SubscribeApplied`, and the other when they come from an
`UnsubscribeApplied`. `QueryRows` from `SubscribeApplied` translate to a
`DbUpdate` of all inserts, while one from `UnsubscribeApplied` will be
all deletes.
## Legacy subscriptions
"Legacy subscriptions" are removed. These were only used for
`subscribe_to_all_tables`, which as of now is stubbed. I will follow up
with a change to re-implement `subscribe_to_all_tables` by
code-generating a list of all known tables, and having it subscribe to
`select * from {table}` for every table in that list.
## `subscribe_to_all_tables` via a list
Previously, `subscribe_to_all_tables` worked by sending a legacy
subscription with the query `SELECT * FROM *`, which the host had
special handling to expand to subscribing to all tables. As legacy
subscriptions are no longer usable in V2 clients, this can't work.
Instead, we code-generate `SpacetimeModule::ALL_TABLE_NAMES`, a list of
all the known table names. `subscribe_to_all_tables` then maps across
this list to construct a list of queries in the form `SELECT * FROM
{table_name}`, and subscribes to all of those queries. This has the
upside that defining a new table in the module without regenerating
client bindings will no longer result in the client seeing rows of
tables it does not know about and cannot parse.
## Light mode removed
Light mode is no longer meaningful in the V2 WS format, so all code
related to it is removed.
## Internal changes
### Renamed WS messages
The SDK's internal code is updated to account for various renames:
- `QueryId` -> `QuerySetId`, `query_id` -> `query_set_id`.
- `SubscribeMulti` -> `Subscribe`, `UnsubscribeMulti` -> `Unsubscribe`.
## Incidental changes in this PR, not necessary for other client SDKs
### Don't filter out empty ranges in `RowSizeHint`
The Rust implementation of `RowSizeHint` in `BsatnRowList` got regressed
in the base branch to not work with zero-sized rows. This change fixes
that.
# API and ABI breaking changes
Boy howdy is it!
# Expected complexity level and risk
3? Changes ended up being less complicated than I feared, but we do have
some fiddly code here, and we have internal dependencies on the SDK.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Updated automated test suite.
- Known failures:
- [ ] `subscribe_all_select_star`, which is currently broken because
it's trying to subscribe to rows from private tables. #4241 will fix
this.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
Co-authored-by: = <cloutiertyler@gmail.com>
# Description of Changes
This adds the v2 websocket protocol and adds support on the server side.
For context on many of the changes/decisions, you can look at the
discussion on https://github.com/clockworklabs/SpacetimeDB/pull/4023.
To restate some of the key changes:
- The reducer event information is no longer sent with transaction
updates (because we don't want to broadcast reducer call information
anymore).
- If a client calls a reducer, they are sent a `ReducerResult` which
includes the outcome of the reducer call and and related row updates for
queries that the client is subscribed to.
- We no longer dedupe queries that appear in multiple query sets for the
same client. This is because we are moving toward per-query storage.
- Related to that, Unsubscribe requests have an option to send the
related rows. We need this for now, since clients don't have per-query
storage implemented yet.
- We don't have the json format in v2.
Notes for reviewers:
- This moves around the messages in
`crates/client-api-messages/src/websocket` (into `common`, `v1`, and
`v2`), and this renaming of existing messages adds a lot of noise to the
PR.
- In many places, I chose to duplicate a lot of code to have a v1
version and a v2 version. I went with this to make it easier to remove
the v1 version in the future (hopefully we can just fully delete most of
the v1 functions).
- `module_subscription_manager.rs` has probably has the biggest changes,
since we now track queries by query_set_id, and we get to remove some
complexity of v1's FormatSwitch.
<!-- Please describe your change, mention any related tickets, and so on
here. -->
# API and ABI breaking changes
The v1 protocol still works, though we won't send the reducer event info
for v10 modules.
# Expected complexity level and risk
4. This touches a lot of places.
# Testing
Unit testing is pretty minimal for the new code paths. I've done some
manual e2e testing with the typescript quickstart, and this has been
tested with a different branch implementing the v2 rust client.
---------
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Modules are like programs, databases are like processes. Your client
connects to a remote database, not a remote module.
<!-- Please describe your change, mention any related tickets, and so on
here. -->
# API and ABI breaking changes
Yep!
# Expected complexity level and risk
2? I made this change with find + replace, and it's possible that I
borked the edit, or missed some reference somehow. It seems unlikely,
however, that this change will have deeper implications we haven't
considered.
# Testing
N/a
# Description of Changes
Updated the codegen table/function iteration functions to take in a
parameter to check visibility in all locations for the supported
languages.
- Updated the util.rs functions for iterating tables/functions to check
for a CodegenVisibility enum (IncludePrivate, or OnlyPublic)
- Added a new CodegenOptions struct to pass around the CodegenVisibility
and future flags, defaulted visibility to OnlyPublic
- Updated the CLI to return a list of all private tables not included
(added a TODO to check the --include-private opt):
```bash
Optimising module with wasm-opt...
Build finished successfully.
Skipping private tables during codegen: secret_note, secret_order, secret_person.
Generate finished successfully.
```
# API and ABI breaking changes
Technically API breaking as the private tables will no longer be
available. (GitHub labels are not working at the moment)
# Expected complexity level and risk
1 - Simple change the testing took longer
# Testing
Turns out when you remove private tables you invalidate most of the
module_bindings across the system!
- [x] Rust test SDK for all languages
- [x] C# SDK tests
- [x] C# dotnet tests
- [x] Updated and checked snap files
- [x] Updated Blackholio (Unreal + Unity) module_bindings and tested
- [x] Ran Unreal SDK tests
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
NOTE: Cherry-picking
https://github.com/clockworklabs/SpacetimeDB/pull/4127 from the
`2.0-breaking-changes` branch.
## Original PR Description
This changes generated types in ts client bindings. We currently
generate a few different types of types: reducer args, procedure args,
rows, and user defined types. To avoid potential conflicts between these
types (for example, if a user defined a type called `FooRow`, and also
had a tabled named `foo`, we would end up with two types named
`FooRow`), this puts each set of types in a different file and
namespace. We also stopped exporting the `xxxRow` types, because there
is always another type generated for those. We now have a `types`
directory, which has an `index.ts` with user defined types, along with
`reducers.ts` and `procedures.ts` for the types generated for
reducer/procedure parameters.
```
import type * as Types from './module_bindings/types';
var currentMessages: Types.Message[] = [];
```
or
```
import { type Message } from './module_bindings/types';
var currentMessages: Message[] = [];
```
This has a couple other changes:
- For procedure and reducer types, this adds a suffix of `Args`, since
we may want types for the return values in the future.
- For all of the types, instead of exposing the schema object, we are
now giving the typescript type (e.g. `export type Message =
__Infer<typeof MessageRow>;`). I couldn't think of a reason for users to
want the schema object, so this should save users from needing to do all
of the `Infer` boilerplate.
This is a breaking change for v2.
2. This only changes typescript, and it should generally make thing
easier to use.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Blocks procedures from requesting private ip ranges after dns
resolution.
Adds a new cargo feature to `spacetimedb-standalone` permitting loopback
http requests in test environments only.
# API and ABI breaking changes
None
# Expected complexity level and risk
2. I may have missed a range.
# Testing
- [x] Unit tests for IP address matching
- [x] Smoketests for blocking a private IP address
# Description of Changes
This adds C++ server bindings (/crate/bindings-cpp) to allow writing C++
20 modules.
- Emscripten WASM build system integration with CMake
- Macro-based code generation (SPACETIMEDB_TABLE, SPACETIMEDB_REDUCER,
etc)
- All SpacetimeDB types supported (primitives, Timestamp, Identity,
Uuid, etc)
- Product types via SPACETIMEDB_STRUCT
- Sum types via SPACETIMEDB_ENUM
- Constraints marked with FIELD* macros
# API and ABI breaking changes
None
# Expected complexity level and risk
2 - Doesn't heavily impact any other areas but is complex macro C++
structure to support a similar developer experience, did have a small
impact on init command
# Testing
- [x] modules/module-test-cpp - heavily tested every reducer
- [x] modules/benchmarks-cpp - tested through the standalone (~6x faster
than C#, ~6x slower than Rust)
- [x] modules/sdk-test-cpp
- [x] modules/sdk-test-procedure-cpp
- [x] modules/sdk-test-view-cpp
- [x] Wrote several test modules myself
- [x] Quickstart smoketest [Currently in progress]
- [ ] Write Blackholio C++ server module
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
The first commit defines a type `TableName` that is used in e.g.,
`TxData` and where determined profitable and necessary to do this
change.
`TableName` is backed by
[`ecow::EcoString`](https://docs.rs/ecow/0.2.6/ecow/string/struct.EcoString.html)
which affords O(1) clones and 15 bytes of inline storage and
`mem::size_of::<EcoString>() == 16`.
The second commit does the same for `ReducerName`. This is also used in
reducer execution.
Together, these commits increase TPS by around 5-7k TPS.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
Client Query builder for rust, as per proposal -
https://github.com/clockworklabs/SpacetimeDBPrivate/pull/2356.
1. Pach moves query builder to its separate crate, so that it can be
shared between module and sdk.
2. Implements `TypedSubscriptionBuilder` in `sdks/rust` as mentioned in
proposal
3. Modify codegen to extend types to support query builder as mentioned
in proposal
4. a test
# API and ABI breaking changes
NA, additive changes.
# Expected complexity level and risk
2
# Testing
Added a test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
We would like to move all of the templates to a central directory
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
---------
Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3240.
Non-unique indices are now backed by a type `SameKeyEntry` which holds
the `RowPointer`s for the same key.
When these `RowPointer`s exceed 4KiB (512 entries), the data structure
switches from using an array list to a hash set.
# API and ABI breaking changes
None
# Expected complexity level and risk
2?
# Testing
Covered by existing tests, though more test will come in future PRs.
# Description of Changes
Closes#3673
*NOTE*: C++ part will be in another PR
# Expected complexity level and risk
2
Adding a new type touch everywhere
# Testing
- [x] Adding smoke and unit test
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).
Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.
# API and ABI breaking changes
We add a new type
# Expected complexity level and risk
2
it impacts all over the code
# Testing
- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.
The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue
So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.
Fixes#3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?
# Expected complexity level and risk
2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.
# Testing
- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
# Description of Changes
Moves a reducer call to inside an `on_insert` callback to avoid race
condition.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Testing
Fixes a test
# Description of Changes
Applies the same implicit filter for views to delta tables that we
already did for physical tables.
Removes a single condition `scan.delta.is_none()` from the already
existing rewrite rule.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Testing
- [x] Rust sdk test
This reverts commit b2e37e8008.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Reverts #3704 which I'm pretty sure contains some sort of bug which is
causing the smoketests to hang.
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI passing again
# Description of Changes
Reworks how `SchedulerActor::handle_queued` works so that it first
determines the parameters of the call to a reducer or the parameters of
the call to the procedure. This also enables the removal of the special
case `call_scheduled_reducer`.
Fixes#3645.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
A test `schedule_procedure` is added.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
Rust SDK test suite for views
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
This patch only adds tests, it does not change functionality.
# Description of Changes
There were mentions of `hashbrown` in the repo that did not go through
`spacetimedb_data_structures::map`.
This caused compile errors on master when running certain tests locally.
These have been replaced with the proper imports.
The PR also bump hashbrown to 0.16.1 and foldhash to 0.2.0.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
Covered by existing tests.
# Description of Changes
Closes#3517 .
With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.
Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.
I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.
This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close#3644 .
Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
rather than running it on the single-threaded database executor.
# API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.
# Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.
# Testing
- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Closes: #3658
With the work on Views we ran into trouble with the Return Type and
certain C# code layout where the types are defined inside the Module
class. This fixes the bug in our current approach but needs
consideration to possibly be changed entirely.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Minor change
# Testing
- [x] Retested with the linked code with and without the namespace
- [x] Re-ran regression tests
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
Adds `ProcedureContext::{with_tx, try_with_tx}`.
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3515.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
An integration test `test_calling_with_tx` is added.
# Description of Changes
This commit adds support to the Rust client SDK for calling procedures.
Similar to reducers, each `DbContext` implementor has a `pub procedures:
RemoteProcedures` field, with methods provided by extension traits for
each procedure.
Unlike reducers, the provided methods are invoke and invoke-then.
Invoke-then takes a `FnOnce` callback to run
when the SDK is notified of the procedure's termination status, while
invoke ignores that notification.
No mechanism is provided for observing procedures invoked by other
clients.
Procedure callbacks are implemented by storing a map from `request_id`
to `ProcedureCallback`, with the callback closure internally knowing how
to deserialize the return value. It's mildly unfortunate to deserialize
within the callback instead of on the preprocess background task, but it
saves significant complexity.
This commit also adds a new sdk-test module, `sdk-test-procedures`, and
a new Rust test client, `procedure-client`.
Together, these are used in two tests of invoking and observing
procedures. I've left TODOs for other tests that we should write as we
implement additional procedure features.
I also had to fix a few minor bugs in the Rust codegen which were not
strictly related to procedures: we previously assumed that the sets of
reducers and of tables were non-empty, which led to wonky invalid
codegen on modules which did not define any reducers or which did not
define any tables.
I'm sneaking a change to the Nix flake into this PR as well - when
initially writing it I had included `cargoArtifacts` (the pre-built and
cached dependencies of our actual builds) in its `packages`, but that
was neither necessary or useful, and just made building the shell take a
long time after dependency changes.
# API and ABI breaking changes
Breaks the internal interface between the Rust client SDK and codegen,
so users will have to re-run `spacetime generate`.
# Expected complexity level and risk
2-ish? Pretty simple change to the Rust SDK overall.
# Testing
- [x] Added new automated integration tests exercising the new
functionality.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This commit builds support for executing procedures in WASM modules.
This includes an HTTP endpoint,
`/v1/database/:name_or_address/procedure/:name POST`, as well as an
extension to the WS protocol. These new APIs are not wired up to the CLI
or SDKs, but I have manually tested the HTTP endpoint via `curl`. The
new WS extensions are completely untested.
Several TODOs are scattered throughout the new code, most notably for
sensibly tracking procedure execution time in the metrics.
I also expect that we will want to remove the `procedure_sleep_until`
syscall and the `ProcedureContext::sleep_until` method prior to release.
# API and ABI breaking changes
Adds new APIs and ABIs.
# Expected complexity level and risk
3? 4? Unlikely to break existing stuff, 'cause it's mostly additive, but
adds plenty of potentially-fragile new stuff. Notably is the first time
we're doing anything actually `async`hronous on a database core Tokio
worker, and we don't yet have strong evidence of how that will affect
reducer execution.
# Testing
- [x] Manually published `modules/module-test` and executed procedures
with the following `curl` invocations:
- `curl -X POST -H "Content-Type:application/json" -d '[]'
http://localhost:3000/v1/database/module-test/procedure/sleep_one_second`
- `curl -X POST -H "Content-Type:application/json" -d '[1223]'
http://localhost:3000/v1/database/module-test/procedure/return_value`
- [ ] Need to write automated tests.
---------
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
This is a draft of the new functionality for `spacetime init`. In order
to run it with built-in templates you have to set the path to the config
file:
```
export SPACETIMEDB_CLI_TEMPLATES_FILE=crates/cli/.init-templates.json
```
In the future it will fetch the list from GH.
A few notes:
* the previous functionality of `spacetime init` does not work at the
moment
* the code needs a bit more cleanup and tests before merging
* there is a bit of a mix in how we generate empty server and client
projects. For Rust we use the existing way of generating. For TypeScript
we clone an empty project from the repo. I wanted to play with both ways
of doing things, and I'm still not sure which is better. Generation in
Rust means that the generated code will match the CLI version and not
necessarily whatever is in Git. On the other hand, for the builtin
templates we will be fetching the newest version from GH, which I guess
might also not what we want, ie. we probably want only stable templates.
More discussion is needed here
* we use `spacetimedb` directory for the server files
* I don't particularly like the inability to disable interactive mode
easily. We discussed disabling it by default if all of the required
arguments are passed, but I don't think it's feature proof. For example,
if someone relies on a non-interactive mode, and we add a new required
argument, instead of printing a message `missing --foo`, we will
automatically launch interactive mode, which is harder to debug. That's
why I think I'd prefer to implement `--non-interactive` argument
* it's kind of hard to keep the legacy behaviour. If you don't pass any
arguments, we go into interactive mode. In the legacy version, we would
print required arguments. If someone passes `--lang` or `--project-path`
explicitly, I guess we could run the legacy workflow, but not sure if
it's worth it, as the command was marked as unstable anyway
* the project path defaults to the project name, but I think we should
probably replace change whitespaces to dashes, or at least ask for the
project path with the project name being the default (or both)
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This exposes JWT claims for csharp modules, similar to how they are
exposed to rust modules in
https://github.com/clockworklabs/SpacetimeDB/pull/3288.
This adds the new types `AuthCtx` and `JwtClaims`, and adds an `AuthCtx`
to the `ReducerContext`.
`AuthCtx` represents the credentials associated with the request, and
`JwtClaims` represents a jwt token.
One difference from the rust version is that I didn't create helpers to
build an `AuthCtx` from a jwt payload. The reason is that we would need
to be able to compute the identity from the payload claims, which
requires a blake3 hash implementation. The first two c# libraries I
found had issues at runtime
([Blake3](https://www.nuget.org/packages/Blake3) is wrapping a rust
implementation, and
[HashifyNet](https://github.com/Deskasoft/HashifyNET/tree/main/HashifyNet/Algorithms/Blake3)
seems to be broken by our trimming because it uses reflection heavily).
I can look into taking the implementation from `HashifyNet`, since it is
MIT licensed, but I don't think we need to block merging on that.
# API and ABI breaking changes
This adds the new types `AuthCtx` and `JwtClaims`, and adds an `AuthCtx`
to the `ReducerContext`.
This also adds a csharp wrapper for the get_jwt ABI function added in
https://github.com/clockworklabs/SpacetimeDB/pull/3288.
# Expected complexity level and risk
2.
# Testing
This has a very minimal unit test of JwtClaims.
I manually tested using this locally with the csharp quickstart, and I
was able to print jwt tokens inside the module.
# Description of Changes
This exposes client credentials in reducer calls for rust.
# API and ABI breaking changes
API Changes:
The main API change is the addition of `AuthCtx` and the `sender_auth`
in `ReducerContext`. This also adds JwtClaims, which has some helpers
for getting commonly used claims.
ABI Changes:
This adds one new functions `get_jwt`. This uses
`st_connection_credentials` to look up the credentials associated with a
connection id.
This adds ABI version 10.2.
# Expected complexity level and risk
2. This adds new ABI functions
# Testing
I've done some manual testing with modified versions of the quickstart.
We should add some examples that use the new API.
# Description of Changes
Currently based on #3361
Implements most of the TS module API (not yet a function for type
aliases).
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Extremely basic module stuff works
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Necessary for pulling in rolldown.
# API and ABI breaking changes
None
# Expected complexity level and risk
1, with the caveat that this updates the Rust version and therefore
touches all the code.
# Testing
- [ ] Just the automated testing
# Description of Changes
Each reducer gets its arguments through an `ArgSource`, a Unix-file-like
abstraction for streams of bytes. Prior to this commit, we had an ABI
designed as if it could support other args sources, but it actually
hardcoded the ID of the reducer args source, and errored elsewhere.
This commit extends the `BytesSource` infrastructure to support other
bytes sources. This will be useful for exposing JWT payloads and HTTP
responses. No other `BytesSource` uses are actually included in this
commit, only the infrastructure.
This commit also defines a new host call,
`bytes_source_remaining_length`. This is intended to allow callers to
pre-allocate a buffer correctly sized to read the entire `BytesSource`
all at once. The new host function is added to a new ABI minor version,
10.1, so that old SpacetimeDB hosts can detect and reject too-new
compiled modules. I have added uses of this new function to
`__call_reducer__` in both Rust and C#, even though it's not strictly
necessary,
and I haven't removed the loop which repeatedly calls
`bytes_source_read` and grows the buffer.
# API and ABI breaking changes
Adds a new ABI minor version, `spacetime_10.1`. This means that old
SpacetimeDB hosts will reject new compiled modules.
# Expected complexity level and risk
2-ish? WASM ABI code is always fiddly, but this is a pretty simple case.
# Testing
- [x] New behavior and new host function are both hit through existing
tests that instantiate modules and call reducers against them, so I
believe automated testing is sufficient.
---------
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Jeff Rooks