# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bumps version to 2.2.0
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
- 1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Version number is correct (`2.2.0`)
- [x] BSL license file has been updated with the new date and version
number
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Closes#4751
# Description of Changes
- Add `useProcedure` React hook that mirrors `useReducer` which returns
a stable, typed callback that queues calls until the connection is ready
- Add `ProcedureParamsType` and `ProcedureReturnType` utility types to
`type_utils.ts`
- Update TypeScript codegen to emit `export const procedures =
__convertToAccessorMap(proceduresSchema.procedures)` in generated module
bindings, matching the existing pattern for reducers
```ts
import { procedures } from './module_bindings';
import { useProcedure } from 'spacetimedb/react';
const doSomeThing = useProcedure(procedures.doSomeThing);
const result = await doSomeThing({ foo: "..." });
```
# API and ABI breaking changes
None. Additive only — new hook export and new codegen line.
# Expected complexity level and risk
Low. The hook is a near-copy of `useReducer` adapted for procedure
signatures. The codegen change adds one line following the identical
pattern used for reducers.
# Testing
- [x] TypeScript SDK compiles with no new errors (`tsc --noEmit`)
- [x] All 170 existing tests pass (17 test files)
- [x] Tested end-to-end in a React app calling a procedure
---------
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Adds a non-repeating scheduled reducer integration test to the existing
Rust and C# integration tests, in order to address #3213
# API and ABI breaking changes
No
# Expected complexity level and risk
1
# Testing
- [X] Ran `cargo test -p spacetimedb-testing` and all tests passed
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
Fixes#4584. Now we check, fully recursively, whether compound
`AlgebraicTypeUse`s contain a ref.
# Expected complexity level and risk
1
# Testing
- [x] Added a test to ensure correct codegen
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
This PR adapts the Rust SDK test suite to work with the wasm version
added in https://github.com/clockworklabs/SpacetimeDB/pull/4089 (which
I've closed in favor of this PR).
Most of the changes revolve around wasm's different async semantics -
everything runs in one thread, so things that relied on background
threads didn't work directly. Several tests would lock up because
something in them blocked synchronously, which blocked any background
work from progressing.
We moved the test-clients contents into a `test_handlers.rs` so that it
could be called from both `main` (for native tests) and `lib` (for wasm
tests). To show what actually changed, use:
```bash
git diff --no-index -- <(git show origin/master:sdks/rust/tests/procedure-client/src/main.rs) sdks/rust/tests/procedure-client/src/test_handlers.rs
```
(or similar for other test-clients)
# API and ABI breaking changes
None, I think/hope.
# Expected complexity level and risk
2
# Testing
- [x] I've augmented the CI to also run the test suite with the `web`
feature
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Thales R <thlsrmsdev@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Bump versions to 2.1.0
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
CI
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bumps version to 2.0.5
# API and ABI breaking changes
None - this is just a version bump
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License file has been properly updated, including the date change
- [x] CI is passing other than the internal check
# Description of Changes
In the Rust client SDK, with this PR, doing
`.with_debug_to_file("path.txt")` on the connection builder will result
in the SDK logging additional verbose info to `path.txt`.
# API and ABI breaking changes
Adds a new user-facing-ish API to the Rust client SDK.
# Expected complexity level and risk
2? Some possibility of deadlock due to adding a new mutex, but this is
explicitly not for production use.
# Testing
- [x] Ran `chat-console-rs` locally with this enabled and got some debug
logs out of it.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bumps version to 2.0.4
# API and ABI breaking changes
None - this is just a version bump
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License file has been updated with correct version + time
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Adds implicit query builder conversions from `bool` to `BoolExpr` so
that you can write:
```rust
ctx.from.user().r#where(|u| u.online)
```
instead of
```rust
ctx.from.user().r#where(|u| u.online.eq(true))
```
Also removes `NullableCol` and `NullableIxCol` types from C# query
builder.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Unit and smoketests
# Description of Changes
This patch contains two main changes:
1. It makes `accessor` a required argument for table-level index defs in
typescript to align with rust.
2. It removes unsound index typing in the TypeScript bindings by
splitting index data into two explicit representations:
- `indexes`: declarative user config (`IndexOpts`) used for type
inference
- `resolvedIndexes`: normalized runtime metadata (`UntypedIndex`)
derived from `RawTableDefV10`
`TableCacheImpl` now builds index accessors from `resolvedIndexes`
instead of re-casting `indexes` at runtime.
This addressed the following comment:
```
// TODO: horrible horrible horrible. we smuggle this
`Array<UntypedIndex>`
// by casting it to an `Array<IndexOpts>` as `TableToSchema` expects.
// This is then used in `TableCacheImpl.constructor` and who knows where
else.
// We should stop lying about our types.
```
> Why?
We were conflating two different concepts under `tableDef.indexes`:
1. declared table-level index options authored by users
2. resolved runtime index definitions (including field-level inferred
indexes)
This required unsafe casts (`T['idxs']` / `UntypedIndex`) and made the
type model unsound.
Note, (2) was largely ai assisted.
# API and ABI breaking changes
Technically breaks the module api, although I believe this is the
behavior that is outlined in the spec, and so the current behavior
should really be considered a bug.
# Expected complexity level and risk
2
# Testing
Added unit tests covering the following:
1. Table-level explicit index without accessor throws.
2. Table-level duplicate accessor throws.
3. Table-level explicit accessor is accepted and used.
4. Field-level `.index(...)` derives accessor from the field name
(`displayName`).
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade 2.0.3
# API and ABI breaking changes
- None, this is a version bump
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
- 1 - this is a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License has been updated
- [x] Version number is correct (2.0.3)
# Description of Changes
Bumping all versions to 2.0.2 for the release.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
CI
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Just bumping versions to 2.0.1.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Update the Default casing policy to `snake_case` for `RawModuleDefV10`.
Messy PR contains changes at different places, so that CI can pass:
Here are the main changes as follows:
- `bindings-macro` & `bindings` crate: `name` macro in Indexes for
canonical name and supply it to `RawModuleDefV10` via `ExplicitNames`.
- `bindings-typescript`:
- Changes has been reviewed through this PR -
https://github.com/clockworklabs/SpacetimeDB/pull/4308.
- `binding-csharp`: a single line change to pass `sourceName` of index
instead of null.
- `codegen`:
- Changes has been merged from branch -
https://github.com/clockworklabs/SpacetimeDB/pull/4337.
- Except a fix in rust codegen to use canonical name in Query buillder
instead of accessor.
- `lib/db/raw_def`: Extends `RawDefModuleV10` structure to support case
conversion.
- `schema` crate:
- `validate/v9` - Nothing itself should change or changes in v9
validation logic but the file contains a `CoreValidator` which is shared
with `validate/v10`. No test have t be updated to `validate/v9` which
ensures we aren't regressing it.
- `validate/v10`: This is the main meat, look at the new tests added in
bottom to understand what it does.
- Rest of the files are either test updates or module bindings.
## Testing:
1. Extensive unit tests have been added to verify generated `ModuleDef`
is correct.
2. I have done some e2e testing to verify rust codegen with rust and
typescript modules.
3. I would have like to do more testing for other codegens , I am
continue doing .
I have removed `sql.py` smoketest, as that seems to be already migated
in new framework and was headache to update.
## Expected complexity level and risk
4, It could have side-effect which aren't easily visible.
- - -
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
All types are now defined and exported in `module_bindings/types.ts`,
meanings there's only one module you need to import from to access
types.
# Expected complexity level and risk
2
# Testing
n/a, no change in behavior.
## Description of Changes
This PR primarily affects the `bindings-macro` and `schema` crates to
review:
### Core changes
1. Replaces the `name` macro with `accessor` for **Tables, Views,
Procedures, and Reducers** in Rust modules.
2. Extends `RawModuleDefV10` with a new section for:
* case conversion policies
* explicit names
New sections are not validated in this PR so not functional.
3. Updates index behavior:
* Index names are now always **system-generated** for clients. Which
will be fixed in follow-up PR when we start validating RawModuleDef with
explicit names.
* The `accessor` name for an index is used only inside the module.
## Breaking changes (API/ABI)
1. **Rust modules**
* The `name` macro must be replaced with `accessor`.
2. **Client bindings (all languages)**
* Index names are now system-generated instead of using explicitly
provided names.
**Complexity:** 3
A follow-up PR will reintroduce explicit names with support for case
conversion.
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Re-opened from #4289 which was accidentally merged into its base branch
(`rekhoff/csharp-module-defs-v10-no-et`) instead of `master`.
This is the same PR, now correctly targeting `master`.
---
Original description from #4289:
This PR migrates the C# SDK client path to WebSocket v2 semantics and
aligns behavior with the Rust SDK direction for 2.0, including follow-on
fixes in generation/tests.
See #4289 for full description.
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Implements the rest of the casing proposal.
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
Standardizes the query builder API across all three language SDKs (Rust,
TypeScript, C#) for consistency.
**Rust:**
- Rename `Query` struct to `RawQuery`, make `Query` a trait with `fn
into_sql(self) -> String`
- All builder types (`Table`, `FromWhere`, `LeftSemiJoin`,
`RightSemiJoin`) implement `Query<T>` trait
- Views can return `-> impl Query<T>` instead of specifying exact
builder types
- The `#[view]` macro auto-detects `impl Query<T>` and rewrites to
`RawQuery<T>`
- Add `Not` variant to `BoolExpr` with `.not()` method
**TypeScript:**
- Add `ne()` to `ColumnExpression`
- Refactor `BooleanExpr` to `BoolExpr` class with chainable `.and()`,
`.or()`, `.not()` methods
- Make builders valid queries directly (`.build()` deprecated but still
works)
- Deprecate `from()` wrapper — use `tables.person.where(...)` directly
- Merge `query` export into `tables` so table refs are also query
builders
- Add subscription callback form: `subscribe(ctx =>
ctx.from.person.where(...))`
- Unify `useTable` with query builder syntax; deprecate `filter.ts`
**C#:**
- Add `Not()` method to `BoolExpr<TRow>`
- Add `IQuery<TRow>` interface implemented by all builder types
(`Table`, `FromWhere`, `LeftSemiJoin`, `RightSemiJoin`, `Query`)
- Add `ToSql()` to all builder types so `.Build()` is no longer required
- Update `AddQuery` to accept `IQuery<TRow>` instead of `Query<TRow>`
# API and ABI breaking changes
- Rust: `Query<T>` is now a trait (was a struct). The struct is renamed
to `RawQuery<T>`. This is a breaking change for any code that used
`Query<T>` as a type directly.
- TypeScript: `BooleanExpr` is now a `BoolExpr` class (was a
discriminated union type). The `query` export is deprecated in favor of
`tables`.
- C#: `AddQuery` now accepts `Func<QueryBuilder, IQuery<TRow>>` instead
of `Func<QueryBuilder, Query<TRow>>`. Existing `.Build()` calls still
work since `Query<TRow>` implements `IQuery<TRow>`.
# Expected complexity level and risk
3 — Changes touch multiple language SDKs and codegen, but each
individual change is straightforward. The Rust macro rewrite for `impl
Query<T>` detection is the most complex piece. All existing
`.build()`/`.Build()` calls continue to work.
# Testing
- [x] `cargo test -p spacetimedb-query-builder` — 16/16 tests pass
- [x] `cargo check -p spacetimedb` — clean, no warnings
- [x] `cargo check` on views-query, views-sql, views-basic,
views-trapped smoketest modules — all clean
- [x] `cargo test -p spacetimedb-codegen codegen_csharp` — snapshot
updated, passes
- [x] `npm test` (TypeScript) — 101/101 tests pass
- [x] C# QueryBuilder tests — new tests for `Not()`, `IQuery<T>`
interface
- [ ] CI passes
# Description of Changes
Update the Rust client SDK to use the new V2 WebSocket format, and
present the V2 user-facing API.
## Reducer events
### Remove on-reducer callbacks
It's no longer possible to observe reducers called by other clients by
registering callbacks with `ctx.reducers.on_{my_reducer}`. We no longer
code-generate those methods, or the associated
`ctx.reducers.remove_on_{my_reducer}`. Internal plumbing for storing and
invoking those callbacks is also removed.
### Add specific reducer invocation callbacks
In addition to the previous way to invoke reducers,
`ctx.reducers.{my_reducer}(args...)`, we add a method that registers a
callback to run after the reducer is finished. This method has the
suffix `_then`, as in `ctx.reducers.{my_reducer}_then(args...,
callback)`.
The callback will accept two arguments:
- `ctx: &ReducerEventContext`, the same context as was previously passed
to on-reducer callbacks.
- `status: Result<Result<(), String>, InternalError>`, denoting the
outcome of the reducer.
- `Ok(Ok(())` means the reducer committed. This corresponds to
`ReducerOutcome::Ok` or `ReducerOutcome::Okmpty` in the new WS format.
- `Ok(Err(message))` means the reducer returned an "expected" or "user"
error. This corresponds to `ReducerOutcome::Err` in the new WS format.
- `Err(internal_error)` means something went wrong with host execution.
This corresponds to `ReducerOutcome::InternalError` in the new WS
format.
Internally, the SDK stores the callbacks in its `ReducerCallbacks` map.
This is keyed on `request_id: u32`, a number that is generated for each
reducer call (from an `AtomicU32` that we increment each time), and
included in the `ClientMessage::CallReducer` request. The
`ServerMessage::ReducerResult` includes the same `request_id`, so the
SDK pops out of the `ReducerCallbacks` and invokes the appropriate
callback when processing that message.
These new callbacks are very similar to the existing procedure
callbacks.
### The `Event` exposed to row callbacks
Row callbacks caused by a reducer invoked by this client will see
`Event::Reducer`, the same as they would prior to this PR. These
callbacks will be the result of a `ServerMessage::ReducerResult` with
`ReducerOutcome::Ok`. In order to expose the reducer name and arguments
to this event, the client stores them in its `ReducerCallbacks` map,
alongside the callback for when the reducer is complete.
Row callbacks caused by any other reducer, or any non-reducer
transaction, are now indistinguishable to the client. These will see
`Event::Transaction`, which is renamed from the old
`Event::UnknownTransaction`.
### Less metadata in `ReducerEvent`
Some metadata is removed from `ReducerEvent`, as the V2 WebSocket format
no longer publishes it, even to the caller.
## `CallReducerFlags` are removed
All machinery for setting, storing and applying call reducer flags is
removed from the SDK, as the new WS format does not have any non-default
flags.
## Requesting rows in unsubscribe
When sending a `ClientMessage::Unsubscribe`, we always request that the
server include the matching rows in its response
`ServerMessage::UnsubscribeApplied`. This saves us having to update the
SDK to store query sets separately, at least for now. (We'll do that
later.)
## Handling rows
The new SDK does some additional parsing to wrangle rows in the new
WebSocket format into the same internal data structures as before,
rather than re-writing the client cache. (We'll do that later.)
Specifically, parsing of `DbUpdate` is changed so that:
- We parse raw `TransactionUpdate` into the generated `DbUpdate` type,
which requires an additional loop compared to the previous version, to
cope with the new WS format's dividing updates by query set. We define a
function `transaction_update_iter_table_updates` which encapsulates this
nested loop in an iterator.
- We have two new functions for parsing raw `QueryRows` into the
generated `DbUpdate` type, one for when they come from a
`SubscribeApplied`, and the other when they come from an
`UnsubscribeApplied`. `QueryRows` from `SubscribeApplied` translate to a
`DbUpdate` of all inserts, while one from `UnsubscribeApplied` will be
all deletes.
## Legacy subscriptions
"Legacy subscriptions" are removed. These were only used for
`subscribe_to_all_tables`, which as of now is stubbed. I will follow up
with a change to re-implement `subscribe_to_all_tables` by
code-generating a list of all known tables, and having it subscribe to
`select * from {table}` for every table in that list.
## `subscribe_to_all_tables` via a list
Previously, `subscribe_to_all_tables` worked by sending a legacy
subscription with the query `SELECT * FROM *`, which the host had
special handling to expand to subscribing to all tables. As legacy
subscriptions are no longer usable in V2 clients, this can't work.
Instead, we code-generate `SpacetimeModule::ALL_TABLE_NAMES`, a list of
all the known table names. `subscribe_to_all_tables` then maps across
this list to construct a list of queries in the form `SELECT * FROM
{table_name}`, and subscribes to all of those queries. This has the
upside that defining a new table in the module without regenerating
client bindings will no longer result in the client seeing rows of
tables it does not know about and cannot parse.
## Light mode removed
Light mode is no longer meaningful in the V2 WS format, so all code
related to it is removed.
## Internal changes
### Renamed WS messages
The SDK's internal code is updated to account for various renames:
- `QueryId` -> `QuerySetId`, `query_id` -> `query_set_id`.
- `SubscribeMulti` -> `Subscribe`, `UnsubscribeMulti` -> `Unsubscribe`.
## Incidental changes in this PR, not necessary for other client SDKs
### Don't filter out empty ranges in `RowSizeHint`
The Rust implementation of `RowSizeHint` in `BsatnRowList` got regressed
in the base branch to not work with zero-sized rows. This change fixes
that.
# API and ABI breaking changes
Boy howdy is it!
# Expected complexity level and risk
3? Changes ended up being less complicated than I feared, but we do have
some fiddly code here, and we have internal dependencies on the SDK.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Updated automated test suite.
- Known failures:
- [ ] `subscribe_all_select_star`, which is currently broken because
it's trying to subscribe to rows from private tables. #4241 will fix
this.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
Co-authored-by: = <cloutiertyler@gmail.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade to 2.0.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License file has been updated
- [x] CI passing
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Updated the codegen table/function iteration functions to take in a
parameter to check visibility in all locations for the supported
languages.
- Updated the util.rs functions for iterating tables/functions to check
for a CodegenVisibility enum (IncludePrivate, or OnlyPublic)
- Added a new CodegenOptions struct to pass around the CodegenVisibility
and future flags, defaulted visibility to OnlyPublic
- Updated the CLI to return a list of all private tables not included
(added a TODO to check the --include-private opt):
```bash
Optimising module with wasm-opt...
Build finished successfully.
Skipping private tables during codegen: secret_note, secret_order, secret_person.
Generate finished successfully.
```
# API and ABI breaking changes
Technically API breaking as the private tables will no longer be
available. (GitHub labels are not working at the moment)
# Expected complexity level and risk
1 - Simple change the testing took longer
# Testing
Turns out when you remove private tables you invalidate most of the
module_bindings across the system!
- [x] Rust test SDK for all languages
- [x] C# SDK tests
- [x] C# dotnet tests
- [x] Updated and checked snap files
- [x] Updated Blackholio (Unreal + Unity) module_bindings and tested
- [x] Ran Unreal SDK tests
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
NOTE: Cherry-picking
https://github.com/clockworklabs/SpacetimeDB/pull/4127 from the
`2.0-breaking-changes` branch.
## Original PR Description
This changes generated types in ts client bindings. We currently
generate a few different types of types: reducer args, procedure args,
rows, and user defined types. To avoid potential conflicts between these
types (for example, if a user defined a type called `FooRow`, and also
had a tabled named `foo`, we would end up with two types named
`FooRow`), this puts each set of types in a different file and
namespace. We also stopped exporting the `xxxRow` types, because there
is always another type generated for those. We now have a `types`
directory, which has an `index.ts` with user defined types, along with
`reducers.ts` and `procedures.ts` for the types generated for
reducer/procedure parameters.
```
import type * as Types from './module_bindings/types';
var currentMessages: Types.Message[] = [];
```
or
```
import { type Message } from './module_bindings/types';
var currentMessages: Message[] = [];
```
This has a couple other changes:
- For procedure and reducer types, this adds a suffix of `Args`, since
we may want types for the return values in the future.
- For all of the types, instead of exposing the schema object, we are
now giving the typescript type (e.g. `export type Message =
__Infer<typeof MessageRow>;`). I couldn't think of a reason for users to
want the schema object, so this should save users from needing to do all
of the `Infer` boilerplate.
This is a breaking change for v2.
2. This only changes typescript, and it should generally make thing
easier to use.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Switch Wasm module to `__describe_module_v10__` from
`__describe_module__` to expose `RawModuleDefV10`.
Note: Originally merged into the 2.0 breaking changes branch
https://github.com/clockworklabs/SpacetimeDB/pull/4115.
# API and ABI breaking changes
2.0 breaking change.
# Expected complexity level and risk
1
# Testing
Exising smoketests should be enough.
---------
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Copied from: https://github.com/clockworklabs/SpacetimeDB/pull/4084
We originally reverted this because it was causing testsuite flakes in
private. Now we have solved the issue that was causing the flakes so
this should be safe to merge.
Version upgrade to `v1.12.0`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this is just a version upgrade
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
The testsuite failures are fixed by
https://github.com/clockworklabs/SpacetimeDB/pull/4120
- [x] License has been properly updated including version number and
date
- [x] CI passes
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This reverts the version bump, since it seems to be causing test
flakiness somehow.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This PR implements the C# client-side typed query builder, as assigned
in https://github.com/clockworklabs/SpacetimeDB/issues/3759.
Key pieces:
* Added a small C# runtime query-builder surface in the client SDK
(`sdks/csharp/src/QueryBuilder.cs`):
* `Query` (wraps the generated SQL string)
* `Table<TRow, TCols, TIxCols>` (entry point for `All()` / `Where(...)`)
* `Col<TRow, TValue>` and `IxCol<TRow, TValue>` (typed column
references)
* `BoolExpr` (typed boolean expression composition)
* SQL identifier quoting + literal formatting helpers (`SqlFormat`)
* `Join(...)` with `WhereLeft(...)` / `WhereRight(...)`
* `LeftSemijoin(...)` / `RightSemijoin(...)` with `Where(...)` chaining
* Extended C# client bindings codegen (`crates/codegen/src/csharp.rs`)
to generate:
* Per-table/view `*Cols` and `*IxCols` helper classes used by the typed
query builder.
* A generated per-module `QueryBuilder` with a `From` accessor for each
table/view, producing `Table<...>` values.
* A generated `TypedSubscriptionBuilder` which collects
`Query<TRow>.Sql` values and calls the existing subscription API.
* An `AddQuery(Func<QueryBuilder, Query> build)` entry point off
`SubscriptionBuilder`, mirroring the proposal’s Rust API.
* Fixed a codegen naming collision found during regression testing:
* `*Cols`/`*IxCols` helpers are now named after the table/view accessor
name (PascalCase) instead of the row type, since multiple tables/views
can share the same row type (e.g. alias tables / views returning an
existing product type).
* Kept `Cols`/`IxCols` off the public surface:
* `Table.Cols` and `Table.IxCols` are internal, so consumers only access
columns via the `Where(...)`/join predicate lambdas.
C# usage examples (mirroring the proposal’s Rust examples)
1) Typed subscription flow (no raw SQL)
```csharp
void Subscribe(SpacetimeDB.Types.DbConnection conn)
{
conn.SubscriptionBuilder()
.OnApplied(ctx => { /* ... */ })
.OnError((ctx, err) => { /* ... */ })
.AddQuery(qb => qb.From.Users().Build())
.AddQuery(qb => qb.From.Players().Build())
.Subscribe();
}
```
2) Typed `WHERE` filters and boolean composition
```csharp
conn.SubscriptionBuilder()
.OnApplied(ctx => { /* ... */ })
.OnError((ctx, err) => { /* ... */ })
.AddQuery(qb => qb.From.Players().Where(p => p.Name.Eq("alice").And(p.IsOnline.Eq(true))).Build())
.Subscribe();
```
3) “Admin can see all, otherwise only self” (proposal’s “player” view
logic, but client-side)
```csharp
Identity self = /* ... */;
conn.SubscriptionBuilder()
.AddQuery(qb =>
qb.From.Players().Where(p =>
p.Identity.Eq(self)
)
)
.Subscribe();
```
4) Index-column access for query construction (IxCols)
```csharp
conn.SubscriptionBuilder()
.AddQuery(qb =>
qb.From.Players().Where(
qb.From.Players().IxCols.Identity.Eq(self)
)
)
.Subscribe();
```
# API and ABI breaking changes
None.
* Additive client SDK runtime types.
* Additive client bindings codegen output.
* No wire-format changes.
# Expected complexity level and risk
2 - Low to moderate
* Mostly additive code + codegen.
* The main risk is correctness/compat of generated SQL strings and
name/casing conventions across languages; this is mitigated by targeted
unit tests + full C# regression test runs.
# Testing
- [X] Ran run-regression-tests.sh successfully after regenerating C#
bindings.
- [X] Ran C# unit tests using `dotnet test
sdks/csharp/tests~/tests.csproj -c Release`
- [X] Added a new unit test suite
(`sdks/csharp/tests~/QueryBuilderTests.cs`) validating:
* Identifier quoting / escaping
* Literal formatting (including `Identity`/`ConnectionId`/`Uuid` hex
literals; `U128` integer literal)
* null + enum unsupported behavior throws
* Boolean expression parenthesization (`And`/`Or`/`Not`)
* `Where(...)` overloads including `IxCols`-based predicates
* left/right semijoin SQL formatting and predicate chaining
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Version upgrade to `v1.12.0`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this is just a version upgrade
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
The testsuite failures are fixed by
https://github.com/clockworklabs/SpacetimeDB/pull/4120
- [x] License has been properly updated including version number and
date
- [x] CI passes
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Client Query builder for rust, as per proposal -
https://github.com/clockworklabs/SpacetimeDBPrivate/pull/2356.
1. Pach moves query builder to its separate crate, so that it can be
shared between module and sdk.
2. Implements `TypedSubscriptionBuilder` in `sdks/rust` as mentioned in
proposal
3. Modify codegen to extend types to support query builder as mentioned
in proposal
4. a test
# API and ABI breaking changes
NA, additive changes.
# Expected complexity level and risk
2
# Testing
Added a test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
This moves the query builder code out of the `server` package and into
`lib` so it can be shared by the client and server.
I put the query builder in the `index.ts` of module bindings as a
`query` object that can be imported. The typescript `test-app` has an
example of using it with the subscription builder.
This is branched off of
`https://github.com/clockworklabs/SpacetimeDB/pull/3980`.
# API and ABI breaking changes
This extends the client subscription builder API to allow `string |
RowTypedQuery<any, any>`, so existing client code should be fine.
# Expected complexity level and risk
1.5. This is low risk.
# Testing
I manually tested that the test app still works locally.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version bump to `v1.11.3` for just the CLI and rust
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CLI version has been updated
- [x] Version + date in the license file has been updated
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade to `1.11.2`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
- None, this is just a version bump
# Expected complexity level and risk
1 - no real changes here
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- NA this is just a version bump, no functionality change here.
# Description of Changes
Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).
Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.
# API and ABI breaking changes
We add a new type
# Expected complexity level and risk
2
it impacts all over the code
# Testing
- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bump version numbers to `1.11.1`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- [x] Verified that the license has been updated
- [x] `spacetime --version` on this commit is correct
There is also a corresponding private PR.
# Description of Changes
This helps with the issue reported in
https://github.com/clockworklabs/SpacetimeDB/issues/3811.
Right now we have a type representing the reducer, and a type for the
reducer args, and both have the same name. This adds `Reducer` to the
end of the args type, which is similar to what we are doing for
procedure arguments or the `Row` suffix for tables.
This will still cause some potential problems, since someone could have
a type that ends in `Reducer` (or `Row`), but this will fix the majority
of issues that are currently breaking people.
This also has some changes to get the basic react example to build.
# API and ABI breaking changes
This is technically a breaking change if people are using this for type
annotations (which doesn't seem too likely), but it should be an easy
one for people to fix.
# Expected complexity level and risk
1.
# Testing
I tested the quickstart.
# Description of Changes
Fixes#3807. Not sure what the best way to model this in the API is
(another argument to `t.row()`?), but it does work.
# Expected complexity level and risk
1
# Testing
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
This should fix part of #3503. Adds an override modifier to generated
code and fixes a warning from the angular compiler.
# Expected complexity level and risk
1
# Testing
- [x] Code works fine with the added `override` modifier.
- [ ] Perhaps we should have an angular test project?
# Description of Changes
This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.
The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue
So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.
Fixes#3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?
# Expected complexity level and risk
2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.
# Testing
- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3729
I genuinely don't know what came over me.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5 very straightforward but not strictly trivial
# Testing
Adds automated integration tests (written in Rust and run with `cargo
test`, although note this comment from @matklad about integration tests
for the future
https://internals.rust-lang.org/t/running-test-crates-in-parallel/15639/2):
- [x] Can publish an updated module if no migration is required
- [x] Can publish an updated module if auto-migration is required (with
the yes-break flag true/false)
- [x] Cannot publish if a manual migration is required
- [x] Can publish if a manual migration is required but the user
specified `--delete-data`
- [x] Can publish if a manual migration is required by the user
specified `--delete-data=on-conflict`
- [x] No data deletion occurs if no migration is required and
`--delete-data=on-conflict` is specified
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Bumping versions to 1.11.0 in preparation for an upcoming release.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Existing CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fixes:
- [This
issue](https://discord.com/channels/1037340874172014652/1398209084699709492/1441556875647647816)
by exporting the `SubscriptionHandle` type with the `REMOTE_MODULE` type
applied.
- [This
issue](https://discord.com/channels/1037340874172014652/1398209084699709492/1441559246213746749)
by converting to `camelCase` for column names in code generation.
- Fixes an issue where `onMyReducer` callbacks were passing arguments as
variadic params, while the types indicated they would be passed as an
object. `onMyReducer((ctx, argA, argB, argC) => {})` vs
onMyReducer((ctx, { argA, argB, argC}) => {})`
- [Fixes an
issue](https://github.com/clockworklabs/SpacetimeDB/issues/3503#issuecomment-3566715928)
where the table type name was used instead of the table name in code
generation for constructing tables.
- [Fixes
issue](https://discord.com/channels/1037340874172014652/1398209084699709492/1441886030436499466)
with `ScheduleAt` being used in non-table types.
- [Fixes
issue](https://github.com/clockworklabs/SpacetimeDBPrivate/issues/2168)
where template projects do not use the correct lifecycle reducer setup
# API and ABI breaking changes
Adds a new export, and fixes casing in code genreation.
# Expected complexity level and risk
2
# Testing
- I have tested that the `SubscriptionHandle` is correctly exported
- I have tested that the constraint name is now output in `camelCase`
- I have tested that `onMyReducer` callbacks now return a single
argument
- I have tested that the table name (and view name) is now used instead
of the type name for code generation
- I have tested that the new lifecycle reducers correctly compile
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This upgrades the SpacetimeDB version to 1.10.0.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
This is just a version bump - not tested.
This reverts commit b2e37e8008.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Reverts #3704 which I'm pretty sure contains some sort of bug which is
causing the smoketests to hang.
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI passing again
# Description of Changes
Reworks how `SchedulerActor::handle_queued` works so that it first
determines the parameters of the call to a reducer or the parameters of
the call to the procedure. This also enables the removal of the special
case `call_scheduled_reducer`.
Fixes#3645.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
A test `schedule_procedure` is added.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
# Expected complexity level and risk
2
# Testing
- [x] The types for generated module bindings look right
- [ ] Actually test the client behavior (how?)
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Upgrade to version 1.9.0.
# API and ABI breaking changes
None - just a version upgrade.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that the license has been updated
- [x] The version number looks correct (1.9.0)
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes: #3533
Updated the C# SDK to handle procedures and procedure callbacks in a
similar fashion to the Rust client as well as added the codegen to
support it.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - This adds a new testing frame that should be removed once procedures
are handled with C# module bindings
# Testing
Added /sdks/csharp/examples~/regression-tests/procedure-client to match
modules/sdk-test-procedure which we can roll into the standard
regression-tests once C# supports the procedure attribute.
- [x] Add C# client test of sdk-test-procedure
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Closes#3517 .
With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.
Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.
I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.
This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close#3644 .
Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
rather than running it on the single-threaded database executor.
# API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.
# Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.
# Testing
- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
John Detter