# Description of Changes
This reverts the version bump, since it seems to be causing test
flakiness somehow.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This PR implements the C# client-side typed query builder, as assigned
in https://github.com/clockworklabs/SpacetimeDB/issues/3759.
Key pieces:
* Added a small C# runtime query-builder surface in the client SDK
(`sdks/csharp/src/QueryBuilder.cs`):
* `Query` (wraps the generated SQL string)
* `Table<TRow, TCols, TIxCols>` (entry point for `All()` / `Where(...)`)
* `Col<TRow, TValue>` and `IxCol<TRow, TValue>` (typed column
references)
* `BoolExpr` (typed boolean expression composition)
* SQL identifier quoting + literal formatting helpers (`SqlFormat`)
* `Join(...)` with `WhereLeft(...)` / `WhereRight(...)`
* `LeftSemijoin(...)` / `RightSemijoin(...)` with `Where(...)` chaining
* Extended C# client bindings codegen (`crates/codegen/src/csharp.rs`)
to generate:
* Per-table/view `*Cols` and `*IxCols` helper classes used by the typed
query builder.
* A generated per-module `QueryBuilder` with a `From` accessor for each
table/view, producing `Table<...>` values.
* A generated `TypedSubscriptionBuilder` which collects
`Query<TRow>.Sql` values and calls the existing subscription API.
* An `AddQuery(Func<QueryBuilder, Query> build)` entry point off
`SubscriptionBuilder`, mirroring the proposal’s Rust API.
* Fixed a codegen naming collision found during regression testing:
* `*Cols`/`*IxCols` helpers are now named after the table/view accessor
name (PascalCase) instead of the row type, since multiple tables/views
can share the same row type (e.g. alias tables / views returning an
existing product type).
* Kept `Cols`/`IxCols` off the public surface:
* `Table.Cols` and `Table.IxCols` are internal, so consumers only access
columns via the `Where(...)`/join predicate lambdas.
C# usage examples (mirroring the proposal’s Rust examples)
1) Typed subscription flow (no raw SQL)
```csharp
void Subscribe(SpacetimeDB.Types.DbConnection conn)
{
conn.SubscriptionBuilder()
.OnApplied(ctx => { /* ... */ })
.OnError((ctx, err) => { /* ... */ })
.AddQuery(qb => qb.From.Users().Build())
.AddQuery(qb => qb.From.Players().Build())
.Subscribe();
}
```
2) Typed `WHERE` filters and boolean composition
```csharp
conn.SubscriptionBuilder()
.OnApplied(ctx => { /* ... */ })
.OnError((ctx, err) => { /* ... */ })
.AddQuery(qb => qb.From.Players().Where(p => p.Name.Eq("alice").And(p.IsOnline.Eq(true))).Build())
.Subscribe();
```
3) “Admin can see all, otherwise only self” (proposal’s “player” view
logic, but client-side)
```csharp
Identity self = /* ... */;
conn.SubscriptionBuilder()
.AddQuery(qb =>
qb.From.Players().Where(p =>
p.Identity.Eq(self)
)
)
.Subscribe();
```
4) Index-column access for query construction (IxCols)
```csharp
conn.SubscriptionBuilder()
.AddQuery(qb =>
qb.From.Players().Where(
qb.From.Players().IxCols.Identity.Eq(self)
)
)
.Subscribe();
```
# API and ABI breaking changes
None.
* Additive client SDK runtime types.
* Additive client bindings codegen output.
* No wire-format changes.
# Expected complexity level and risk
2 - Low to moderate
* Mostly additive code + codegen.
* The main risk is correctness/compat of generated SQL strings and
name/casing conventions across languages; this is mitigated by targeted
unit tests + full C# regression test runs.
# Testing
- [X] Ran run-regression-tests.sh successfully after regenerating C#
bindings.
- [X] Ran C# unit tests using `dotnet test
sdks/csharp/tests~/tests.csproj -c Release`
- [X] Added a new unit test suite
(`sdks/csharp/tests~/QueryBuilderTests.cs`) validating:
* Identifier quoting / escaping
* Literal formatting (including `Identity`/`ConnectionId`/`Uuid` hex
literals; `U128` integer literal)
* null + enum unsupported behavior throws
* Boolean expression parenthesization (`And`/`Or`/`Not`)
* `Where(...)` overloads including `IxCols`-based predicates
* left/right semijoin SQL formatting and predicate chaining
# Description of Changes
The first commit defines a type `TableName` that is used in e.g.,
`TxData` and where determined profitable and necessary to do this
change.
`TableName` is backed by
[`ecow::EcoString`](https://docs.rs/ecow/0.2.6/ecow/string/struct.EcoString.html)
which affords O(1) clones and 15 bytes of inline storage and
`mem::size_of::<EcoString>() == 16`.
The second commit does the same for `ReducerName`. This is also used in
reducer execution.
Together, these commits increase TPS by around 5-7k TPS.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Version upgrade to `v1.12.0`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this is just a version upgrade
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
The testsuite failures are fixed by
https://github.com/clockworklabs/SpacetimeDB/pull/4120
- [x] License has been properly updated including version number and
date
- [x] CI passes
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
(jdetter): Github marked Ryan's PR as merged because I rebased it
without updating the base branch first - my bad.
Original PR: https://github.com/clockworklabs/SpacetimeDB/pull/4120
This PR fixes C#-related CI failures when testing against `v1.12.0`
before the corresponding NuGet packages are available on nuget.org.
* CI: Ensure local C# NuGet packages are used consistently
* Pack `crates/bindings-csharp/{BSATN.Runtime,Runtime}` with `-c
Release` so the configured local package sources (`bin/Release`)
actually contain the `.nupkg`s.
* Run `./sdks/csharp/tools~/write-nuget-config.sh` in CI to generate a
`NuGet.Config` that maps `SpacetimeDB.BSATN.Runtime` /
`SpacetimeDB.Runtime` to those local sources (with `nuget.org` as
fallback for non-SpacetimeDB dependencies).
* Add an explicit `dotnet restore --configfile NuGet.Config` step before
tests, and `run dotnet test --no-restore`, so restore always uses the
intended config and does not “accidentally” restore from `nuget.org`.
* `write-nuget-config.sh`: Make NuGet config discovery + mapping
deterministic
* Write `NuGet.Config` (capitalized) and include `<clear />` + an
explicit `nuget.org` source to avoid inherited sources and to keep
`PackageSourceMapping` functional.
* Also write a repo-root `NuGet.Config` so `dotnet publish/pack` invoked
from templates (outside `sdks/csharp/`) still discovers the same
override behavior.
* Smoketests quickstart: avoid `PackageSourceMapping` restore failures
* Ensure smoketest helper packing uses `Release`, and when repo-root
`NuGet.Config` exists, perform `dotnet restore --configfile ...` +
`dotnet pack --no-restore` so pack/restore are evaluated with the same
sources/mapping.
* When editing configs, prefer `NuGet.Config` casing for Linux discovery
and ensure `nuget.org` exists as a source for the fallback mapping.
# API and ABI breaking changes
No changes
# Expected complexity level and risk
1
# Testing
- [X] Locally tested concept after simulating local repro.
- [x] Confirmed CI passes in this branch
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This PR renames the templates to always use shorthand for the language,
specify a framework (or console) if necessary, and shorten the naming in
general
# Expected complexity level and risk
1
# Testing
I've tested generating templates manually
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
* Added a new `cargo ci dlls` subcommand to build/pack the in-repo C#
NuGet packages and the C# SDK.
* `cargo ci dlls` restores `sdks/csharp/SpacetimeDB.ClientSDK.csproj`
using the freshly built local package outputs as to populate
`sdks/csharp/packages/**`.
* Added a Unity `.meta` skeleton under
`sdks/csharp/unity-meta-skeleton~/**` and overlays those `.meta` files
onto the latest restored versioned package directory to keep Unity GUIDs
stable and import settings consistent.
* Unity-specific import fixes are captured in the skeleton overlay
(notably: preventing Unity from importing incompatible TFMs like
`net8.0`, and marking analyzer DLLs with the `RoslynAnalyzer` label so
Unity can recognize them).
# How to use (local)
```bash
# Build/pack + restore local packages into sdks/csharp/packages/**
cargo ci dlls
```
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - Local developer tooling + file overlay into restore output; no
runtime/SDK API behavior changes.
# Testing
- [x] `cargo check -p ci`
- [x] Ran `cargo ci dlls` and verified the output under
`sdks/csharp/packages/**` and the various NuGet package locations.
- [x] Tested a Unity project importing the SpacetimeDB SDK after
generating output and confirmed no errors.
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Client Query builder for rust, as per proposal -
https://github.com/clockworklabs/SpacetimeDBPrivate/pull/2356.
1. Pach moves query builder to its separate crate, so that it can be
shared between module and sdk.
2. Implements `TypedSubscriptionBuilder` in `sdks/rust` as mentioned in
proposal
3. Modify codegen to extend types to support query builder as mentioned
in proposal
4. a test
# API and ABI breaking changes
NA, additive changes.
# Expected complexity level and risk
2
# Testing
Added a test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
This PR adds a regression test covering nullable reference-type view
returns in C# modules (e.g. Account? where Account is a class), as
reported in
[#3962](https://github.com/clockworklabs/SpacetimeDB/issues/3962).
```csharp
public static Account? MyAccount(ViewContext ctx)
```
* Updated the C# regression-test server module to include a
reference-type table and views that exercise the RefOption path:
* A new public reference-type table row:
```csharp
[SpacetimeDB.Table(Name = "account", Public = true)]
public partial class Account { ... }
```
* A public at-most-one view that returns a nullable reference type
(Account?) via Find(...):
```csharp
[SpacetimeDB.View(Name = "my_account", Public = true)]
public static Account? MyAccount(ViewContext ctx)
{
return ctx.Db.account.Identity.Find(ctx.Sender) as Account;
}
```
* A second public view that returns null to ensure the “empty result”
case is exercised:
```csharp
[SpacetimeDB.View(Name = "my_account_missing", Public = true)]
public static Account? MyAccountMissing(ViewContext ctx) => null;
```
* Updated ClientConnected to ensure an Account row exists for the
connecting identity so the “one row” case is deterministic.
* Updated the C# regression-test client to:
* Subscribe to the new views:
* `SELECT * FROM my_account`
* `SELECT * FROM my_account_missing`
* Assert correct semantics for nullable reference-type view returns:
* MyAccount.Count == 1
* MyAccountMissing.Count == 0
* Updated the regression-test server project to use local C#
runtime/codegen project references so the regression module exercises
the in-repo generator/runtime behavior (instead of the published
SpacetimeDB.Runtime package).
# API and ABI breaking changes
None.
* No changes to public module schema/wire format semantics beyond adding
regression-test-only tables/views.
* No behavior changes outside the C# regression test module + harness.
# Expected complexity level and risk
2 - Low
* Changes are isolated to regression tests and project wiring.
* The scenario specifically guards the nullable reference-type
“Option-like view return” path against regressions.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [X] Ran C# regression tests with no failures in new View tests
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
- Updated Unreal codegen to work with ::Result to match how ::Option
works to create a new struct for each type pair identified
- Added new integration tests to confirm Ok state for all new types
added to sdk-tests
- Added new UE_SPACETIMEDB_RESULT macro to capture de/serialization for
the new structs
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - Added new type generated from Result which creates new structs for
each one identified
# Testing
- [x] Updated the integration tests to include the new Result types
---------
Co-authored-by: Mario Alejandro Montoya Cortes <mamcx@elmalabarista.com>
# Description of Changes
Updated the handler from OnRawMessage to OnBinaryMessage as OnRawMessage
seems to have a bug that won't report the BytesRemaining for incomplete
messages. OnBinaryMessage properly returns a bool flag like how the C#
implementation works.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - Underlying framework change on how messages are handled
# Testing
Blackholio was no longer working and is back in working order; however
Unreal test framework has been broken due to the work on Result<> as the
companion PR has not been merged.
- [x] Rebuilt and ran Unreal Blackholio
# Description of Changes
This PR fixes a C# SDK/BSATN serialization issue where inserting a
`null` string into a non-nullable `string` column could throw an
`ArgumentNullException` with an unhelpful stack trace, as reported in
[#3960](https://github.com/clockworklabs/SpacetimeDB/issues/3960).
* Updated the C# BSATN runtime to reject `null` for non-nullable
`string` serialization with a clear, actionable error message (guides
users to model nullable strings as `string?` so BSATN encodes an
option).
* Added BSATN runtime unit tests to cover:
* Non-nullable string rejects `null` with an actionable message
* Nullable string (`RefOption<string, BSATN.String>`) round-trips `null`
* Added end-to-end C# regression-test module + client assertions for
#3960:
* Inserting `""` into non-nullable `string` succeeds
* Inserting `null` into non-nullable `string` fails with the expected
message
* Inserting `null` into nullable `string?` succeeds and round-trips as
`null`
# API and ABI breaking changes
None.
* No schema or wire-format changes.
* Behavior change is limited to improving the error surfaced when
attempting to serialize null as a non-nullable BSATN `String`, plus
stricter input validation for `ConnectionId` / `Identity` parsing.
# Expected complexity level and risk
2 - Low
* Changes are isolated to regression test coverage.
* No impact on valid serialization paths; only affects invalid/malformed
inputs and improves diagnostics.
# Testing
- [X] Ran C# regression-tests client and verified new tests pass.
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This PR fixes a C# reducer error-reporting issue where exceptions thrown
in reducers were returned to clients with full stack traces, as reported
in [#3959](https://github.com/clockworklabs/SpacetimeDB/issues/3959).
- Updated the C# reducer host-call error serialization to return only
`Exception.Message` (instead of `Exception.ToString()`), preventing
server-side stack traces from being leaked to clients.
- Added a C# regression-test client assertion to validate reducer
failures do not contain stack traces:
- `exception.Message` matches the thrown message exactly
- the message does not contain newlines or `" at "` stack-trace markers
# API and ABI breaking changes
None.
- No schema or wire-format changes.
- Behavior change is limited to the *contents* of reducer error strings
returned to clients (stack traces removed).
# Expected complexity level and risk
2 - Low
- Change is isolated to C# reducer error formatting plus a regression
assertion.
- Removes unintended information exposure without affecting reducer
success paths.
# Testing
- [X] Ran C# regression-tests client; verified reducer error message is
preserved and stack trace is not present.
# Description of Changes
We would like to move all of the templates to a central directory
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
---------
Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3240.
Non-unique indices are now backed by a type `SameKeyEntry` which holds
the `RowPointer`s for the same key.
When these `RowPointer`s exceed 4KiB (512 entries), the data structure
switches from using an array list to a hash set.
# API and ABI breaking changes
None
# Expected complexity level and risk
2?
# Testing
Covered by existing tests, though more test will come in future PRs.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade to `1.11.2`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
- None, this is just a version bump
# Expected complexity level and risk
1 - no real changes here
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- NA this is just a version bump, no functionality change here.
# Description of Changes
Closes#3673
*NOTE*: C++ part will be in another PR
# Expected complexity level and risk
2
Adding a new type touch everywhere
# Testing
- [x] Adding smoke and unit test
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Rust added procedure-scoped HTTP via the `procedure_http_request` ABI
(see Rust PR #3684) to C# host bindings.
What changed:
1) Fix for BSATN wire format
* Updated C# BSATN wire types for HTTP
(`BSATN.Runtime/HttpWireTypes.cs`) to match exactly the Rust stable
types:
* `spacetimedb_lib::http::Request` fields and order: `method`,
`headers`, `timeout`, `uri`, `version`
* `spacetimedb_lib::http::Response` fields and order: `headers`,
`version`, `code`
* Header pairs are (`name: string, value: bytes`); no additional
metadata is encoded.
* Added explicit “do not reorder/extend” note in the C# wire type file
since the host BSATN-decodes these bytes directly and may trap on
mismatch.
2) C# runtime HTTP client implementation
* Implemented `ProcedureContext.Http` support (`Runtime/Http.cs`) that:
* BSATN-serializes `HttpRequestWire` + sends body bytes via
`procedure_http_request`
* On success, BSATN-decodes `HttpResponseWire` and returns `(response,
body)` as a typed `HttpResponse`
* On `HTTP_ERROR`, decodes the error payload as a BSATN `string`
* On `WOULD_BLOCK_TRANSACTION`, returns a stable error message (host
rejects blocking HTTP while a mut tx is open)
* Clamps timeouts to 500ms max (matching host behavior documented on the
Rust side)
3) ABI wiring / import plumbing
* Added the `procedure_http_request` import to the C# wasm shim
(`Runtime/bindings.c`) under the `spacetime_10.3` import module.
* Verified the generated C# P/Invoke signature matches the host ABI
(request ptr/len, body ptr/len, out `[BytesSource; 2]`).
4) Procedure entrypoint contract (trap vs errno)
* Updated/clarified the module procedure trampoline behavior
(`Runtime/Internal/Module.call_procedure`): it must either return
`Errno.OK` or trap (log + rethrow). This mirrors the host’s expectations
and avoids “unexpected errno values from guest entrypoints” behavior.
Behavioral notes vs Rust
* Rust bindings may `expect(...)`/panic on “should never happen”
serialization failures; C# runtime explicitly avoids throwing across the
procedure boundary for the HTTP client path and instead returns
`Result.Err` for unexpected failures. This prevents whole-module traps
from user-space HTTP usage while still surfacing rich errors.
* The host remains authoritative on timeout enforcement; C# now mirrors
the effective host clamp (500ms) to keep behavior aligned.
# API and ABI breaking changes
No new host ABI introduced, and does not change the syscall signature.
* This is not an ABI “break” in the host, but it is a guest-side wire
compatibility correction: older C# guests built with the incorrect wire
types could cause the host to trap during BSATN decode. After this PR,
C# guests are compatible with the host’s expected layout.
**Adds API**: `ProcedureContextBase.Http` is available for procedures
No existing public types/method signatures are removed
* A notable difference from Rust's panic behavior is that Rust code
sometimes `expect(...)`s and may panic/trap on internal “should not
happen” cases. On the other hand, C#'s `HttpClient.Send` explicitly
converts unexpected failures into `Result.Err` to avoid trapping the
module. This is a behavioral difference, but not an API break.
# Expected complexity level and risk
2 - Mostly just creating equivalents to the Rust version.
# Testing
C# regression tests updated to cover:
- [X] Successful request path (`ReadMySchemaViaHttp`)
- [X] Invalid request path (`InvalidHttpRequest`) returning error
without trapping
Testing performed:
- [X] Full regression suite run against local node (see chat log); no
wasm traps, all suites reported `Success`.
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
This PR fixes a C# codegen performance/behavior issue triggered by views
that include nullable value-type fields (e.g. `DbVector2?`), as reported
in #3914.
* Updated the C# BSATN code generator to emit non-boxing equality for
`Nullable<T>` members by using `System.Nullable.Equals(a, b)` rather
than `a.Equals(b)` (which can box and cause excessive host logging/work
in view diff/equality paths).
This causes code generation to change from something like:
``` csharp
// From:
var ___eqNullableIntField = this.NullableIntField.Equals(that.NullableIntField);
// To:
var ___eqNullableIntField = System.Nullable.Equals(
this.NullableIntField,
that.NullableIntField
);
```
* Added a regression scenario to the C# regression-test module that
exercises the problematic pattern:
* A table containing a nullable struct field (`DbVector2? Pos`).
* A public view that returns rows containing that nullable field.
* A reducer to mutate the nullable field from `some` → `none` → `some`
to force view re-evaluation and diffing.
* Updated the regression-test client to:
* Subscribe to the new view.
* Validate that the view evaluates successfully and contains the
expected rows.
* Call the reducer and validate view state after updates.
* Fail the test if view evaluation/diffing produces errors.
# API and ABI breaking changes
None.
* No changes to public SpacetimeDB schema or wire format semantics.
* Changes are limited to generated equality code and regression tests.
# Expected complexity level and risk
2 - Low
* The codegen change is small and localized (special-casing
`System.Nullable<T>` equality generation).
* Risk is primarily around subtle behavior differences in equality for
nullable value types; however, `System.Nullable.Equals` matches the
expected semantics and avoids boxing.
* Regression tests specifically cover the nullable-struct-in-view
scenario to guard against regressions.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [X] Ran the C# regression test suite via `run-regression-tests.sh` and
confirmed no errors.
# Description of Changes
Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).
Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.
# API and ABI breaking changes
We add a new type
# Expected complexity level and risk
2
it impacts all over the code
# Testing
- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Removes the `TimestampCapabilities` test from the C# SDK's regression
test suite, due to creating inconsistent failures in automated CI
testing.
# API and ABI breaking changes
Not API/ABI breaking
# Expected complexity level and risk
1
# Testing
- [X] Ran locally, no failures in building or running C# regression
tests
# Description of Changes
Avoid warning if table is valid.
# API and ABI breaking changes
-
# Expected complexity level and risk
1
# Testing
Accessing tables via Blueprint and no warning in output log, if table is
valid.
# Description of Changes
Implements the C# equivalent of #3638
This implement uses inheritance, where abstract base classes (like
`ProcedureContextBase` in `ProcedureContext.cs`) store the core of the
implementation, and then generated wrappers (like `ProcedureContext` in
the generated FFI.cs file) inherit from them.
For error handling, we work like Rust's implementation of `Result<T,E>`
but we require `where E : Exception` because of how exceptions work in
C#. Transaction-level failures come back as a `TxOutcome` and user
errors should follow the `Result<T,E>` pattern. In this implementation,
we have `UnwrapOrThrow()` throws exceptions directly because of C#'s
error handling pattern.
Unlike the Rust implementation's direct `Result` propagation, we are
using an `AbortGuard` pattern (in `ProcedureContext.cs`) for exception
handling, which uses `IDisposable` for automatic cleanup.
Most changes should have fairly similar Rust-equivalents beyond that.
For module authors, the changes here allow for the transation logic to
work like:
```csharp
ctx.TryWithTx<ResultType, Exception>(tx => {
// transaction logic
return Result<ResultType, Exception>.Ok(result);
});
```
This change includes a number of tests added to the
`sdks/csharp/examples~/regression-tests/`'s `server` and `client` to
validate the behavior of the changes. `server` changes provide further
usage examples for module authors.
# API and ABI breaking changes
Should not be a breaking change
# Expected complexity level and risk
2
# Testing
- [x] Created Regression Tests that show transitions in procedures
working in various ways, all of which pass.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bump version numbers to `1.11.1`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- [x] Verified that the license has been updated
- [x] `spacetime --version` on this commit is correct
There is also a corresponding private PR.
# Description of Changes
This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.
The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue
So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.
Fixes#3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?
# Expected complexity level and risk
2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.
# Testing
- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
# Description of Changes
Closes: #3781
- Added and tested procedure docs for Unreal C++ & Unreal Blueprint
- Fixed a small issue with the blueprint accessors for the event status
- Fixed a bug in the C# procedure docs
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
I built a new local test framework to do another clean end to end test
of procedures with Unreal to properly test the Blueprint work as well.
- [x] Built the Rust server alongside a new Unreal client and tested
each procedure/callback in C++ and Blueprint
- [x] Reviewed the docs locally
# Description of Changes
Bumping versions to 1.11.0 in preparation for an upcoming release.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Existing CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Moves a reducer call to inside an `on_insert` callback to avoid race
condition.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Testing
Fixes a test
# Description of Changes
Applies the same implicit filter for views to delta tables that we
already did for physical tables.
Removes a single condition `scan.delta.is_none()` from the already
existing rewrite rule.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Testing
- [x] Rust sdk test
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This upgrades the SpacetimeDB version to 1.10.0.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
This is just a version bump - not tested.
This reverts commit b2e37e8008.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Reverts #3704 which I'm pretty sure contains some sort of bug which is
causing the smoketests to hang.
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI passing again
# Description of Changes
Reworks how `SchedulerActor::handle_queued` works so that it first
determines the parameters of the call to a reducer or the parameters of
the call to the procedure. This also enables the removal of the special
case `call_scheduled_reducer`.
Fixes#3645.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
A test `schedule_procedure` is added.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
Rust SDK test suite for views
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
This patch only adds tests, it does not change functionality.
# Description of Changes
Implements the C# module bindings for Procedures (#3510)
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
- [X] Locally tested against existing C# Procedures regression test
client (minus the Transaction and HTTP portions, as those are not in
this change).
# Description of Changes
There were mentions of `hashbrown` in the repo that did not go through
`spacetimedb_data_structures::map`.
This caused compile errors on master when running certain tests locally.
These have been replaced with the proper imports.
The PR also bump hashbrown to 0.16.1 and foldhash to 0.2.0.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
Covered by existing tests.
# Description of Changes
Closes: #3535
Updated the Unreal SDK to handle procedures and procedure callbacks as
closely matched to Rust + C#.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - This adds a new testing frame that should be removed once procedures
are handled in /modules/sdk-test.
# Testing
Added a mirror of /sdks/unreal/tests/TestClient to use the new
/modules/sdk-test-procedure which adds complexity we'll want to remove.
- [x] Add Unreal client test of sdk-test-procedure
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Upgrade to version 1.9.0.
# API and ABI breaking changes
None - just a version upgrade.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that the license has been updated
- [x] The version number looks correct (1.9.0)
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes: #3533
Updated the C# SDK to handle procedures and procedure callbacks in a
similar fashion to the Rust client as well as added the codegen to
support it.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - This adds a new testing frame that should be removed once procedures
are handled with C# module bindings
# Testing
Added /sdks/csharp/examples~/regression-tests/procedure-client to match
modules/sdk-test-procedure which we can roll into the standard
regression-tests once C# supports the procedure attribute.
- [x] Add C# client test of sdk-test-procedure
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Closes#3517 .
With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.
Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.
I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.
This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close#3644 .
Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
rather than running it on the single-threaded database executor.
# API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.
# Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.
# Testing
- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
1. Updates the Replication Tests in
`sdks/csharp/examples~/regression-tests` to include better coverage of
Views
2. Added missing linkage for __call_view__ and __call_view_anon__
3. Updated *ViewDispatcher Invoke to transform BSATN.ValueOption<> into
BSATN.List<>
4. Fixed issues with the indexing of views to match correctly during
__call_view__ and __call_view_anon__
# API and ABI breaking changes
No
# Expected complexity level and risk
2
# Testing
- [x] Running `run-regression-tests.sh` passes.
---------
Signed-off-by: rekhoff <r.ekhoff@clockworklabs.io>
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Closes: #3658
With the work on Views we ran into trouble with the Return Type and
certain C# code layout where the types are defined inside the Module
class. This fixes the bug in our current approach but needs
consideration to possibly be changed entirely.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Minor change
# Testing
- [x] Retested with the linked code with and without the namespace
- [x] Re-ran regression tests
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
Adds `ProcedureContext::{with_tx, try_with_tx}`.
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3515.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
An integration test `test_calling_with_tx` is added.
# Description of Changes
Fixes client codegen for views.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
I'm not sure what tests we have for C++.
- [x] Updated client snapshots for rust, C#, and typescript
- [ ] Rust sdk test
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
This PR is a very large change to the workings of the TypeScript SDK and
as such requires a higher bar of testing than other PRs. However, it
does several important things:
1. Unifies the API of the server and client so they not only have the
same API, but they actually implement it with the same TypeScript types.
This fixes several inconsistencies between them and fixes several small
bugs as well.
2. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3365
3. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3431
4. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3435
5. Subsumes the work done in
https://github.com/clockworklabs/SpacetimeDB/pull/3447
6. Derives all type information on the client from a single
`RemoteModule` type which vastly cleans up the correctness of type
checking on the client and helped me to find several small bugs
It accomplishes this by changing code generation of TypeScript on the
client to code generation approximately what a developer would manually
write in their module. The ultimate goal would be to allow the developer
to use the types and functions that they define on in their module
directly on the client without needing to do any code generation at all,
provided they are using TypeScript on the server and client.
https://github.com/clockworklabs/SpacetimeDB/issues/3365 is resolved by
`.build()`ing the `DbConnection` inside a React `useEffect` rather than
doing it directly in line with the render of the provider. In order to
do that we needed to not expose the `DbConnection` directly to
developers by returning a different type from `useSpacetimeDB`.
`useSpacetimeDB` now returns a `ConnectionState` object which is stored
as React state and updates when any of the fields change. This change
also resolves https://github.com/clockworklabs/SpacetimeDB/issues/3431.
https://github.com/clockworklabs/SpacetimeDB/issues/3435 was the issue
that initially lead me down the rabbit hole of unifying the server and
the client because it was nearly impossible to track down all the
various type functions and how they connect to the values that we code
generate on the server. After several hours of attempting this, I
decided to clean up the types a bit to be more uniform.
Implementing the unification between the client and the server also
necessitated fully implemented parts of the API that were fully
implemented on the server, but were broken or missing on the client.
# API and ABI breaking changes
[Unification] -> Means that this is breaking behavior for the client
SDK, but that the new behavior is identical to the server's existing
behavior
## Breaking changes:
- Table accessor names and index accessor names are converted to
camelCase on the `ctx`, so `ctx.db.foo_bar` is now `ctx.db.fooBar`
- [Unification] On the client `my_table.iter()` returns
`IterableIterator` instead of an `Array`
- [Unification] `module_bindings` now export `TypeBuilder`s for all
types instead of a `type MyType` and object `MyType`, so instead of
using `MyType` as a type directly, you need to infer the type `MyType`
-> `Infer<typeof MyType>`.
- [Unification] We no longer generate and export `MyTypeVariants` for
sum types (these are now accessed by `Infer<typeof
MyType.variants.myVariant>`)
- [Unification] `MyType.getTypeScriptAlgebraicType()` has been replaced
with `MyType.algebraicType`
- `useSpacetimeDB()` no longer takes type parameters
- `useTable()` now takes a `TableDef` parameter and type params are
inferred
- `useTable()` now just returns an `Array` directly instead of a object
with `{ rows }`
- [Unification] `ctx.reducers.createPlayer(argA, argB)` ->
`ctx.reducers.createPlayer({ argA, argB })`
- [Unification] `ctx.reducers.onCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.onCreatePlayer(ctx, { argA, argB })`
- [Unification] `ctx.reducers.removeOnCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.removeOnCreatePlayer(ctx, { argA, argB })`
- [Unification] `myTable.count(): number` -> `myTable.count(): bigint`
## Additive changes:
- `Infer<>` now also does `InferTypeOfRow<>` if applicable
- Added a `useReducer()` React hook
- `module_bindings` now exports a `tables` object with references to all
the `TableDef`s
- `module_bindings` now exports a `reducers` object with references to
all the `ReducerDef`s
- Added a new `MyType.create('MyVariant', ...)` function in addition to
the `MyType.MyVariant(...)` constructors (this is private)
## Notable things that did not change:
- `MyType.serialize(writer: BinaryWriter, value: Infer<typeof MyType>)`
and `MyType.deserialize(reader: BinaryReader): Infer<typeof MyType>` are
still supported exactly as before.
- The `MyType.MyVariant(...)` constructor function on sum types is still
present, but implemented with the private `MyType.create('MyVariant',
...)`. We could choose to move away from this API later if we didn't
like the variants polluting the namespace
# Expected complexity level and risk
4 - This is a deep reaching an complex change for the SDK. For the
server, it is much less deep reaching since it reuses much of the same
machinery, although it does require thorough testing there as some of
the code was modified.
This change is fully localized to TypeScript and does not touch the host
(or other languages) at all, and therefore only impacts a beta aspect of
SpacetimeDB.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Added regression test for
https://github.com/clockworklabs/SpacetimeDB/issues/3435
- [x] Manually tested `test-app` and `test-react-router-app`
- [ ] Add test cases for camelCase-ing
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Noa <coolreader18@gmail.com>
Allows the SpacetimeDBClient to be configured with or without confirmed
reads. Like for [TypeScript], the parameter is optional -- the server
chooses the default if not set explicitly.
[TypeScript]: https://github.com/clockworklabs/SpacetimeDB/pull/3247
# Expected complexity level and risk
1
# Testing
I don't actually know what I'm doing 😅
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- This upgrades the versions of all SDKs, the CLI, etc. to 1.8.0
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that all versions seem to be updated including the BSL
license update <!-- maybe a test you want to do -->
We have 1 `1.7.0` that didn't get upgraded automatically because it is
part of the module bindings for a template:
```
crates/cli/.templates/parent_parent_crates_bindings-typescript_examples_quickstart-chat/src/module_bindings/index.ts: cliVersion: '1.7.0',
```
A case could possibly be made for bumping the template but it shouldn't
cause any issues as the module bindings directory should just get
regenerated by the user. @cloutiertyler should we be bumping module
bindings for templates when we upgrade versions?
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Zeke Foppa