# Description of Changes
- Fixed an Unreal SDK cache bug with overlapping subscriptions on the
same table called out in
[Discord](<https://discordapp.com/channels/1037340874172014652/1507200761900171405>)
from @defohost
- Merged repeated transaction table updates by table name while
preserving all row-set payloads
- Fixes unique-index `Find(...)` returning empty while `Iter()` still
sees the updated row
- Aligned Unreal transaction update handling with the other SDKs
- Applies one accumulated per-table update per transaction instead of
multiple partial passes
# API and ABI breaking changes
- No intended API or ABI breaking changes to released Unreal SDK
behavior
# Expected complexity level and risk
2 - Small transaction update normalization change, risk is preserving
row multiplicity for overlapping
subscriptions
# Testing
What I've done so far:
- [x] Added and ran a throwaway Unreal repro for overlapping identical
subscriptions on one table
- [x] Ran the full `sdk-unreal-test-harness`
- [x] Tested Unreal Blackholio
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bumps version to 2.7.0
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
- 1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Version number is correct (`2.7.0`)
- [x] BSL license file has been updated with the new date and version
number
# Description of Changes
- Added a client-side query builder for the Unreal SDK
- Added typed query-builder subscriptions for Unreal C++ via
SubscriptionBuilder.AddQuery(...).Subscribe()
- Added generated Blueprint query-builder support with source query
nodes, column nodes, predicates, Where, AddQuery, and Subscribe
- Added Blueprint autocasts at the AddQuery boundary so source-specific
Blueprint queries can connect cleanly into the generic subscription
builder for better devex in Blueprint
- Synced the Unreal query-builder core with the C++ module
implementation
- Updated the copied Unreal core query-builder headers to match the
shared C++
- Moved Unreal-specific literal/type adapters out of the shared core
copy and into Unreal-specific expansion code
- Added Unreal SDK test coverage and documentation for the new
query-builder surface
- Added new test harnesses to start to match the View/ViewPk tests
- Added documentation for the Unreal query builder
- Refactor generation for modules with more than 255 reducers, required
to get the TestClient back in working order
# API and ABI breaking changes
- No intended API or ABI breaking changes to released Unreal SDK
behavior
- Adds a new public client-side query-builder API to the Unreal SDK for
C++ and Blueprint
- Raw SQL subscriptions remain available
# Expected complexity level and risk
3 - Adds new query-builder surface with large changes to the code-gen,
the risk here is keeping the C++ module core mirrored to the SDK in the
future
# Testing
What I've done so far:
- [x] Updated and ran the following tests:
- [x] TestClientEditor
- [x] TestViewClientEditor
- [x] TestViewPkClientEditor
- [x] Ran the Unreal harness suites covering the query-builder client
paths
- [x] Verified the generated Unreal query-builder surface in both C++
and Blueprint testing supported fields with a small sample project
Should be done at least once by a reviewer:
- [ ] Re-run the full `sdk-unreal-test-harness` as it's disabled in CI
- [ ] Play with the C++ and Blueprint versions in a small Unreal project
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Adds primary keys to procedural views in C++. This mirrors the work from
#5111, #5246, and #5327 adding the feature and the docs changes.
# API and ABI breaking changes
None
# Expected complexity level and risk
3
# Testing
- [x] Equivalent tests as were added in #5111 and #5246 for rust,
typescript, and C#
# Description of Changes
Use an isolated server process per SDK test instead of a single process
for all of the tests. In addition to reducing the memory footprint of
each test run, this should also allow for more parallelism among the
individual tests.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5
# Testing
The SDK test suite should continue to work
# Description of Changes
* Bump version to 2.6.0
# API and ABI breaking changes
None
# Expected complexity level and risk
* 1 - this is just a version bump
# Testing
- [X] Version number is correct (`2.6.0`)
- [X] BSL license file has been updated with the new date and version
number
# Description of Changes
Bump version to `2.5.0` for new release
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
None
# Description of Changes
Adds primary keys to procedural views in C#. See for #5111 for the
equivalent feature in rust and C# as well as a more detailed
description.
# API and ABI breaking changes
None
# Expected complexity level and risk
3
# Testing
- [x] Equivalent tests as were added in #5111 for rust and typescript
# Description of Changes
Add several new tests of concurrency behavior re: procedures.
The new tests are in the SDK tests, 'cause I thought the easiest way to
observe this behavior was by connecting a client to a database and
calling some functions in it. This is yet another mild misuse of the SDK
test suite, as the behavior in question is not in the SDK, it's in the
host. The tests also have a new module/client pair added, as we don't
(yet?) expose `procedure_sleep_until` to any module languages other than
Rust, so we can't implement the same test in any other languages.
### `procedure_reducer_interleaving`
Verifies that a procedure and a reducer can run concurrently, with the
procedure cooperatively yielding using `ctx.sleep_until`. Uses two
separate connections due to #4954 .
### `procedure_reducer_same_client_interleaved`
Same as previous, but with only a single connection. Now that #4954 is
closed, this has the same semantics as previous.
### `procedure_concurrent_with_scheduled_reducer`
Verifies that a non-scheduled procedure can schedule a reducer and then
sleep, and the reducer will execute before the procedure wakes back up.
### `scheduled_procedure_scheduled_reducer_not_interleaved`
Schedules a procedure and a reducer, which you might expect to execute
concurrently, but don't in a way similar to
`procedure_reducer_same_client_not_interleaved`.
This is the behavior that kicked off this whole thing. The scheduler
subsystem behaves like a single client, in the sense that it waits for a
single function to terminate before scheduling the next function.
# API and ABI breaking changes
N/a
# Expected complexity level and risk
1 - tests
# Testing
- [x] Ran the tests!
I didn't bother intentionally breaking the host to verify that the tests
would fail. It feels pretty apparent to me based on just the test code
that we won't see false negatives.
# Description of Changes
The previous PR only bumped the Rust packages since that's what we were
releasing. Since we may release more, we can now bump the rest of the
versions.
(This is harmless if we don't end up releasing the other packages
anyway; I could have just done this in the first place).
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
CI only
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Adds support for primary keys to procedural views in rust and typescript
modules. Now clients can receive update events when subscribed to such
views.
```rust
#[spacetimedb::view(accessor = my_players, public, primary_key = id)]
pub fn my_players(ctx: &spacetimedb::ViewContext) -> Vec<Player> {
ctx.db.players().owner().filter(ctx.sender()).collect()
}
```
```ts
const Player = t.row('Player', {
id: t.u64().primaryKey(),
owner: t.identity().index('btree'),
name: t.string(),
});
export const my_players = spacetimedb.view(
{ public: true },
t.array(players.rowType),
ctx => Array.from(ctx.db.players.owner.filter(ctx.sender))
);
```
In rust, a view's primary key is declared within the `#[view]` macro,
whereas in typescript it is declared at the column/row level as it is
for tables. I could have made it a view level attribute for typescript
as well, but since primary keys are already row-level attributes in
typescript, I just kept that as is.
Note, care must be taken to ensure that the view never returns duplicate
primary keys, or else it will fail which will currently result in the
transaction that triggered the view refresh to be rolled back. Better
error handling for this exact scenario will be added in a separate
follow up patch.
## Alternative Considered: #[primary_key] attribute on `SpacetimeType`s
in Rust
This would be equivalent to what we do in typescript.
# API and ABI breaking changes
None, although adding a primary key to an existing view will require a
client update.
# Expected complexity level and risk
3
# Testing
Compile-time failure scenarios:
- [x] Primary key on non-existent column
- [x] Primary key doesn't reference custom accessor name (rust)
- [x] Multiple primary key columns specified (typescript)
- [x] Primary keys must be `FilterableValue`s
SDK tests (rust sdk, rust and ts modules):
- [x] `OnUpdate`
- [x] `OnUpdate` is sender-scoped
- [X] Can join on primary key columns
Smoketests:
- [x] Modifying primary key columns breaks clients (auto-migrations)
# Description of Changes
* Bumps version to 2.4.0
# API and ABI breaking changes
None
# Expected complexity level and risk
* 1 - this is just a version bump
# Testing
- [X] Version number is correct (`2.4.0`)
- [X] BSL license file has been updated with the new date and version
number
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
## Summary
- repoint selected BSL crate `LICENSE` symlinks to
`licenses/apache2.txt`
- add the missing Unreal SDK `LICENSE` symlink to `licenses/apache2.txt`
Requested crates/SDK:
- bindings-macro
- bindings-sys
- bindings
- lib
- primitives
- query-builder
- sats
- unreal SDK
Note: `query-builder` was already pointing at `licenses/apache2.txt`, so
it has no diff.
## Testing
- Verified all requested `LICENSE` paths resolve to
`../../licenses/apache2.txt`
- `cargo run -p check-license-symlinks` does not complete cleanly in
this checkout because it also scans unrelated existing copied template
files and `node_modules` license files that are not symlinks
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
- Completes the Blackholio demo with username selection, leader board,
split mechanic, improved visuals...
- Add Godot tests
# API and ABI breaking changes
- Nothing
# Expected complexity level and risk
1. It just updates the Blackholio demo project
# Testing
- [X] Play the updated demo
- [X] Run the tests locally
# Description of Changes
A follow-on to #5133 that fixes another Godot flake that was observed
after #5133 merged. Specifically a C# SDK regen flake where the Godot
package restore path reused `sdks/csharp/packages`, which is also the
Unity-visible package cache.
During `cargo regen csharp dlls`, the Godot flow previously did this:
- cleared `sdks/csharp/packages/godotsharp`
- cleared `sdks/csharp/obj~/godot`
- restored the Godot project back into `sdks/csharp/packages`
- packed the Godot project using that same Unity-visible package area
That meant a Godot-only cleanup could delete package content under the
Unity SDK package cache. If a previous GodotSharp restore was partial or
stale, regen tried to fix it by deleting `packages/godotsharp`, but that
path is visible to Unity and can cause dirty/missing package content
during regen.
With this change, Godot restore state is now isolated under
`sdks/csharp/obj~/godot/packages`, and the regen flow becomes:
- clear only `sdks/csharp/obj~/godot`
- restore the Godot project into `sdks/csharp/obj~/godot/packages`
- pack the Godot project under the same path
Clearing `obj~/godot` now removes both Godot intermediates and
Godot-only NuGet packages, while leaving the normal Unity SDK package
cache under `sdks/csharp/packages` untouched.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
- [x] `unity-testsuite` passes
# Description of Changes
Before this change, we used a single async-enabled wasm runtime for all
requests, even though procedures are the only operation that can yield.
Now each module gets two separate runtimes. We continue to use the same
async runtime for procedures, but now reducers are executed against a
synchronous wasm runtime, backed by a single OS-thread instead of a
Tokio runtime.
The purpose of this change is to remove from the critical path the
overhead associated with async calls that really aren't async at all.
Also includes the following fix from #5135:
> After #4973, WASM procedures can execute concurrently with later
operations on the same WebSocket.
> Before this change, the C# regression testsuite queued several
procedures, then immediately queued `UnsubscribeThen`. After #4973, the
unsubscribe could be applied before the `SubscriptionEventOffset`
procedure callback ran, clearing `MyTable` from the local subscribed
cache. The callback then failed while asserting that the `offset-test:`
row was present.
> This change treats unsubscribe as a separate phase. It is scheduled
after the main work is queued, but only starts once `waiting == 0`, so
all callbacks that inspect subscribed state run before the cache is
cleared.
# API and ABI breaking changes
None
# Expected complexity level and risk
2.5
# Testing
Pure refactor. Relies on current test coverage. #5078 will ensure the
performance is on par with V8.
# Description of Changes
Bump version numbers to 2.3.0 in preparation for an upcoming release
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
CI 🤷
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Just documentation fixes
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this is just a docs change
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] All links in the README work now.
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Very small additions to the C# SDK specific for Godot.
I wrote the Godot Blackholio tutorial and updated the Unity one.
I added the image assets necessary for the tutorial.
I added the files for the Godot demo.
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
1. There's really no risk for current systems or projects, it's all new
additions to support nicely(-ish) Godot, new Blackholio demo for Godot
and updates to the tutorials.
# Testing
- [X] Follow the tutorial and verify everything works.
- [X] Build for Windows and verify it works.
- [x] Build for Linux and verify it works.
- [x] Build SDK
- [x] Publish to Nuget
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
CI was running `gen-quickstart.sh` and then checking for a diff.. but it
was checking in the wrong directory.
I have also regenerated the files because the fixed check was failing.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [x] updated CI failed without the changes to the other files
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This is a fix for #4959.
Unity 6 upgraded to a newer Emscripten version that removed the
`dynCall()` library function. The SpacetimeDB C# SDK's
[WebSocket.jslib](cci:7://file:///D:/Projects/ClockworkLabs/SpacetimeDB/sdks/csharp/src/Plugins/WebSocket.jslib:0:0-0:0)
plugin used `dynCall()` in 6 places to invoke C# callbacks from
JavaScript WebSocket events.
This PR adds a `$WebSocketDynCall` helper function that detects which
API is available at runtime:
- On Unity 6+: uses `getWasmTableEntry(ptr).apply(null, args)` (direct
WASM function table access)
- On Unity 2022 and earlier: uses the legacy `dynCall(sig, ptr, args)`
All `dynCall` call sites in `WebSocket.jslib` now route through this
helper, ensuring we maintain backward-compatiblity.
# API and ABI breaking changes
No breaking changes. This is a backward-compatible fix that works on
both older and newer Unity versions.
# Expected complexity level and risk
1. Very low risk. The change is isolated to the WebGL `jslib` plugin and
uses runtime feature detection to choose the appropriate calling
mechanism. Both code paths are well-tested:
- Unity 2022.3.62f2: uses legacy `dynCall` path
- Unity 6000.4.5f1: uses `getWasmTableEntry` path
# Testing
- [x] Create test server module with reducers and tables
- [x] Unity 2022.3.62f2 Editor: Connect and subscribe works
- [x] Unity 2022.3.62f2 WebGL Build: Connect and subscribe works
- [x] Unity 6000.4.5f1 Editor: Connect and subscribe works
- [x] Unity 6000.4.5f1 WebGL Build: Connect and subscribe works (was
previously failing with `dynCall is not defined`)
# Description of Changes
This PR updates Rust SDK subscription unsubscribe handling to return an
error instead of panicking when the internal pending mutation channel is
closed.
## Motivation
`SubscriptionHandle::unsubscribe_then` currently calls `unwrap()` after
sending an unsubscribe mutation through the internal pending mutation
channel. If the connection or subscription manager has already shut
down, that send can fail with a disconnected channel error, causing the
application to panic.
I encountered this in [`bevy_stdb`](https://github.com/onx2/bevy_stdb),
which tracks subscriptions across reconnects and unsubscribes old
handles after applying a new subscription for the same key.
## Changes
- Replaces `unwrap()` on `pending_mutation_sender.unbounded_send(...)`
with error propagation.
- Returns `crate::Error::Internal(...)` when the unsubscribe mutation
cannot be sent.
- Updates local unsubscribe state only after the unsubscribe mutation is
successfully queued.
# API and ABI breaking changes
None expected.
This changes an internal error path from panicking to returning an
existing `crate::Error::Internal(...)` through the existing
`crate::Result<()>` return type.
# Expected complexity level and risk
Complexity: 1/5
This is a small, localized change. The method already returns
`crate::Result<()>`, so replacing the `unwrap()` with error propagation
fits the existing API shape.
Risk is low. Callers now receive an error if the internal pending
mutation channel is closed instead of panicking. Existing `AlreadyEnded`
and `AlreadyUnsubscribed` behavior is unchanged.
# Testing
- [x] Verified that `unsubscribe_then` returns
`Err(crate::Error::Internal(...))` instead of panicking when the pending
mutation channel is disconnected.
- [ ] Verified that successful unsubscribe behavior is unchanged.
- [ ] Verified that `AlreadyEnded` is still returned when the
subscription has already ended.
- [ ] Verified that `AlreadyUnsubscribed` is still returned when
unsubscribe was already requested.
---------
Signed-off-by: Jeff Rooks <onx2rj@gmail.com>
# Description of Changes
This pull request improves the handling of connection lifecycle events
in the Rust client SDK for SpacetimeDB, particularly distinguishing
between connection failures and disconnections. It introduces a new
`ConnectionLifecycle` state machine to track connection progress,
ensures that the correct callback (`on_connect_error` or
`on_disconnect`) is invoked based on the connection state.
**Changes**
* `ConnectionLifecycle` enum to track the connection state
(`Connecting`, `Connected`, `Ended`)
* Refactored error handling so that if a connection fails before
establishment, the `on_connect_error` callback is invoked; if the
connection fails after establishment, the `on_disconnect` callback is
invoked. See `end_connection`.
* Updated where disconnections are handled
(`advance_one_message_blocking`, `advance_one_message_async`, and
message processing) to use `finish_connection`
* Improved handling of user-initiated disconnects during the connection
process to avoid reporting them as connection errors and to ensure
proper cleanup.
# API and ABI breaking changes
I guess maybe if people relied on the `on_connect_error` to actually
fire the `on_disconnect` then this changes that behavior.
# Expected complexity level and risk
Maybe a 2? Seems pretty low risk but I'm still new to the codebase,
please double check.
This doesn't fix the websocket issues, that'll be for another day. I
noticed websocket.rs has some places it just drops and the error isn't
handled properly. We could technically surface that information and run
our callbacks with more specific error messages.
# Testing
I had an agent build and run loads of tests for this but didn't commit
those since it would have made the PR massive. I was planning on testing
locally though to see if I could trigger a connection failure at some
point, maybe via an invalid access token.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bumps version to 2.2.0
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
- 1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Version number is correct (`2.2.0`)
- [x] BSL license file has been updated with the new date and version
number
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
`UClientCache::ApplyDiff` (`sdks/unreal/.../DBCache/ClientCache.h`) has
an asymmetry between its insert and delete paths.
**Phase 1 (deletes)** correctly only emits a `Diff.Deletes` entry when
the row's refcount transitions to 0 — overlapping subscriptions just
decrement.
**Phase 2 (inserts)** always appends to `Diff.Inserts`, regardless of
whether the row was already cached:
```cpp
// before
FRowEntry<RowType>* Entry = Table->Entries.Find(Key);
if (!Entry) { /* refcount = 1 */ }
else { /* refcount + 1 */ }
Diff.Inserts.Add(Key, *NewRow); // ← fires for both branches
```
`BroadcastDiff` then fires `OnInsert` for every `Diff.Inserts` entry, so
any table subscribed by two overlapping queries (e.g. a global `SELECT *
FROM t` plus a per-row `WHERE id = ...`) re-fires its insert handler on
every later subscription apply — once per cached row, every time. Game
code that does work in `OnInsert` (positioning, spawning, snapping to
terrain) re-runs and clobbers state that was meant to be set once.
The intent is documented in `RowEntry.h`: *"Wrapper storing a row value
with a reference count used by overlapping subscriptions."* Phase 1
follows that design; phase 2 doesn't.
### Fix
Move the `Diff.Inserts.Add` into the `!Entry` branch only, so it fires
only on the absent → refcount=1 transition:
```cpp
if (!Entry) {
Table->Entries.Add(Key, FRowEntry<RowType>{NewRow, 1});
Diff.Inserts.Add(Key, *NewRow);
}
else {
Table->Entries.Add(Key, FRowEntry<RowType>{NewRow, Entry->RefCount + 1});
}
```
### Why real updates still work
Cache keys are BSATN row-bytes, not primary keys. A real update arrives
as a `(delete old_bytes, insert new_bytes)` pair where `old_bytes ≠
new_bytes` — so the insert side still takes the `!Entry` branch and gets
a `Diff.Inserts` entry. `FTableAppliedDiff::DeriveUpdatesByPrimaryKey`
then pairs the delete and insert by PK into
`UpdateInserts`/`UpdateDeletes`, and `OnUpdate` (not `OnInsert`) fires,
exactly as today.
Edge cases:
| Scenario | Phase 2 branch | Result | Correct? |
|---|---|---|---|
| New row | `!Entry` | `OnInsert` | ✓ |
| Real update (different bytes) | `!Entry` | `OnInsert`+`OnDelete`
reconciled to `OnUpdate` by PK | ✓ |
| Overlapping sub re-delivers cached row | `else` | refcount bump, no
event | ✓ (was broken — fired duplicate `OnInsert`) |
| Trivial update (identical bytes) | `else` | refcount bump | irrelevant
— server diffs identical rows away before emitting |
# API and ABI breaking changes
None. Purely internal cache bookkeeping. Existing
`OnInsert`/`OnDelete`/`OnUpdate` semantics are preserved for all
non-overlapping cases; the only behavior change is that overlapping
subscriptions stop emitting duplicate `OnInsert` events for
already-cached rows — which matches the documented `RowEntry` refcount
contract.
# Expected complexity level and risk
**1.** Two lines moved into a branch; comments updated. Mirrors logic
already present and known-correct in phase 1.
# Testing
Reproduced and validated downstream in an Unreal project. The repro
setup is straightforward to replicate against any module:
1. A table `t` with ~150 rows.
2. A global subscription `SELECT * FROM t`, applied first.
3. A diagnostic actor that binds `t.OnInsert`, then every 10s submits a
new overlapping subscription (e.g. `SELECT * FROM t` again, or any
`SELECT * FROM t WHERE 'id' = X` covering already-cached rows) and
counts the `OnInsert` events that arrive in that round.
Expected: round 1 fires once per row that is *new to the cache*;
subsequent rounds against already-cached rows fire 0.
Observed (~161 rows cached after initial load):
```
Pre-fix Post-fix
Global sub (empty → 161) 161 161 ← genuine inserts; unchanged
Round 1 (overlapping) 134 0 ← 134 cached dupes
Round 2 (overlapping) 134 0
Round 3 (overlapping) 134 0
Round 4–6 (overlapping) 134 each 0 each
```
The genuine "empty cache → 161 entries" wave on the initial global
subscription is unaffected — same `OnInsert` count both pre- and
post-fix. Only the duplicate fires from later overlapping subscriptions
on already-cached rows are eliminated. `OnUpdate` still fires correctly
when underlying rows actually change.
- [x] Reviewer: confirm an existing real-update (different bytes, same
PK) test still produces `OnUpdate` and not `OnInsert`+`OnDelete`.
- [x] Reviewer: confirm any test that relies on `OnInsert` firing on
every subscription apply (rather than only on cache 0→1 transition) is
not present — if it exists, it was relying on the bug.
# Description of Changes
Tests for case conversion.
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
---------
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Make Timestamp a FilterableValue in Rust, C#, and Typescript. I'm not
sure this is changing all the places because we have the server and the
client in those 3 languages.
# API and ABI breaking changes
It's an additive change.
# Expected complexity level and risk
3. There are some designs decisions, like comparing timestamp to
strings/numbers.
# Testing
Added unit tests for the 3 languages.
# Description of Changes
- Updated the Unreal SDK test harness to allow Unreal Editor to work
with MacOS
- Updated the Unreal SDK test handler to work with Nil as it's a special
case
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Small changes to the Unreal SDK tests
# Testing
- [x] Ran full suite of tests on Mac for Unreal SDK
- [x] Ran full suite of tests on Windows + Linux for Unreal SDK to
confirm no regression
---------
Co-authored-by: Jason Larabie <jasonlarabie@Mac.lan>
# Description of Changes
Reapply changes from #4515 after reversion
# API and ABI breaking changes
No API or ABI changes
# Expected complexity level and risk
2 - This PR change itself is trivial, as it just reimplements #4515,
however as #4515 had broken the `quickstart` smoketest, this should be
considered when reviewing this PR.
# Testing
- [X] Tested against `python3 -m smoketests quickstart` locally
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This is a copy of #4006 with only the updates to the way Unreal handles
ticks. As per @brougkr findings:
- **FTickableGameObject initialization order bug** - Replaced with
FTSTicker for reliable tick registration:
- Removed FTickableGameObject inheritance
- Added FTSTicker::FDelegateHandle for manual tick registration
- Added destructor to clean up ticker registration
- Added OnTickerTick() method
### Issue
FTickableGameObject registers itself in its constructor BEFORE
UDbConnectionBase's constructor body runs. Even with
ETickableTickType::Never, UE's GENERATED_BODY() macro can interfere with
base class initialization order, causing the default constructor to be
called instead.
# API and ABI breaking changes
- Refactor of an underlying component of the SDK and non-breaking.
# Expected complexity level and risk
2 - This changes the structure of how the database tooling can
auto-tick, but is invisible to the developer
# Testing
As this changes a core feature I tested all aspects:
- [x] Reproduced the bug and confirmed against `master` and this branch
to see the fix working
- [x] Ran full suite of Unreal tests
- [x] Manually tested Unreal Blackholio
# Description of Changes
Revert the following PRs that have caused some breakage:
```
a32cffa76 Finish refactoring out replay (#4850)
d639be0af Replay: some code motion & reuse `ReplayCommittedState` (#4849)
78d6b6f7d Update NativeAOT-LLVM infrastructure to current ABI (#4515)
d5c1738c1 Better module backtraces for panics and whatnot (#577)
6f23b19f3 Wait for database update to become durable (#4846)
81c9eab86 Add `spacetime lock/unlock` to prevent accidental database deletion (#4502)
809aebd7c Move field `replay_table_updated` to `ReplayCommittedState` (#4807)
21b58ef99 Update axum (#2713)
b5cadff7a Extract replay stuff out of `CommittedState`, part 1 (#4804)
```
I also updated the Python smoketests for breakage introduced in
https://github.com/clockworklabs/SpacetimeDB/pull/4502. Reverting that
PR caused conflicts, so this fix is more straightforward.
# API and ABI breaking changes
Maybe kind of, but we haven't released any of these.
# Expected complexity level and risk
1
# Testing
Ask @bfops about testing
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
## Summary
- Update the experimental NativeAOT-LLVM build path
(`EXPERIMENTAL_WASM_AOT=1`) to include all host function imports from
ABI versions 10.0 through 10.4
- Fix the compiler package reference to work on both Windows x64 and
Linux x64 (was hardcoded to Windows only)
- Add a CI smoketest to verify AOT builds work on Linux x64
## Context
See #4514 for the full writeup on the C# AOT situation. The
`wasi-experimental` workload that all C# module builds depend on is
deprecated and removed from .NET 9+. NativeAOT-LLVM is the recommended
path forward for ahead-of-time compilation of C# to WebAssembly.
The existing NativeAOT-LLVM support (added by RReverser in #713) was
stale: missing imports added since then and a Windows-only package
reference.
## Changes
**`SpacetimeDB.Runtime.targets`:**
- Add 10 missing `WasmImport` declarations across spacetime_10.0 through
10.4
- Replace `runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM` with
`runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM`
so it resolves correctly on Linux x64 as well
- Use explicit version strings instead of the `$(SpacetimeNamespace)`
variable
**`ci.yml`:**
- Add AOT build smoketest step in the `csharp-testsuite` job
## Test plan
- [x] CI smoketest passes: `EXPERIMENTAL_WASM_AOT=1 dotnet publish -c
Release` builds successfully on Linux x64
- [ ] Existing C# tests continue to pass (no changes to the default
interpreter path)
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Adds a `sdk-test-procedure-cs` test to the other test modules in
`sdks/rust/tests/test.rs`.
Functionally `sdk-test-procedure-cs` performs all the operations as the
Rust test `sdk-test-procedure` but through C#, fulfilling the
requirements of Issue #3951
# API and ABI breaking changes
No API or ABI changes, this only adds a test.
# Expected complexity level and risk
1
# Testing
- [X] Ran `cargo test -p spacetimedb-sdk csharp_procedures` locally
without errors.
## Summary
On macOS/Apple platforms, Objective-C++ defines `Nil` as a macro
(`#define Nil nullptr` in `objc/objc.h`). Unreal Engine compiles all C++
as Objective-C++ on Apple platforms, so `FSpacetimeDBUuid::Nil()` in
`Builtins.h` collides with this macro and produces:
```
error: expected member name or ';' after declaration specifiers; 'nullptr' is a keyword in Objective-C++
```
Every UE project using the SpacetimeDB Unreal SDK on macOS hits this
build failure.
## Affected platforms
- macOS (any version) — Unreal Engine 5.x compiles as Objective-C++ on
Apple platforms
- Affects all SpacetimeDB Unreal SDK versions that expose
`FSpacetimeDBUuid::Nil()`
## Reproduction steps
1. Create or open any Unreal Engine project on macOS
2. Add the SpacetimeDB Unreal SDK (v2.1.0)
3. Build the project
4. Observe the build failure in `Builtins.h` at
`FSpacetimeDBUuid::Nil()`
## Fix
Added `#pragma push_macro` / `#undef Nil` / `#pragma pop_macro` guards
in `Builtins.h`:
- **Before struct definitions**: Save and undefine the `Nil` macro so it
doesn't interfere with `FSpacetimeDBUuid::Nil()` and any other
identifiers using `Nil`
- **After the last closing brace**: Restore the original `Nil` macro so
downstream Objective-C++ translation units are unaffected
This is the standard UE pattern for handling platform macro collisions
(similar to how UE itself handles `check`, `verify`, `TEXT`, etc. macro
conflicts).
## Test plan
- [ ] Verify macOS UE build succeeds with the SDK included
- [ ] Verify Windows/Linux builds are unaffected (the `#ifdef Nil` guard
is a no-op when the macro is not defined)
- [ ] Verify `FSpacetimeDBUuid::Nil()` and `NilUuid()` Blueprint
functions work correctly at runtime
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
This is the implementation to resolve#4424
# Description of Changes
* Add support for `IEnumerable<T>` as a valid C# View return type in
codegen, so view implementations can return query results without
forcing `ToList()`. The generator now detects `IEnumerable<T>`,
serializes it via the list serializer.
* Snapshots have been updated with a test to confirm that
`IEnumerable<T>` rather than checking it is unsupported (which we had
added when removing support during the transition away from `Query<T>`
usage.
* Refreshes docs/examples to reflect the new supported signature.
# API and ABI breaking changes
No breaking changes. This is additive support for an existing .NET
interface.
# Expected complexity level and risk
2 - Low
# Testing
- [X] Rebuilt CLI locally, ran regression tests, and `dotnet tests` in
C# module bindings.
- [X] Tested updated docs code snippets to ensure they resolve in an IDE
and publish.
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
This PR adapts the Rust SDK test suite to work with the wasm version
added in https://github.com/clockworklabs/SpacetimeDB/pull/4089 (which
I've closed in favor of this PR).
Most of the changes revolve around wasm's different async semantics -
everything runs in one thread, so things that relied on background
threads didn't work directly. Several tests would lock up because
something in them blocked synchronously, which blocked any background
work from progressing.
We moved the test-clients contents into a `test_handlers.rs` so that it
could be called from both `main` (for native tests) and `lib` (for wasm
tests). To show what actually changed, use:
```bash
git diff --no-index -- <(git show origin/master:sdks/rust/tests/procedure-client/src/main.rs) sdks/rust/tests/procedure-client/src/test_handlers.rs
```
(or similar for other test-clients)
# API and ABI breaking changes
None, I think/hope.
# Expected complexity level and risk
2
# Testing
- [x] I've augmented the CI to also run the test suite with the `web`
feature
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Thales R <thlsrmsdev@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Bump versions to 2.1.0
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
CI
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
- Updated the Unreal SDK and generated Unreal bindings for the websocket
2.0 protocol/model
- Reworked DbConnectionBase to handle the updated message shapes
- Switched subscription handling over to new message types and
QuerySetId
- Updated reducer to ReducerResult, removal of callbacks, and set
reducer flags
- Added event table support
- Baked in multi-module support replacing [the old
PR](<https://github.com/clockworklabs/SpacetimeDB/pull/3417>)
- Added functionality to generate module support for multiple folders in
the Unreal project (add <module>.Build.cs, <module>.h, <module>.cpp)
using the --module-name
- Add new configuration option for spacetime generate to handle module
prefix
- Regenerated Unreal Blackholio/TestClient/QuickstartChat bindings
- Rebuilt Unreal Blackholio's consume entity to use event tables
- Updated migration documentation
- Updated the version bump tool to impact C++
# API and ABI breaking changes
- Unreal websocket/message handling updated to the new protocol
- Unreal generation now expects a real .uproject target and will stop
immediately if project
metadata is invalid instead of continuing past setup issues.
# Expected complexity level and risk
3 - A large set of changes to update the websocket/message handling
along with heavy codegen changes to handle multi-module support
# Testing
Test coverage of the Unreal SDK will need expansion in a future ticket
once our issues with flakiness on CI is resolved.
- [x] Updated Unreal Blackholio
- [x] Ran full Unreal SDK test suite
- [x] Built new test project using the new `--module-prefix`
- [x] Run through Unreal Blackholio (C++ and Blueprint)
- [x] Rebuilt Unreal Blackholio with multi-module, and duplicate
generated module testing side-by-side modules that would overlap
# Review Question(s)
- [x] Updates to `spacetime init` have made the tutorial a little
confusing with pathing for the Unreal Blackholio tutorial. To fix though
we'd have to update all the commands to be more explicit, or update the
tutorial `spacetime init` to use `--project-path .` to keep pathing
simpler, thoughts?
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
This bug was introduced when we added the v2 subscription protocol.
Before this change, when a v2 client disconnected or unsubscribed, other
clients could stop receiving updates for sender-scoped views they were
subscribed to.
This happened because `SubscriptionManager::remove_all_subscriptions()`
was cleaning up v2 subscription state correctly, but we would also run
the legacy v1 cleanup loop which mixed v1 and v2 queries. This meant the
same query hash could be processed again in the v1 cleanup loop after
the v2 removal had already happened. In the sender-view case, that
second pass could remove metadata (`indexes` / `search_args`) for active
subscribers.
Specifically, we could have the following sequence:
1. Client B disconnects.
2. The v2 loop removes B’s v2 subscription for `query_hash`.
3. `query_hash` now has no subscribers, so
`remove_query_from_tables(query_hash)` runs once.
4. That compatibility loop then visits `query_hash` again via
`subscription_ref_count.keys()`.
5. `query_hash` still has no subscribers, so
`remove_query_from_tables(query_hash)` runs a second time.
and when `remove_query_from_tables(query_hash)` runs a 2nd time, it
calls `index_ids.delete_index_ids_for_query()` which is refcounted, not
idempotent. So calling it twice for the same query decrements the shared
index count twice, and the 2nd decrement drops an index that client A’s
query still needs.
The fix was to:
1. Collect the disconnected client’s v1 query hashes from
`client_info.v1_subscriptions`
2. Run the v2 cleanup loop over `client_info.v2_subscriptions` (as we
did before this change)
3. Run the old compatibility cleanup loop only over the deduplicated v1
hashes
# API and ABI breaking changes
None
# Expected complexity level and risk
2
Small fix to a rather intricate bug. Most of the code in this patch is
testing code.
# Testing
This bug was not reproducible with the current smoketest harness,
because `spacetime subscribe` still uses the `v1` subscription protocol.
I added a Rust SDK test, which did reproduce the bug before the fix,
because the Rust SDK uses the `v2` subscription protocol.
# Description of Changes
Same change set as
https://github.com/clockworklabs/SpacetimeDB/pull/4614, just targeting
master.
The return type of a query builder view is now a special SATS product
type `{ __query__: T }`. A view with this return type now has a primary
key if `T` has a primary key. This means that client codegen will
generate an `OnUpdate` callback for such views. It will also generate
query builder index bindings for the primary key column.
# API and ABI breaking changes
None. Old modules with query builder views still work, they just don't
have primary keys.
# Expected complexity level and risk
2
# Testing
Added equivalent tests to the ones that were added in #4573 and #4572.
# Description of Changes
In the Rust client SDK, with this PR, doing
`.with_debug_to_file("path.txt")` on the connection builder will result
in the SDK logging additional verbose info to `path.txt`.
# API and ABI breaking changes
Adds a new user-facing-ish API to the Rust client SDK.
# Expected complexity level and risk
2? Some possibility of deadlock due to adding a new mutex, but this is
explicitly not for production use.
# Testing
- [x] Ran `chat-console-rs` locally with this enabled and got some debug
logs out of it.
# Description of Changes
Query builder views return a subset of rows from a physical table. If
that table has a primary key, then so should the view. What this means
concretely is that the view should expose the same api as the table,
specifically as it relates to the primary key column.
With that in mind, this patch commits the following changes:
1. Annotates `ViewDef` with a `primary_key`
2. Updates the return type of query builder views in the raw module def
to a special product type
3. Adds an index for the primary key on the view's backing table
4. Updates the query planner to use this index
5. Updates rust client codegen to generate `on_update` for such views
# API and ABI breaking changes
None
Old `impl Query` views compiled with an older version of SpacetimeDB
will continue to work as they did before - without a primary key.
# Expected complexity level and risk
3
# Testing
- [x] New rust sdk integration suite exercising `on_update` for PK views
and semijoin scenarios
- [x] Smoketests for PK views and semijoin scenarios
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bumps version to 2.0.4
# API and ABI breaking changes
None - this is just a version bump
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License file has been updated with correct version + time
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Adds implicit query builder conversions from `bool` to `BoolExpr` so
that you can write:
```rust
ctx.from.user().r#where(|u| u.online)
```
instead of
```rust
ctx.from.user().r#where(|u| u.online.eq(true))
```
Also removes `NullableCol` and `NullableIxCol` types from C# query
builder.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Unit and smoketests
# Description of Changes
- Migrated the C++ module-definition assembly path to V10-first
internals:
- Added v10_builder and module_type_registration systems.
- Switched Module::__describe_module__ to serialize RawModuleDef with
V10 payload.
- Updated macro registration pipeline to register through V10
- Added explicit naming support across macro surface (*_NAMED variants
for reducer/procedure/
view and field/index macros).
- Reworked multi-column index macros (FIELD_MultiColumnIndex,
FIELD_MultiColumnIndex_NAMED) with
migration alias.
- Added SPACETIMEDB_SETTING_CASE_CONVERSION(...) to support case
conversion policy
- Error-path hardening by adding explicit constraint-registration error
tracking and preinit validation
- Codegen updates:
- Updated C++ moduledef regen to V10 builder types.
- Adjusted C++ codegen duplicate-variant wrapper generation to emit
proper product-type
wrappers.
- Test/harness updates:
- type-isolation-test runner now defaults to focused V10 regression
checks; --v9 runs broader
legacy/full suite.
- Added focused modules for positive/negative V10 checks:
- test_multicolumn_index_valid
- error_multicolumn_missing_field
- error_default_missing_field
- Re-enabled C++ paths in sdks/rust/tests/test.rs procedure/view/test
suites.
# API and ABI breaking changes
- Refactor of the underlying module definition
- New *_NAMED variant macros for explicit canonical naming
- FIELD_NamedMultiColumnIndex renamed to FIELD_MultiColumnIndex
# Expected complexity level and risk
3 - Large set of changes moving over to V10 with underlying changes to
make future updates a little easier
# Testing
- [x] Ran the type isolation test and expanded it
- [x] Ran the spacetimedb-sdk test framework to confirm no more drift
between C++ and other module languages
- [x] Ran Unreal test suite though not really applicable
- [x] New app creation with `spacetime init --template basic-cpp`
- [x] Ran describe module tests against Rust + C# matching with C++ on
the /modules/sdk-test* modules to find any possible mis-alignment
# Review
- [x] Another look at the new features with C++
- [x] Thoughts on *_NAMED macros, I couldn't come up with a better
solution with C++20
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade 2.0.3
# API and ABI breaking changes
- None, this is a version bump
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
- 1 - this is a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License has been updated
- [x] Version number is correct (2.0.3)
# Description of Changes
Bumping all versions to 2.0.2 for the release.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
CI
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Just bumping versions to 2.0.1.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Jason Larabie