Fixes#4592
Adds a "Supported Column Types" section to the Index documentation page
listing:
- **Supported types**: integers (all widths), bool, String, Identity,
ConnectionId, Uuid, Hash, and no-payload enums with
`#[derive(SpacetimeType)]`
- **Unsupported types**: floats (no total ordering),
ScheduleAt/TimeDuration/Timestamp (not yet supported, links #2650),
Vec/arrays, enums with payloads, nested structs
- **Workaround tip**: scaled integers for floating-point coordinates
- **Direct index restrictions**: only u8/u16/u32/u64 and no-payload
enums
The list is derived from the `FilterableValue` trait implementations in
`crates/lib/src/filterable_value.rs` and the direct index validation in
`crates/schema/src/def/validate/v9.rs`.
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Introduces types `TypedIndexKey` and `IndexKey` to free the table index
code on its direct dependency on `AlgebraicValue`, allowing index scans
by BSATN, `RowRef`s as well without first going through
`AlgebraicValue`.
This also has the effect of optimizing string scans by avoiding
allocating in `AlgebraicValue::String`.
This also adds a byte-array based future optimization for
all-primitive-multi-column indexes.
Also in the future, this will enable optimizing `iter_by_col_eq`, which
is used by the frequent connect/disconnect logic.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? Unsafe code and very load bearing code.
# Testing
Should be covered by existing tests.
Use a bounded channel for submitting transactions to the durability
layer, so as to apply backpressure to the db when transaction volume is
too high.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
This cleans up the `keynote-2` benchmark CLI and fixes the `bench` path
that was broken after `demo` and `bench` started sharing a single
parser. `demo` and `bench` now parse their own command grammars instead
of sharing one import-time parser. This avoids `bench` inheriting
`demo`-style validation and breaking on `test-1 --connectors ...`. in
addition, `bench` also uses the same `cac`-builder as `demo` now, and
I've deleted the unused `runner_1.ts`.
# API and ABI breaking changes
None
# Expected complexity level and risk
3
# Testing
Manually tested the following from `templates/keynote-2`:
- `pnpm bench test-1 --seconds 10 --concurrency 50 --alpha 1.5
--connectors spacetimedb`
- `pnpm demo --seconds 1 --concurrency 5 --alpha 1.5 --systems
spacetimedb --skip-prep --no-animation`
- `deno run --sloppy-imports -A src/demo.ts --help`
- `deno run --sloppy-imports -A src/cli.ts --help`
- `deno run --sloppy-imports -A src/cli.ts test-1 --seconds 10
--concurrency 50 --alpha 1.5 --connectors spacetimedb`
# Description of Changes
While testing #4663, I discovered the server would crash from a V8 out
of memory error after processing many requests. Before #4663, this would
not happen. I theorized that because we now have a single JS worker that
can process an unbounded number of reducer calls over its lifetime, any
V8 heap retention that would previously have been spread across several
pooled isolates now accumulates in one isolate.
This patch now periodically collects heap statistics and forces GC or
replaces the isolate if memory cannot be reclaimed. This greatly reduces
the risk of hitting the V8 heap limit and crashing the server. It
doesn't remove the risk entirely however. But this risk was still
present before we switched to a single worker model in #4663. In order
to remove the risk of crashing the server entirely, we would need to run
V8 in a separate process.
# API and ABI breaking changes
None
# Expected complexity level and risk
3
# Testing
TBD
# Description of Changes
The current keynote-2 benchmarks pipelines operations via
`MAX_INFLIGHT_PER_WORKER` in order to simulate a large number of client
connections while running the benchmark locally or on a single machine.
This patch adds a distributed benchmark mode for `templates/keynote-2`
so explicit SpacetimeDB client connections can be spread across multiple
machines without changing the existing single-process benchmark flow.
This is a pure extension. `npm run bench` and the current
`src/core/runner.ts` path remain intact. The new distributed path adds a
small coordinator/generator/control-plane harness specifically for
multi-machine ts client runs.
- New CLI entry points `bench-dist-coordinator`, `bench-dist-generator`,
and `bench-dist-control` were added
- The coordinator defines the benchmark window
- Generators begin submitting requests during warmup, but warmup
transactions are excluded from TPS
- Throughput is measured from the server-side committed transfer
counter, not client-local TPS
- Each connection runs closed-loop with one request at a time in this
distributed mode
- Connection startup is bounded-parallel (`--open-parallelism`) to avoid
a connection storm
- Verification is run by the coordinator after the epoch
- Late generators can be registered after a run to increase load on the
server incrementally
- If a participating generator dies and never sends `/stopped`, the
epoch result is flagged with an error so the run can be retried cleanly
See `DEVELOP.md` for instructions on how to run.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
3
# Testing
Manual
# Description of Changes
This recovers a bit of lost performance from event tables by:
- avoiding putting the `row: ProductValue` in a vector before merging
into the committed state.
- directly constructing `Arc<[ProductValue]>`
This also shaves off 8 bytes from `ReplicaCtx`.
# API and ABI breaking changes
None
# Expected complexity level and risk
2?
# Testing
Covered by existing tests.
# Description of Changes
Now we get a `--help` for the benchmark, which is nicer. Also now can
run under deno, with `deno --sloppy-imports -A src/demo.ts` (might be
useful, deno's websocket is implemented in native code while node's is
implemented in JS). I removed the
[BOM](https://en.wikipedia.org/wiki/Byte_order_mark) because it seems
unintentional (only found in `templates/keynote-2`) and was causing a
little bit of weirdness.
Also, fix the rust benchmark client as a follow-up to #4616
# Expected complexity level and risk
1
# Testing
- [x] Works under deno and has usage
# Description of Changes
Before this change, JS reducer requests borrowed a `JsInstance` from a
pool. If no idle instance was available, we created another instance,
which meant another V8 worker thread. Under load, this meant reducers
bouncing across multiple OS threads.
After this change, JS reducers go through a single long-lived
`JsInstance` fed by a FIFO queue which results in much better cache
locality. More accurately, each module now allocates a single OS thread,
on which reducers (and most operations) run. Modules do not share
workers/threads. And modules do not create multiple threads for running
reducers.
Note, the original instance pool is still used for procedures. It should
probably be bounded, but I didn't make any changes to it. It's also used
for executing views during initial subscription to avoid a reentrancy
deadlock. The latter should be fixed and moved over to the JS worker
thread at some point.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
4
# Testing
```
NODE_OPTIONS="--max-old-space-size=8192" \
MAX_INFLIGHT_PER_WORKER=512 \
BENCH_PRECOMPUTED_TRANSFER_PAIRS=1000000 \
pnpm bench test-1 --seconds 10 --concurrency 50 --alpha 1.5 --connectors spacetimedb
```
```
50K TPS -> 85K TPS on m2 mac
```
This is the implementation to resolve#4424
# Description of Changes
* Add support for `IEnumerable<T>` as a valid C# View return type in
codegen, so view implementations can return query results without
forcing `ToList()`. The generator now detects `IEnumerable<T>`,
serializes it via the list serializer.
* Snapshots have been updated with a test to confirm that
`IEnumerable<T>` rather than checking it is unsupported (which we had
added when removing support during the transition away from `Query<T>`
usage.
* Refreshes docs/examples to reflect the new supported signature.
# API and ABI breaking changes
No breaking changes. This is additive support for an existing .NET
interface.
# Expected complexity level and risk
2 - Low
# Testing
- [X] Rebuilt CLI locally, ran regression tests, and `dotnet tests` in
C# module bindings.
- [X] Tested updated docs code snippets to ensure they resolve in an IDE
and publish.
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
This PR adapts the Rust SDK test suite to work with the wasm version
added in https://github.com/clockworklabs/SpacetimeDB/pull/4089 (which
I've closed in favor of this PR).
Most of the changes revolve around wasm's different async semantics -
everything runs in one thread, so things that relied on background
threads didn't work directly. Several tests would lock up because
something in them blocked synchronously, which blocked any background
work from progressing.
We moved the test-clients contents into a `test_handlers.rs` so that it
could be called from both `main` (for native tests) and `lib` (for wasm
tests). To show what actually changed, use:
```bash
git diff --no-index -- <(git show origin/master:sdks/rust/tests/procedure-client/src/main.rs) sdks/rust/tests/procedure-client/src/test_handlers.rs
```
(or similar for other test-clients)
# API and ABI breaking changes
None, I think/hope.
# Expected complexity level and risk
2
# Testing
- [x] I've augmented the CI to also run the test suite with the `web`
feature
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Thales R <thlsrmsdev@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fix the SQLite RPC benchmark so transfers actually persist and
verification produces useful output.
**`sqlite-rpc-server.ts`:**
- Add `.run()` to both Drizzle `tx.update()` chains in `rpcTransfer`.
Without this, the UPDATE statements were never executed — the benchmark
was only measuring SELECT + HTTP overhead, not real transactional
writes. This inflated SQLite TPS numbers significantly.
- Replace the generic `"internal error"` catch-all in `handleRpc` with
`rpcErr()`, which returns the actual error message to the client. Also
wrap the outer HTTP handler in a try/catch so errors outside `handleRpc`
are surfaced too.
**`sqlite_rpc.ts` (connector):**
- Detect `{ skipped: true }` from the server's verify endpoint and throw
a clear error telling the user to set `SEED_INITIAL_BALANCE` on the RPC
server process. Previously this was silently treated as success.
**`runner.ts` / `runner_1.ts`:**
- Log "Verification passed" on success and "Verification failed:
\<reason\>" (as a string, not a raw Error object) on failure, so the
outcome is always visible in bench output.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [ ] Run `npx tsx src/rpc-servers/sqlite-rpc-server.ts` (with
`SEED_INITIAL_BALANCE` set), then `npm run test-1 -- --connectors
sqlite_rpc --seconds 5` with `VERIFY=1`. Confirm TPS drops significantly
vs the old (broken) numbers and "Verification passed" appears.
- [ ] Run the same without `SEED_INITIAL_BALANCE` on the server process.
Confirm "Verification failed" with a message about the missing env var
(not "internal error").
Fixes#4651
When `spacetime.json` has a top-level `module-path` and `generate`
entries:
```json
{
"module-path": "./server",
"generate": [
{ "language": "rust", "out-dir": "./game_client/src/bindings" }
]
}
```
`spacetime dev` passed the generate entries to `exec_from_entries`
without inheriting the top-level `module-path`. The generate step then
fell back to the hardcoded `spacetimedb` default, failing with:
```
Error: Could not find module source at '.../spacetimedb'.
```
The standalone `spacetime generate` command already handled this
correctly via `collect_all_targets_with_inheritance()` (which merges
entity fields into generate entries), but `spacetime dev` bypassed that
path and read `config.generate` directly.
**Fix**: Inject the top-level `module-path` into generate entries that
do not specify their own, before passing them to `exec_from_entries`.
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Per Joshua: the note about `Math.random()` in the reducer context docs
is out of date and incorrect. TypeScript modules should use `ctx.random`
instead.
**Changes:**
- Remove `Math.random()` from the warning about non-deterministic RNGs
(it was listed alongside C#'s `Random` as something to avoid)
- Replace the incorrect `:::note` block with a proper table row
documenting `ctx.random` in the TypeScript context properties reference
- Same fixes applied to versioned docs (v1.12.0)
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Make confirmed reads the default for the typescript benchmark client.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
N/A
`RelationalDB::commit_tx` and `RelationalDB::commit_tx_downgrade`
release the exclusive transaction lock before calling
`DurabilityWorker::request_durability`. This can lead to transactions
appearing out-of-order on the worker queue, if transactions on the same
database instance are committed from multiple threads.
To mitigate this, the worker now keeps a small min-heap to re-order
transactions in that case. We expect the ordering issue to happen very
rarely, so the overhead of this should be negligible.
Supersedes #4661, which addressed the issue by holding the transaction
lock until durability request submission is completed.
# Expected complexity level and risk
2
# Testing
- [ ] Unit tests for the min-heap implementation
- [ ] Ran the keynote benchmark with various parameters
that would trigger the error without this patch
# Description of Changes
Bump versions to 2.1.0
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
CI
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Implements the fix for #4130 by updating docs with accurate usage
instructions for Query with Indexes in C#.
# API and ABI breaking changes
Docs only, no API or ABI changes
# Expected complexity level and risk
1
# Testing
- [X] Tested code snippets through addition of regression tests in #4123
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Disabling this for now since it's buggy and I'm about to be out for a
while.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
This can't be tested since it runs using the version in the base branch.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
This resolves an issue found while testing C# module behavior against
the Blackholio demo in an unrelated PR.
# Description of Changes
Updates the C# and C++ server modules for the Blackholio demo, to
reflect changes made to the Rust module in #4217
This is needed if testing against Blackholio using either the C# or C++,
or else the `ConsumeEntityEvent` will be missing from the generated
client bindings.
# API and ABI breaking changes
Not API or ABI breaking.
# Expected complexity level and risk
1
# Testing
- [X] Tested Unity Blackholio demo using the C# module
- [X] Tested Unity Blackholio demo using the C++ module
# Description of Changes
It looks like we could be creating a lot of instances, so it would be
nice to have a metric to track it.
# Expected complexity level and risk
1
# Description of Changes
Adds `tools/templates/` scripts to derive template metadata from
manifests and generate READMEs from quickstart docs. Replaces slug-based
`builtWith` with manifest-derived data and hardcoded quickstart mappings
with discovery from docs.
**Manifest-based `builtWith`** (`update-template-jsons.ts`)
- Reads `package.json`, `Cargo.toml`, and `.csproj` to populate
`builtWith` in `.template.json`.
- Scoped npm packages normalize to scope (`@angular/core` → `angular`).
Excludes `@types/*`. Adds `nodejs` only for nodejs-ts when `@types/node`
is present.
- Root manifests processed before subdirs; primary framework first (e.g.
`react` before `spacetimedb` in react-ts). Dependencies reordered in
package.json where needed.
**Dynamic quickstart discovery** (`generate-template-readmes.ts`)
- Discovers template → quickstart by parsing `--template X` from files
in `docs/docs/00100-intro/00200-quickstarts/`.
- Optional `quickstart` override in `.template.json`; must resolve under
quickstarts dir.
- chat-react-ts has no quickstart; uses manual README.
**New:** `tools/templates/` (update-template-jsons.ts,
generate-template-readmes.ts, README, package.json, pnpm-lock).
**Modified:** all `templates/*/.template.json` (added `builtWith`),
new/generated `templates/*/README.md`.
# API and ABI breaking changes
None.
# Expected complexity level and risk
**1** – Dev tooling only. No runtime or API changes. Scripts are
isolated; failures only affect generated metadata and READMEs.
# Testing
- [ ] `cd tools/templates && pnpm run generate` completes without errors
- [ ] Spot-check `builtWith` and generated READMEs for a few templates
## Summary
This PR fixes an issue where C# reserved keywords (like `params`,
`class`, `event`, etc.) used as field or parameter names in SpacetimeDB
types would cause compilation errors in the generated code.
## Problem
When a user defines a table or reducer with a field named using a C#
reserved keyword:
```csharp
[SpacetimeDB.Table]
public partial struct MyTable
{
public int @params; // User escapes it in their code
public string @class;
}
```
The codegen would generate invalid C# like:
```csharp
// Generated code (broken)
public int params; // Error: keyword used as identifier
public void Read(BinaryReader reader) {
params = ...; // Error
}
```
## Solution
1. Added an `Identifier` property to `MemberDeclaration` in the codegen
that automatically detects C# reserved keywords using
`SyntaxFacts.GetKeywordKind()` and prefixes them with `@` when needed.
2. Updated all code generation sites to use `Identifier` instead of
`Name` when generating:
- Field declarations
- Property accessors
- Constructor parameters
- BSATN serialization code
- Index accessors
- Reducer/procedure parameters
3. Added a regression test that verifies tables, reducers, and
procedures with keyword field names compile successfully.
## Test Plan
- Added `CSharpKeywordIdentifiersAreEscapedInGeneratedCode` test that
creates a table with `@class` and `@params` fields, plus a reducer and
procedure with keyword parameters
- Existing tests continue to pass (verified locally with
`FormerlyForbiddenFieldNames` fixture which already tests edge cases
like `Read`, `Write`, `GetAlgebraicType`)
Fixes#4529
---------
Co-authored-by: Stable Genius <259448942+stablegenius49@users.noreply.github.com>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
Ditches `TypedIndexPointIter::Direct` and then does some renaming of
variants in `TypedIndexPointIter` to reflect reality.
Extracted from https://github.com/clockworklabs/SpacetimeDB/pull/4311.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
Adds `Deserialize::validate` and all the friends necessary to make that
work.
The point of this is to enable allocation-less validation of input in
some data format, in particular BSATN, against some expected type, in
particular an `AlgebraicValue` at an `AlgebraicType`.
This is extracted from
https://github.com/clockworklabs/SpacetimeDB/pull/4311 and is used there
to convert raw BSATN to `BytesKey` without constructing the composite
`AlgebraicValue` just for validation only to throw it away immediately.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
TODO: tweak existing SATS roundtrip tests to also check validation.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
This patch removes the dead legacy SQL query engine and the remaining
code that only existed to support it.
Removed:
- Old SQL compiler/type-checker and VM-based execution path in
spacetimedb-core
- `spacetimedb-vm` crate
- Dead vm specific error variants and compatibility code
- Obsolete tests, benchmarks, and config paths that still referenced the
legacy engine
Small pieces still used by the current engine were moved to their proper
homes instead of keeping the `vm` crate around. In particular,
`RelValue` was moved to `spacetimedb-execution`.
The `sqltest` crate was also updated to use the current engine. Notably
though these tests are not run in CI, however I've kept them around as
they may be beneficial as we look to expand our SQL support in the
future.
Requires codeowner review from @cloutiertyler due to the removal of the
`LICENSE` file in the (now removed) `vm` crate.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Title.
Is useful to more easily reuse code.
I asked @joshua-spacetime on discord if this would be reasonable and he
said 👍 .
This mirrors the method available on `ReducerContext` for consistency.
# API and ABI breaking changes
None
# Expected complexity level and risk
1 - minimal api surface addition
# Testing
- [x] Tested in my project and works as expected.
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Update pgwire to 0.37
Make `DataRowEncoder` reusable as enabled in this release, make
introduce 3x performance on encoding.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
N/A
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
When creating sql queries, we were using the column accessor name,
rather than the name used in the database. This should fix that.
# Expected complexity level and risk
1.
# Testing
This has some unit tests.
# Description of Changes
- Fix issue with back to top button displays incorrectly
- Move the button to bottom left not to overlap with Ask AI
# Demo and screenshots
- After
<img width="1459" height="894" alt="image"
src="https://github.com/user-attachments/assets/75b4edda-dc54-493f-9274-0ca78f10fac7"
/>
https://github.com/user-attachments/assets/4cf7c3f4-51cc-4987-87db-b065ce4f7746
- Before
<img width="173" height="78" alt="image"
src="https://github.com/user-attachments/assets/052be945-fd2b-4276-a129-89c34a6a818b"
/>
<!-- Please describe your change, mention any related tickets, and so on
here. -->
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Back to top button displays correctly and works
Closes#4635
In `crates/client-api/src/routes/database.rs` line 1111,
`lookup_database_identity` returns a `Result` but the error was handled
with `.unwrap()`, which panics instead of returning an HTTP 500.
Replaced it with `.map_err(log_and_500)?` to match the rest of the file.
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Summary
When an HTTP request fails inside a procedure, the error message only
shows something like "error sending request for url (https://...)" with
no details about what actually went wrong.
This change walks the full error chain so the message includes the root
cause — things like DNS failures, connection refused, timeouts, etc.
- Before: error sending request for url (https://httpbin.org/get)
- After: error sending request for url (https://httpbin.org/get): error
trying to connect: dns error: failed to lookup address information:
Fixes#4608
Test plan
- Trigger an HTTP failure in a procedure and check that the error
message now shows the actual reason for the failure
- Existing tests should still pass since this only changes error
formatting
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
- Updated the Unreal SDK and generated Unreal bindings for the websocket
2.0 protocol/model
- Reworked DbConnectionBase to handle the updated message shapes
- Switched subscription handling over to new message types and
QuerySetId
- Updated reducer to ReducerResult, removal of callbacks, and set
reducer flags
- Added event table support
- Baked in multi-module support replacing [the old
PR](<https://github.com/clockworklabs/SpacetimeDB/pull/3417>)
- Added functionality to generate module support for multiple folders in
the Unreal project (add <module>.Build.cs, <module>.h, <module>.cpp)
using the --module-name
- Add new configuration option for spacetime generate to handle module
prefix
- Regenerated Unreal Blackholio/TestClient/QuickstartChat bindings
- Rebuilt Unreal Blackholio's consume entity to use event tables
- Updated migration documentation
- Updated the version bump tool to impact C++
# API and ABI breaking changes
- Unreal websocket/message handling updated to the new protocol
- Unreal generation now expects a real .uproject target and will stop
immediately if project
metadata is invalid instead of continuing past setup issues.
# Expected complexity level and risk
3 - A large set of changes to update the websocket/message handling
along with heavy codegen changes to handle multi-module support
# Testing
Test coverage of the Unreal SDK will need expansion in a future ticket
once our issues with flakiness on CI is resolved.
- [x] Updated Unreal Blackholio
- [x] Ran full Unreal SDK test suite
- [x] Built new test project using the new `--module-prefix`
- [x] Run through Unreal Blackholio (C++ and Blueprint)
- [x] Rebuilt Unreal Blackholio with multi-module, and duplicate
generated module testing side-by-side modules that would overlap
# Review Question(s)
- [x] Updates to `spacetime init` have made the tutorial a little
confusing with pathing for the Unreal Blackholio tutorial. To fix though
we'd have to update all the commands to be more explicit, or update the
tutorial `spacetime init` to use `--project-path .` to keep pathing
simpler, thoughts?
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes
Cpp-related tutorial stale links correction in Readme.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Testing
N/A (?)
Signed-off-by: Leonid Satanovsky (home profile) <539045+leosat@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Fixes a bug during view cleanup where we would drop anonymous views that
had live subscribers.
The reason for this bug is that the `st_view_sub` does not have a single
entry for an anonymous view. Rather it has multiple entries - one per
identity subscribed. However the code was assuming only a single entry
per anonymous view, and so when view cleanup ran, we would only look at
the first entry in `st_view_sub` for an anonymous view, and if there
were no subscribers, we would drop the view's backing table and read
set. But there could still have been other clients subscribed to that
view, at which point they would stop receiving updates.
This change fixes the bug by making sure there aren't any rows in
`st_view_sub` with live subscribers before dropping the anonymous view's
backing table and read set.
The better fix would be to update `st_view_sub` so that there's only
ever at most one row per anonymous view. Unfortunately `st_view_sub`
mixes both anonymous and sender-scoped views, and the `sender` column is
of type `Identity`, not `Option<Identity>`. Instead of adding a new
system table to correct this, I chose to keep using `st_view_sub` and
delay this refactor until we add support for view parameters.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5
# Testing
Added regression tests that assert on the underlying system table rows.
A smoketest was **not** added because cleanup runs periodically and
currently there's no way to force a clean up.
# Description of Changes
This bug was introduced when we added the v2 subscription protocol.
Before this change, when a v2 client disconnected or unsubscribed, other
clients could stop receiving updates for sender-scoped views they were
subscribed to.
This happened because `SubscriptionManager::remove_all_subscriptions()`
was cleaning up v2 subscription state correctly, but we would also run
the legacy v1 cleanup loop which mixed v1 and v2 queries. This meant the
same query hash could be processed again in the v1 cleanup loop after
the v2 removal had already happened. In the sender-view case, that
second pass could remove metadata (`indexes` / `search_args`) for active
subscribers.
Specifically, we could have the following sequence:
1. Client B disconnects.
2. The v2 loop removes B’s v2 subscription for `query_hash`.
3. `query_hash` now has no subscribers, so
`remove_query_from_tables(query_hash)` runs once.
4. That compatibility loop then visits `query_hash` again via
`subscription_ref_count.keys()`.
5. `query_hash` still has no subscribers, so
`remove_query_from_tables(query_hash)` runs a second time.
and when `remove_query_from_tables(query_hash)` runs a 2nd time, it
calls `index_ids.delete_index_ids_for_query()` which is refcounted, not
idempotent. So calling it twice for the same query decrements the shared
index count twice, and the 2nd decrement drops an index that client A’s
query still needs.
The fix was to:
1. Collect the disconnected client’s v1 query hashes from
`client_info.v1_subscriptions`
2. Run the v2 cleanup loop over `client_info.v2_subscriptions` (as we
did before this change)
3. Run the old compatibility cleanup loop only over the deduplicated v1
hashes
# API and ABI breaking changes
None
# Expected complexity level and risk
2
Small fix to a rather intricate bug. Most of the code in this patch is
testing code.
# Testing
This bug was not reproducible with the current smoketest harness,
because `spacetime subscribe` still uses the `v1` subscription protocol.
I added a Rust SDK test, which did reproduce the bug before the fix,
because the Rust SDK uses the `v2` subscription protocol.
The commitlog so far assumed that the latest segment is never compressed
and can be opened for writing (if it is intact).
However, restoring the entire commitlog from cold storage results in all
segments being compressed. Make it so the resumption logic reads the
metadata from the potentially compressed last segment, and starts a new
segment for writing if the latest one was indeed compressed.
# Expected complexity level and risk
1.5
# Testing
Added a test.
# Description of Changes
Fixes a bug where clients subscribed through views could miss delete
updates when a one-shot scheduled reducer or procedure completed and the
scheduler automatically deleted the corresponding schedule row. This
patch ensures that scheduled-row cleanup refreshes any stale views
before commit/broadcast.
Note, this only affected one-shot schedules. Interval schedules do not
delete their schedule row after each run, so they were not going through
this path.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5
# Testing
Added regression coverage for:
- one-shot scheduled reducer: table + view subscription sees insert then
delete
- one-shot scheduled procedure: table + view subscription sees insert
then delete
- failing scheduled reducer: cleanup still refreshes the dependent view
delete
- observe transitive scheduled table updates through join
The one-shot tests now assert on the scheduled table and dependent view
in the same subscription updates, which proves the delete comes from the
automatic cleanup transaction itself rather than a later manual write.
# Description of Changes
Adds `count` to rust's procedural view API.
This API already existed in C# and typescript. This patch just adds
tests for those module languages.
We were already tracking reads for `count`, and so all this patch does
is add the new rust module bindings.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Smoketests for rust, C#, and typescript.
# Description of Changes
Adds a new reference documentation page for `spacetime.json` and fixes
several bugs where the CLI behavior diverged from the proposal.
**Docs:**
- New page at `/cli-reference/spacetime-json` covering config structure,
field reference, generate configuration, children/inheritance,
`spacetime dev` config, database selection, flag overrides,
`--no-config`, `--env`/environments, config file discovery, and editor
support
**Bug fixes:**
- `generate` was incorrectly inherited by child databases. A child with
a different `module-path` would silently inherit the parent's generate
entries, causing bindings to be written to the wrong output directory.
Generate is now never inherited, matching the proposal.
- The source conflict rule for `module-path`/`bin-path`/`js-path` was
not implemented during inheritance. A child specifying `module-path`
could still inherit `bin-path` from the parent. Now, if a child
specifies any module source, the others are not inherited.
- `--num-replicas` was not marked as a per-database override, so it
could be used with multiple databases selected without error.
# API and ABI breaking changes
None. These are bug fixes aligning the implementation with the intended
behavior from the proposal:
- `generate` inheritance was never documented or intended
- The source conflict rule was specified in the proposal but not
implemented
- `--num-replicas` as per-database is consistent with the other
module-source flags
# Expected complexity level and risk
2 - The changes are small and well-scoped. The generate inheritance fix
simplifies the code (removes a parameter). The source conflict rule adds
a straightforward check during field inheritance. Tests have been
updated to match.
# Testing
- [x] All 136 existing CLI tests pass
- [x] Updated tests for generate non-inheritance behavior
- [x] Docs site builds successfully, page renders in sidebar
- [ ] Manual test: verify a child with a different `module-path` no
longer inherits parent's `generate`
- [ ] Manual test: verify `--num-replicas` errors when multiple
databases are selected
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Fixes#4559.
## Bug
The `subscribe` callback passed to `useSyncExternalStore` captures
`computeSnapshot` in its closure but did not list it as a dependency.
When `subscribeApplied` flips to `true`:
1. `computeSnapshot` is recreated with `subscribeApplied = true`
2. But `subscribe` still holds the **stale** closure that captured
`subscribeApplied = false`
3. On the next row event, the stale `computeSnapshot` writes `[rows,
false]` into `lastSnapshotRef`
4. `isReady` drops to `false` permanently
## Fix
Add `computeSnapshot` to the `subscribe` dependency array so event
handlers always use the current snapshot function.
Note: PR #4499 previously fixed a related issue where the cached
snapshot was stale after `subscribeApplied` changed. This fix addresses
the remaining case where the `subscribe` closure itself was stale.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Updates the reference docs with the implications of primary key
inference for query builder views, implemented in #4572, #4573, #4614.
# API and ABI breaking changes
None
# Expected complexity level and risk
1. Docs only change
# Testing
N/A
# Description of Changes
Fixes the `reducers.onTransfer is not a function` error - that was old
code left from the 1.x sdk. Also, regenerate the module_bindings and
make sure the rust and typescript modules are identical in that respect.
# Expected complexity level and risk
1
# Testing
- [x] Verified that `spacetime generate --lang typescript --module-path
spacetimedb --out-dir module_bindings` and `spacetime generate --lang
typescript --module-path rust_module --out-dir module_bindings` give the
same output.
- [x] Setting `USE_SPACETIME_METRICS_ENDPOINT` to `0` no longer causes
the error.
clockwork-labs-bot