# Description of Changes
This reverts the version bump, since it seems to be causing test
flakiness somehow.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
The first commit defines a type `TableName` that is used in e.g.,
`TxData` and where determined profitable and necessary to do this
change.
`TableName` is backed by
[`ecow::EcoString`](https://docs.rs/ecow/0.2.6/ecow/string/struct.EcoString.html)
which affords O(1) clones and 15 bytes of inline storage and
`mem::size_of::<EcoString>() == 16`.
The second commit does the same for `ReducerName`. This is also used in
reducer execution.
Together, these commits increase TPS by around 5-7k TPS.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Version upgrade to `v1.12.0`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this is just a version upgrade
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
The testsuite failures are fixed by
https://github.com/clockworklabs/SpacetimeDB/pull/4120
- [x] License has been properly updated including version number and
date
- [x] CI passes
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
- Convert basic-rs template from crates.io deps to workspace deps
(spacetime init will convert them back for users)
- Add basic-rs template to workspace members in root Cargo.toml
- Rename basic-rs package to avoid collision with sdk-test-connect-disconnect
- Simplify default_module_clippy test to run both templates in place
- Add missing st_client table verification to client_disconnected test
Extract static smoketest modules into a nested workspace at
crates/smoketests/modules/ that is pre-compiled during warmup.
This eliminates per-test WASM compilation overhead.
Key changes:
- Add 38 precompiled module crates in nested workspace
- Add module registry (src/modules.rs) for WASM path lookup
- Add precompiled_module() builder and use_precompiled_module() method
- Update xtask warmup to build nested workspace
- Migrate all static tests to use precompiled modules
- Tests using precompiled modules run in ~0.5-3s vs ~4-7s before
Tests that need runtime compilation (auto_migration, detect_wasm_bindgen,
intentionally-broken modules) continue to use module_code().
- Rename tools/xtask to tools/xtask-smoketest
- Add USE_SHARED_TARGET_DIR flag to control caching behavior
- When true: tests share target/smoketest-modules/ and global CARGO_HOME
- When false: each test gets isolated CARGO_HOME (no sharing)
- Shared mode is 1.68x faster (378s vs 636s for all tests)
- Add detailed build timing instrumentation
# Description of Changes
This PR renames the templates to always use shorthand for the language,
specify a framework (or console) if necessary, and shorten the naming in
general
# Expected complexity level and risk
1
# Testing
I've tested generating templates manually
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Create `crates/smoketests/` to translate Python smoketests to Rust:
- Add `Smoketest` struct with builder pattern for test setup
- Implement CLI helpers: `spacetime_cmd()`, `call()`, `sql()`, `logs()`, etc.
- Translate `smoketests/tests/sql.py` → `tests/sql.rs`
- Translate `smoketests/tests/call.py` → `tests/call.rs`
- Reuse `ensure_binaries_built()` from guard crate (now public)
Also fix Windows process cleanup in `SpacetimeDbGuard`:
- Use `taskkill /F /T /PID` to kill entire process tree
- Prevents orphaned `spacetimedb-standalone.exe` processes
# Description of Changes
Client Query builder for rust, as per proposal -
https://github.com/clockworklabs/SpacetimeDBPrivate/pull/2356.
1. Pach moves query builder to its separate crate, so that it can be
shared between module and sdk.
2. Implements `TypedSubscriptionBuilder` in `sdks/rust` as mentioned in
proposal
3. Modify codegen to extend types to support query builder as mentioned
in proposal
4. a test
# API and ABI breaking changes
NA, additive changes.
# Expected complexity level and risk
2
# Testing
Added a test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Small fix for checking to see if a port is available on some given
interface.
updated:
The original implementation here used `bind` to try to discover if a
port is currently in use. This isn't reliable due to platform
differences - especially on windows where it's apparently acceptable to
have a service running on both `0.0.0.0:3000` and `127.0.0.1:3000`. This
would cause bind to return successfully when we wanted it to fail. Also:
binding on an ipv6 interface when a machine doesn't have ipv6 enabled
caused random errors and it was too unreliable to be useful.
This new implementation uses `get_socket_info` which returns info on all
sockets in use on the system. We can then look through this list to find
services which conflict with our requested port.
updated 1/14:
This PR now includes a fix for flaky CLI tests. Originally we were using
`find_free_port` to pick a random free port, but that was causing a race
condition which resulted in test flakes. This PR fixes this issue by
using `127.0.0.1:0` as the listen addr so the kernel will automatically
pick a free port for us.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1 - this is a pretty isolated check, unlikely to introduce larger
issues.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
I tested on macos, windows and linux:
```
ALLOW
docusaurus is already running on 127.0.0.1:3000
SpacetimeDB then tries to start on 192.168.1.10:3000
ALLOW
docusaurus is already running on ::1:3000
SpacetimeDB then tries to start on 192.168.1.10:3000
DENY
docusaurus is already running on ::1:3000
SpacetimeDB then tries to start on 127.0.0.1:3000
DENY
docusaurus is already running on 0:0:0:0:0:0:0:0:3000
SpacetimeDB then tries to start on 0.0.0.0:3000
DENY
docusaurus is already running on 0:0:0:0:0:0:0:0:3000
SpacetimeDB then tries to start on 127.0.0.1:3000
DENY
docusaurus is already running on 0:0:0:0:0:0:0:0:3000
SpacetimeDB then tries to start on 192.168.1.10:3000
DENY
docusaurus is already running on 127.0.0.1:3000
SpacetimeDB then tries to start on 0:0:0:0:0:0:0:0:3000
DENY
docusaurus is already running on 192.168.1.10:3000
SpacetimeDB then tries to start on 0:0:0:0:0:0:0:0:3000
```
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version bump to `v1.11.3` for just the CLI and rust
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CLI version has been updated
- [x] Version + date in the license file has been updated
# Description of Changes
We would like to move all of the templates to a central directory
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
---------
Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade to `1.11.2`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
- None, this is just a version bump
# Expected complexity level and risk
1 - no real changes here
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- NA this is just a version bump, no functionality change here.
# Description of Changes
Introduce a new **LLM benchmarking app** and supporting code.
* **CLI:** `llm` with subcommands `run`, `routes list`, `diff`,
`ci-check`.
* **Runner:** executes globally numbered tasks; filters by `--lang`,
`--categories`, `--tasks`, `--providers`, `--models`.
* **Providers/clients:** route layer (`provider:model`) with HTTP LLM
Vendor clients; env-driven keys/base URLs.
* **Evaluation:** deterministic scorers (hash/equality, JSON
shape/count, light schema/reducer parity) with clear failure messages.
* **Results:** stable JSON schema; single-file HTML viewer to
inspect/filter/export CSV.
* **Build & guards:** build script for compile-time setup;
* **Docs:** `DEVELOP.md` includes `cargo llm …` usage.
This PR is the initial addition of the app and its modules (runner,
config, routes, prompt/segmentation, scorers, schema/types,
defaults/constants/paths/hashing/combine, publishers, spacetime guard,
HTML stats viewer).
### How it works
1. **Pick what to run**
* Choose tasks (`--tasks 0,7,12`), or a language (`--lang rust|csharp`),
or categories (`--categories basics,schema`).
* Optionally limit vendors/models (`--providers …`, `--models …`).
2. **Resolve routes**
* Read env (API keys + base URLs) and build the active set (e.g.,
`openai:gpt-5`).
3. **Build context**
* Start Spacetime
* Publish golden answer modules
* Prepare prompts and send to LLM model
* Attempt to publish LLM module
4. **Execute calls**
* Run the selected tasks within each test against selected models and
languages.
5. **Score outputs**
* Apply deterministic scorers (hash/equality, JSON shape/count, simple
schema/reducer checks).
* Record the score and any short failure reason.
6. **Update results file**
* Write/update the single results JSON with task/route outcomes,
timings, and summaries.
# API and ABI breaking changes
None. New application and modules; no existing public APIs/ABIs altered.
# Expected complexity level and risk
**4/5.** New CLI, routing, evaluation, and artifact format.
* External model APIs may rate-limit/timeout; concurrency tunable via
`LLM_BENCH_CONCURRENCY` / `LLM_BENCH_ROUTE_CONCURRENCY`.
# Testing
I ran the full test matrix and generated results for every task against
every vendor, model, and language (rust + C#). I also tested the CI
check locally using [act](https://github.com/nektos/act).
**Please verify**
* [ ] `llm run --tasks 0,1,2` (explicit `run`)
* [ ] `llm run --lang rust --categories basics` (filters)
* [ ] `llm run --categories basics,schema` (multiple categories)
* [ ] `llm run --lang csharp` (language switch)
* [ ] `llm run --providers openai,anthropic --models "openai:gpt-5
anthropic:claude-sonnet-4-5"` (provider/model limits)
* [ ] `llm run --hash-only` (dry integrity)
* [ ] `llm run --goldens-only` (test goldens only)
* [ ] `llm run --force` (skip hash check)
* [ ] `llm ci-check`
* [ ] Stats viewer loads the JSON; filtering and CSV export work
* [ ] CI works as intended
---------
Signed-off-by: bradleyshep <148254416+bradleyshep@users.noreply.github.com>
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).
Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.
# API and ABI breaking changes
We add a new type
# Expected complexity level and risk
2
it impacts all over the code
# Testing
- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This reverts unnecessary changes to whitespace to prevent conflicts in
old PRs.
If you hide whitespace, you can confirm that this commit is a no-op.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Bump version numbers to `1.11.1`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- [x] Verified that the license has been updated
- [x] `spacetime --version` on this commit is correct
There is also a corresponding private PR.
# Description of Changes
Uses `with_host_stack` to provide a `StackCreator` that pools
`FiberStack`s.
This does not use the pooling instance allocator and is limited to just
stacks.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? Some unsafe and wasmtime internals relied upon.
# Testing
Covered by existing tests.
# Description of Changes
This changes the ci runs to execute `cargo ci` instead of running
commands directly from the github workflow.
The goal here is to unify the commands under `cargo ci` so that it's
easier and more intuitive to run locally
# API and ABI breaking changes
There are no API/ABI changes.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
Complexity: 1
It is not a complex change as it is mostly localized to the ci runs and
is easily reversible if something goes wrong. The biggest risk here is
to have future CI runs break, which can be remediated by reverting these
changes.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] run `cargo ci` and its subcommands locally
- [x] run the github workflow against this branch to check if the CI
jobs are working properly.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Signed-off-by: Roberto Pommella Alegro <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Uses the `sourcemap` crate to map text locations in the bundle to text
locations in the original source code.
# Expected complexity level and risk
1 - essentially only related to diagnostics
# Testing
- [x] Manually tested
- [ ] Add an automated test for backtrace output
# Description of Changes
As discussed; could possibly improve performance/in general it's good to
keep up with patches, it's been over a year since we last bumped this.
# Expected complexity level and risk
2 - large, important dependency but wasmtime is very solid,
well-engineered software.
# Testing
- [x] smoketests run on wasmtime
# Description of Changes
Bumping versions to 1.11.0 in preparation for an upcoming release.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Existing CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This upgrades the SpacetimeDB version to 1.10.0.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
This is just a version bump - not tested.
# Description of Changes
Rust SDK test suite for views
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
This patch only adds tests, it does not change functionality.
# Description of Changes
There were mentions of `hashbrown` in the repo that did not go through
`spacetimedb_data_structures::map`.
This caused compile errors on master when running certain tests locally.
These have been replaced with the proper imports.
The PR also bump hashbrown to 0.16.1 and foldhash to 0.2.0.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
Covered by existing tests.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Upgrade to version 1.9.0.
# API and ABI breaking changes
None - just a version upgrade.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that the license has been updated
- [x] The version number looks correct (1.9.0)
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes#3517 .
With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.
Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.
I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.
This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close#3644 .
Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
rather than running it on the single-threaded database executor.
# API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.
# Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.
# Testing
- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
This PR is a very large change to the workings of the TypeScript SDK and
as such requires a higher bar of testing than other PRs. However, it
does several important things:
1. Unifies the API of the server and client so they not only have the
same API, but they actually implement it with the same TypeScript types.
This fixes several inconsistencies between them and fixes several small
bugs as well.
2. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3365
3. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3431
4. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3435
5. Subsumes the work done in
https://github.com/clockworklabs/SpacetimeDB/pull/3447
6. Derives all type information on the client from a single
`RemoteModule` type which vastly cleans up the correctness of type
checking on the client and helped me to find several small bugs
It accomplishes this by changing code generation of TypeScript on the
client to code generation approximately what a developer would manually
write in their module. The ultimate goal would be to allow the developer
to use the types and functions that they define on in their module
directly on the client without needing to do any code generation at all,
provided they are using TypeScript on the server and client.
https://github.com/clockworklabs/SpacetimeDB/issues/3365 is resolved by
`.build()`ing the `DbConnection` inside a React `useEffect` rather than
doing it directly in line with the render of the provider. In order to
do that we needed to not expose the `DbConnection` directly to
developers by returning a different type from `useSpacetimeDB`.
`useSpacetimeDB` now returns a `ConnectionState` object which is stored
as React state and updates when any of the fields change. This change
also resolves https://github.com/clockworklabs/SpacetimeDB/issues/3431.
https://github.com/clockworklabs/SpacetimeDB/issues/3435 was the issue
that initially lead me down the rabbit hole of unifying the server and
the client because it was nearly impossible to track down all the
various type functions and how they connect to the values that we code
generate on the server. After several hours of attempting this, I
decided to clean up the types a bit to be more uniform.
Implementing the unification between the client and the server also
necessitated fully implemented parts of the API that were fully
implemented on the server, but were broken or missing on the client.
# API and ABI breaking changes
[Unification] -> Means that this is breaking behavior for the client
SDK, but that the new behavior is identical to the server's existing
behavior
## Breaking changes:
- Table accessor names and index accessor names are converted to
camelCase on the `ctx`, so `ctx.db.foo_bar` is now `ctx.db.fooBar`
- [Unification] On the client `my_table.iter()` returns
`IterableIterator` instead of an `Array`
- [Unification] `module_bindings` now export `TypeBuilder`s for all
types instead of a `type MyType` and object `MyType`, so instead of
using `MyType` as a type directly, you need to infer the type `MyType`
-> `Infer<typeof MyType>`.
- [Unification] We no longer generate and export `MyTypeVariants` for
sum types (these are now accessed by `Infer<typeof
MyType.variants.myVariant>`)
- [Unification] `MyType.getTypeScriptAlgebraicType()` has been replaced
with `MyType.algebraicType`
- `useSpacetimeDB()` no longer takes type parameters
- `useTable()` now takes a `TableDef` parameter and type params are
inferred
- `useTable()` now just returns an `Array` directly instead of a object
with `{ rows }`
- [Unification] `ctx.reducers.createPlayer(argA, argB)` ->
`ctx.reducers.createPlayer({ argA, argB })`
- [Unification] `ctx.reducers.onCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.onCreatePlayer(ctx, { argA, argB })`
- [Unification] `ctx.reducers.removeOnCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.removeOnCreatePlayer(ctx, { argA, argB })`
- [Unification] `myTable.count(): number` -> `myTable.count(): bigint`
## Additive changes:
- `Infer<>` now also does `InferTypeOfRow<>` if applicable
- Added a `useReducer()` React hook
- `module_bindings` now exports a `tables` object with references to all
the `TableDef`s
- `module_bindings` now exports a `reducers` object with references to
all the `ReducerDef`s
- Added a new `MyType.create('MyVariant', ...)` function in addition to
the `MyType.MyVariant(...)` constructors (this is private)
## Notable things that did not change:
- `MyType.serialize(writer: BinaryWriter, value: Infer<typeof MyType>)`
and `MyType.deserialize(reader: BinaryReader): Infer<typeof MyType>` are
still supported exactly as before.
- The `MyType.MyVariant(...)` constructor function on sum types is still
present, but implemented with the private `MyType.create('MyVariant',
...)`. We could choose to move away from this API later if we didn't
like the variants polluting the namespace
# Expected complexity level and risk
4 - This is a deep reaching an complex change for the SDK. For the
server, it is much less deep reaching since it reuses much of the same
machinery, although it does require thorough testing there as some of
the code was modified.
This change is fully localized to TypeScript and does not touch the host
(or other languages) at all, and therefore only impacts a beta aspect of
SpacetimeDB.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Added regression test for
https://github.com/clockworklabs/SpacetimeDB/issues/3435
- [x] Manually tested `test-app` and `test-react-router-app`
- [ ] Add test cases for camelCase-ing
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
This makes us go from 3 threads to 2.
The next step is to core pin the V8 worker thread.
# API and ABI breaking changes
None
# Expected complexity level and risk
4
# Testing
Existing tests should cover this.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- This upgrades the versions of all SDKs, the CLI, etc. to 1.8.0
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that all versions seem to be updated including the BSL
license update <!-- maybe a test you want to do -->
We have 1 `1.7.0` that didn't get upgraded automatically because it is
part of the module bindings for a template:
```
crates/cli/.templates/parent_parent_crates_bindings-typescript_examples_quickstart-chat/src/module_bindings/index.ts: cliVersion: '1.7.0',
```
A case could possibly be made for bumping the template but it shouldn't
cause any issues as the module bindings directory should just get
regenerated by the user. @cloutiertyler should we be bumping module
bindings for templates when we upgrade versions?
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
This adds authorization to the relevant API endpoints, updates the CLI
commands and adds smoketests for the teams feature.
**Note**: Authorizing SQL (incl. subscriptions) is a bit more involved,
and submitted as a separate PR in the series.
Depends-on: https://github.com/clockworklabs/SpacetimeDB/pull/3519
# Description of Changes
This commit adds support to the Rust client SDK for calling procedures.
Similar to reducers, each `DbContext` implementor has a `pub procedures:
RemoteProcedures` field, with methods provided by extension traits for
each procedure.
Unlike reducers, the provided methods are invoke and invoke-then.
Invoke-then takes a `FnOnce` callback to run
when the SDK is notified of the procedure's termination status, while
invoke ignores that notification.
No mechanism is provided for observing procedures invoked by other
clients.
Procedure callbacks are implemented by storing a map from `request_id`
to `ProcedureCallback`, with the callback closure internally knowing how
to deserialize the return value. It's mildly unfortunate to deserialize
within the callback instead of on the preprocess background task, but it
saves significant complexity.
This commit also adds a new sdk-test module, `sdk-test-procedures`, and
a new Rust test client, `procedure-client`.
Together, these are used in two tests of invoking and observing
procedures. I've left TODOs for other tests that we should write as we
implement additional procedure features.
I also had to fix a few minor bugs in the Rust codegen which were not
strictly related to procedures: we previously assumed that the sets of
reducers and of tables were non-empty, which led to wonky invalid
codegen on modules which did not define any reducers or which did not
define any tables.
I'm sneaking a change to the Nix flake into this PR as well - when
initially writing it I had included `cargoArtifacts` (the pre-built and
cached dependencies of our actual builds) in its `packages`, but that
was neither necessary or useful, and just made building the shell take a
long time after dependency changes.
# API and ABI breaking changes
Breaks the internal interface between the Rust client SDK and codegen,
so users will have to re-run `spacetime generate`.
# Expected complexity level and risk
2-ish? Pretty simple change to the Rust SDK overall.
# Testing
- [x] Added new automated integration tests exercising the new
functionality.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Bump versions to 1.7.0 in preparation for the release.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
- [x] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This PR implements a new command for the CLI: `spacetime dev`. If run
from outside of a project directory, it will call `init` in interactive
mode to allow user to create a project. If run from a project directory,
it will ask for a module to conect to and it will:
* subscribe to logs
* observe files in the `spacetimedb` directory and automatically publish
to SpacetimeDB
One caveat is that instead of reusing the `logs` code I did some
repetition cause currently the logs code acquires a lock that lives
through await points. It is fixable, but due to limited time I decided
to go with a bit of code duplication. It shouldn't be very hard to fix
later, though.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
This is a draft of the new functionality for `spacetime init`. In order
to run it with built-in templates you have to set the path to the config
file:
```
export SPACETIMEDB_CLI_TEMPLATES_FILE=crates/cli/.init-templates.json
```
In the future it will fetch the list from GH.
A few notes:
* the previous functionality of `spacetime init` does not work at the
moment
* the code needs a bit more cleanup and tests before merging
* there is a bit of a mix in how we generate empty server and client
projects. For Rust we use the existing way of generating. For TypeScript
we clone an empty project from the repo. I wanted to play with both ways
of doing things, and I'm still not sure which is better. Generation in
Rust means that the generated code will match the CLI version and not
necessarily whatever is in Git. On the other hand, for the builtin
templates we will be fetching the newest version from GH, which I guess
might also not what we want, ie. we probably want only stable templates.
More discussion is needed here
* we use `spacetimedb` directory for the server files
* I don't particularly like the inability to disable interactive mode
easily. We discussed disabling it by default if all of the required
arguments are passed, but I don't think it's feature proof. For example,
if someone relies on a non-interactive mode, and we add a new required
argument, instead of printing a message `missing --foo`, we will
automatically launch interactive mode, which is harder to debug. That's
why I think I'd prefer to implement `--non-interactive` argument
* it's kind of hard to keep the legacy behaviour. If you don't pass any
arguments, we go into interactive mode. In the legacy version, we would
print required arguments. If someone passes `--lang` or `--project-path`
explicitly, I guess we could run the legacy workflow, but not sure if
it's worth it, as the command was marked as unstable anyway
* the project path defaults to the project name, but I think we should
probably replace change whitespaces to dashes, or at least ask for the
project path with the project name being the default (or both)
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Remove this custom build profile that was the same as `release`.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This is primarily to get their fixes for the tls negotiation deadloop
that we were experiencing.
[This is the
diff](https://github.com/sunng87/pgwire/compare/v0.32.1...v0.34.1) of
changes between 0.32.1 and 0.34.1.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
2? This was something we discussed wanting to do. It's a change to an
important dependency, but we know it comes with stuff we want.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
n/a - problem not in this repo.
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
Hello team, I'm the original author of pgwire library. In this patch, I
have some updates about spacetimedb's usage of pgwire library:
- Updated pgwire library to 0.34, which includes a critical fix for busy
loop on accepting new connection
- This version will return error on no-op handlers, as requested by
@mamcx
- Update `StartupHandler` for `SslRequest`. `SslRequest` is handled in
`process_socket` automatically before `StartupHandler` kicks in. So it's
safe to remove the match branch. The framework will reject TLS
negotiation automatically if no TLS support compiled
- Updated feature flags. Add `no-default-features` so we will not pull
TLS dependencies.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
Would be nice to verify if you have integration test on postgres
interface.
---------
Co-authored-by: Mario Montoya <mamcx@elmalabarista.com>
# Description of Changes
This PR fixes https://github.com/clockworklabs/SpacetimeDB/issues/3359.
Additionally I have add a new test app to test for this issue and I
changed a perl script to be portable on windows.
The issue was that `useTable` had it's own `isActive` state that it was
incorrectly maintaining, when it should just have been using the
connections `isActive` state directly.
This fixes a critical bug and should be pushed ASAP.
# API and ABI breaking changes
None, fixes a critical bug.
# Expected complexity level and risk
2, quite straightforward fix after figuring it out.
# Testing
- [x] I added a test app and tested the fix in my browser.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Recently, my hacked-together env stopped working, probably due to our
switching linkers. This pushed me to write a proper Nix packaging for
SpacetimeDB, complete with flake and development environment.
Some things I haven't been able to figure out (or in some cases, just
haven't bothered with):
- Unreal.
- C# WASI SDK.
- Exposing Git commit hash for `spacetime --version`.
- Making the Rust SDK's reauth test work. This test tries to write a
file in the home directory, which is inaccessible in the Nix sandbox.
# API and ABI breaking changes
It's a new thing we have to maintain if we're exposing it to users, but
I need to maintain it anyways to be able to develop Spacetime, so...
# Expected complexity level and risk
1
# Testing
- [x] Ran `nix flake check` locally.
- [x] Ran `nix build` locally and then did `spacetime start`, got an
apparently-responsive SpacetimeDB.
- [x] Ran `nix develop` locally, then `cargo build` and `cargo test` in
the dev shell.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Currently based on #3361
Implements most of the TS module API (not yet a function for type
aliases).
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Extremely basic module stuff works
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Bumping versions to 1.6.0 in preparation for upcoming release.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Existing CI only
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Necessary for pulling in rolldown.
# API and ABI breaking changes
None
# Expected complexity level and risk
1, with the caveat that this updates the Rust version and therefore
touches all the code.
# Testing
- [ ] Just the automated testing
# Description of Changes
Pin the `temporal_rs` and `timezone_provider` versions to `0.0.11`,
because future versions such as `0.0.16` are incompatible, but their
version constraints are not correct so we keep getting
auto-rolled-forward to build-breaking versions.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [x] If I `rm Cargo.lock && cargo clippy`, the build still passes
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
In service of adding procedures, which will need to execute WASM code in
an `async` environment so as to suspend execution while making HTTP
requests and whatnot.
Prior to this PR, `JobCores` worked by spawning an OS thread for each
database. These threads would then each be pinned to a specific core,
and in multi-tenant deployments multiple threads could be pinned to the
same core. Now, instead, we spawn one thread per available core at
startup. Each of these threads runs a single-threaded Tokio executor.
Each database is assigned to one of these executors, and runs tasks on
it via `tokio::spawn`.
When we run without core pinning (usually due to having too few hardware
cores), we won't spawn any additional threads or Tokio runtimes at all;
instead we will run database jobs on the "global" Tokio executor. These
jobs may block Tokio worker threads, which might be an issue if a very
core-constrained device runs multiple databases with very long-running
reducers. If this is an issue, we could in this case instead build a
second Tokio runtime only for running database jobs, and let the OS
scheduler figure things out like it did previously.
Previously, we implemented load-balancing among the database cores by
occasionally instructing per-database threads to re-pin themselves. Now,
we instead periodically send the database a new
`wasmtime::runtime::Handle`, which they will `spawn` future jobs into.
Previously, it was possible for a database thread to become canceled,
most likely as a result of `ModuleHost::exit`, after which calls would
fail with `NoSuchModule`. Cancellation is no longer meaningful, as the
database holds a `Handle` to a long-lived `tokio::runtime::Runtime`,
which should always outlive the `ModuleHost`. I have added an
`AtomicBool` flag to `ModuleHost` which is flipped by `exit` and checked
by calls to maintain the previous functionality.
Within this PR, the jobs run on the database-execution Tokio tasks are
not actually asynchronous; they will never yield. This is important
because these jobs may (will) hold a transaction open, and attempting to
swap execution to another job which wants a transaction on the same
database would be undesirable.
Note that this may regress our multi-tenant performance / fairness:
previously, in multi-tenant environments, the OS scheduler would divide
the database cores' time between the per-database threads, potentially
causing one high-load database to be interrupted in the middle of a
reducer in order to run other databases pinned to the same core. Now, a
high-load database will instead run its entire reducer to completion
before any other database gets to run.
We could, in the future, change this by instructing Wasmtime to yield
periodically, either via [epoch
interruption](https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#method.epoch_deadline_async_yield_and_update)
or
[fuel](https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#method.fuel_async_yield_interval),
both of which we're already configuring Wasmtime to track. We'd need (or
at least want) to (re-)introduce a queue s.t. we only attempt to run one
job for each database at a time. I have chosen not to do so within this
patch because I felt the changeset was complex enough already, and we
have so far not treated fairness in multi-tenant environments as a high
priority.
I have also reworked our module host machinery to no longer use dynamic
dispatch and trait polymorphism to manage modules and their instances,
and instead introduced `enum Module` and `enum Instance`, each of which
has a variant for Wasm and another for V8.
During this rewrite, I reworked `AutoReplacingModuleInstance`, which
previously used type-erased trait generics in a way that was brittle and
hard to re-use in the new `async` context. (Specifically, the module
instance no longer lives on the job thread, rather, the database grabs
the instance and sends it to the job thread, then gets it back when the
job exits. This is necessary to allow the re-worked load balancing
described above, as we can't have a single long-lived async task.) While
refactoring, I replaced it with `ModuleInstanceManager`, which can now
maintain multiple instances of the same module. This is not yet useful,
but will become necessary with procedures, as each concurrent procedure
will need its own instance. Relatedly, I changed
`ModuleHost::on_module_thread` (used by one-off and initial subscription
queries) to no longer acquire the/an instance. I discussed this with the
team, and consensus was that "locking" the module instance in that path
was not a useful behavior, just an artifact of the previous
implementation.
I have also switched our Wasmtime configuration to set
`async_support(true)`. This causes a variety of methods, notably
`InstancePre::instantiate` and `TypedFunc::call`, to panic, and requires
that we instead call their `_async` variants. As mentioned above, I have
not yet introduced any actual asynchronicity or concurrency, so these
methods should never yield. Rather than `.await`ing their futures, I
have defined a degenerate `async` executor, `poll_once_executor`, which
polls a future exactly once, failing if it does not return
`Poll::Ready`. This means that we will panic if one of these futures
returns `Poll::Pending` unexpectedly.
The previous `trait Module` had a method `initial_instances`. `Module`
is now a concrete type, and I gave it this method, but it appears to be
unused. This is causing lints to fail. I am unsure what, if anything,
that method was for.
The previous `AutoReplacingModuleInstance` called `create_instance` on
the job thread. I am unsure if this was intentional, or just an artifact
of the previous implementation, where the `AutoReplacingModuleInstance`
lived on the job thread. I have written the new `ModuleInstanceManager`
to call `create_instance` on the calling thread, but it would be easy to
move that call into the job executor if that behavior is desired.
# API and ABI breaking changes
None user-facing
# Expected complexity level and risk
4: significant rewrite of performance-sensitive fiddly concurrency code.
Note specifically in above description:
- Running database jobs on the global Tokio runtime when not using core
pinning.
- Multi-tenant fairness issue: no longer possible to interrupt a
performance-intensive database mid-reducer to run another database
pinned to the same core.
- Unused method `module_instances`.
- Running `create_instance` on the calling thread rather than the
database thread.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Will arrange for a bot test.
- [ ] Determine to what extent we can run with real or synthetic
multi-tenant load in a test or staging environment.
---------
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
It turns out that cargo automatically uses the latest semver-compatible
versions of dependencies, which is not what we expected. tl;dr
specifying `1.5.0` actually means `>=1.5.0 <2.0.0`, but we actually
intend `1.5.*`.
This PR updates our `upgrade-version` tool, and re-runs it to fix the
dep versions.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] I ran `cargo bump-versions 1.5.0 --rust-and-cli` to regenerate the
other committed files.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Zeke Foppa