# Description of Changes
Closes#5250#4636 introduced a regression where the ProcedureContext used to include
the sender and connection_id from the caller while now it is always
empty (which is wrong)
Correct it.
Also fully migrate to `database_identity` which was forgotten about so i
deprecated it for procedures (since they are now stable) and just
changed it for HttpHandlers (because they are still unstable)
@gefjon since you did the lil woopsie (ugh pinging you again lol hope im
not annoying haha)
# API and ABI breaking changes
Breaking the HttpHandler function which is unstable.
Restoring behaviour of 2.3 which got lost with 2.4.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1. Trivial refactoring
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] The caller identity is there again for Procedures.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Graduate **procedures** (and the outgoing HTTP client used from
procedures) out of the `unstable` feature gate /
`SPACETIMEDB_UNSTABLE_FEATURES` across all module libraries. The
`procedure` macro, `ProcedureContext`, `with_tx`/`try_with_tx`,
scheduled procedures, and `ctx.http` are now available without opting
into `unstable`.
**Still gated** (unchanged): HTTP handlers/webhooks, views, RLS /
`client_visibility_filter`, and immediate-scheduling
(`volatile_nonatomic_schedule_immediate`).
Per library:
- **`bindings-sys`**: ungate the `procedure` host-call module + raw ABI
imports (`procedure_start/commit/abort_mut_tx`,
`procedure_http_request`) and `call_no_ret`. Scheduling ABI stays gated.
- **`bindings` (Rust)**: ungate the `procedure` macro,
`ProcedureContext`, `TxContext`/`with_tx`/`try_with_tx`,
`db_read_only`/`get_read_only`, the procedure traits,
`register_procedure`, `__call_procedure__`, and procedure RNG. The
`http` module (previously gated as a unit) now has fine-grained gating
so the outgoing `HttpClient` is exposed while
`HandlerContext`/`Router`/handler macros stay gated.
- **`bindings-csharp`**: drop `[Experimental("STDB_UNSTABLE")]` from
`ProcedureContext.WithTx/TryWithTx` (runtime + codegen) and the
generated `ProcedureTxContext`; FFI snapshots regenerated. Handler
context members + RLS attribute stay gated.
- **`bindings-cpp`**: ungate the procedure ABI (`abi.h`/`FFI.h`),
`tx_execution.h`, the outgoing HTTP client (`http.h`/`http_convert.h`),
and `procedure_context.h`.
`handler_context.h`/`router.h`/`http_handler_macros.h` still `#error`
without `SPACETIMEDB_UNSTABLE_FEATURES`.
- **docs**: remove the procedures beta notices (HTTP handlers stay
marked beta for a later release); regenerate `static/llms.md`.
- **`sdk-test-procedure`**: no longer needs `features = ["unstable"]`.
# API and ABI breaking changes
Not breaking. This is purely additive to the stable surface: procedures
become available **without** the `unstable` feature, while modules that
already opt into `unstable` are unaffected. The wasm host-ABI imports
were already implemented host-side and merely gated module-side, so
there is no ABI version change. (No breaking-change label needed.)
# Expected complexity level and risk
**3/5.** The diff itself is mostly removing
`#[cfg]`/`#ifdef`/`[Experimental]` guards, but the gate bundled
procedures + outgoing HTTP + handlers + the ABI crate together, so the
work was in *separating* procedures from the features that stay gated.
Areas reviewers may want to scrutinize:
- The Rust `http` module went from "gated as a whole" to fine-grained
per-item gating; the outgoing `HttpClient` is exposed while handler
types/macros stay behind `unstable`.
- The `unstable` cut reaches the ABI crate (`bindings-sys`), which is
the only way procedures can compile without the feature.
- C++ separation of the outgoing HTTP client from HTTP handlers across
several headers.
# Testing
- [x] Rust: `cargo check -p spacetimedb` passes in default, `--features
unstable`, and `--no-default-features --features unstable`.
- [x] Rust end-to-end: `sdk-test-procedure` (uses procedures,
`with_tx`/`try_with_tx`, `ctx.http.get`, `new_uuid_v7`, scheduled
procedures) builds **without** `unstable`.
- [x] C#: Codegen + Runtime build; `Codegen.Tests` pass (6/6) after FFI
snapshot regen.
- [x] C++: host syntax-check confirms procedures + `ctx.http.send()`
compile **without** the flag, full headers compile **with** it, and
`handler_context.h` still `#error`s without it. (Not built for the real
wasm/emscripten target locally — relying on CI.)
- [x] `cargo ci lint` components reproduced locally: rustfmt, clippy
(`-D warnings`, default + `unstable`), csharpier, and `cargo doc --deny
warnings`.
- [ ] Reviewer: confirm the wasm bindings + C#/C++ test suites pass in
CI (especially the C++ wasm build, which couldn't run locally).
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Adds support for primary keys to procedural views in rust and typescript
modules. Now clients can receive update events when subscribed to such
views.
```rust
#[spacetimedb::view(accessor = my_players, public, primary_key = id)]
pub fn my_players(ctx: &spacetimedb::ViewContext) -> Vec<Player> {
ctx.db.players().owner().filter(ctx.sender()).collect()
}
```
```ts
const Player = t.row('Player', {
id: t.u64().primaryKey(),
owner: t.identity().index('btree'),
name: t.string(),
});
export const my_players = spacetimedb.view(
{ public: true },
t.array(players.rowType),
ctx => Array.from(ctx.db.players.owner.filter(ctx.sender))
);
```
In rust, a view's primary key is declared within the `#[view]` macro,
whereas in typescript it is declared at the column/row level as it is
for tables. I could have made it a view level attribute for typescript
as well, but since primary keys are already row-level attributes in
typescript, I just kept that as is.
Note, care must be taken to ensure that the view never returns duplicate
primary keys, or else it will fail which will currently result in the
transaction that triggered the view refresh to be rolled back. Better
error handling for this exact scenario will be added in a separate
follow up patch.
## Alternative Considered: #[primary_key] attribute on `SpacetimeType`s
in Rust
This would be equivalent to what we do in typescript.
# API and ABI breaking changes
None, although adding a primary key to an existing view will require a
client update.
# Expected complexity level and risk
3
# Testing
Compile-time failure scenarios:
- [x] Primary key on non-existent column
- [x] Primary key doesn't reference custom accessor name (rust)
- [x] Multiple primary key columns specified (typescript)
- [x] Primary keys must be `FilterableValue`s
SDK tests (rust sdk, rust and ts modules):
- [x] `OnUpdate`
- [x] `OnUpdate` is sender-scoped
- [X] Can join on primary key columns
Smoketests:
- [x] Modifying primary key columns breaks clients (auto-migrations)
# Description of Changes
Part of https://github.com/clockworklabs/SpacetimeDB/issues/4970.
Doing this part in a separate PR because it turns out that this test was
failing, so this PR also includes the fixes.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
CI passes (but only with the "code" changes).
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
## Summary
- repoint selected BSL crate `LICENSE` symlinks to
`licenses/apache2.txt`
- add the missing Unreal SDK `LICENSE` symlink to `licenses/apache2.txt`
Requested crates/SDK:
- bindings-macro
- bindings-sys
- bindings
- lib
- primitives
- query-builder
- sats
- unreal SDK
Note: `query-builder` was already pointing at `licenses/apache2.txt`, so
it has no diff.
## Testing
- Verified all requested `LICENSE` paths resolve to
`../../licenses/apache2.txt`
- `cargo run -p check-license-symlinks` does not complete cleanly in
this checkout because it also scans unrelated existing copied template
files and `node_modules` license files that are not symlinks
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Adds support to Rust modules and the SpacetimeDB host for defining HTTP
handlers and registering them to routes.
## User-facing API
In a Rust module, users can annotate functions with the new macro
`#[spacetimedb::http::handler]`. A function annotated this way must
accept exactly two arguments, of types `&mut
spacetimedb::http::HandlerContext` and `spacetimedb::http::Request`
(which is a type alias for `http::Request<spacetimedb::http::Body>`. It
must also return `spacetimedb::http::Response` (which is a type alias
for `http::Response<spacetimedb::http::Body>`).
Once the user has defined an HTTP handler, they can register it to a
route by annotating a function with `#[spacetimedb::http::router]`. Such
a function must take no arguments and return a
`spacetimedb::http::Router`. (The original design put this annotation on
a `static` variable rather than a function, but that turned out to be
undesirable because it required that constructing a `Router` be
`const`.) `Router` exposes various methods for registering handlers to
routes.
All of a database's user-defined routes are exposed under
`/v1/database/:name_or_identity/route/{*path}`.
## Example
See [the new
smoketest](https://github.com/clockworklabs/SpacetimeDB/blob/phoebe/http-handlers-webhooks/crates/smoketests/tests/smoketests/http_routes.rs)
for a more exhaustive example.
A simpler example, which stores arbitrary byte data in a table via a
`POST` request, returns an ID, and then retrieves that same data via a
`GET` request with a query parameter:
```rust
#[spacetimedb::table(accessor = data)]
struct Data {
#[primary_key]
#[auto_inc]
id: u64,
body: Vec<u8>,
}
#[spacetimedb::http::handler]
fn insert(ctx: &mut HandlerContext, request: Request) -> Response {
let body: Vec<u8> = request.into_body().into_bytes().into();
let id = ctx.with_tx(|tx| tx.db.data().insert(Data { id: 0, body: body.clone() }).id);
Response::new(Body::from_bytes(format!("{id}")))
}
#[spacetimedb::http::handler]
fn retrieve(ctx: &mut HandlerContext, request: Request) -> Response {
let id = request
.uri()
.query()
.and_then(|query| query.strip_prefix("id="))
.and_then(|id| u64::from_str(id).ok())
.unwrap();
let body = ctx.with_tx(|tx| tx.db.data().id().find(id).map(|data| data.body));
if let Some(body) = body {
Response::new(Body::from_bytes(body))
} else {
Response::builder().status(404).body(Body::empty()).unwrap()
}
}
#[spacetimedb::http::router]
fn router() -> Router {
Router::new().post("/insert", insert).get("/retrieve", retrieve)
}
```
## Design and implementation notes
- As mentioned above, the router is registered via a function, not a
`static` or `const` item. This is because `static` or `const`
initializers must be `const`, and it turns out to be a pain to make all
of the `Router` constructors be `const fn`s.
- The `#[handler]` macro clobbers the original function name with a
`const` variable of type `HttpHandler`. This is unfortunate, but AFAICT
necessary, 'cause we need to pass the string identifier for the handler
to the `Router`, not the function pointer, and Rust allows no (stable
and reliable) way to get a unique string identifier out of a function
item/value, nor to attach data or implement traits for function
items/values. The alternative(s) would involve changing the signature of
the `Router` methods to have uglier and more complex callsites, e.g.
like `.get("/retrieve", retrieve::handler())`, `.get("/retrieve",
handler!(retrieve))` or `.get::<retrieve>("/retrieve")`. I believe that
registering handlers will be much more common than calling their
functions, so I've chosen to make it so that registering them gets the
convenient syntax, even though the inability to call them directly will
be somewhat surprising.
- I haven't wired up energy handling or timing metrics for handler
execution to anywhere. Procedures are still in the same boat.
- HTTP requests to user-defined routes bypass the usual SpacetimeDB auth
middleware, meaning that the host does not validate (or inspect in any
way) `Authorization` headers in requests before invoking the
user-defined handler. This is required to allow arbitrary
user-programmable handling of `Authorization` headers, including those
in formations which SpacetimeDB would reject. As a result of this,
`HandlerContext` doesn't expose a `sender` or `sender_connection_id`.
- HTTP route paths may consist only of a very restrictive set of
characters. I've chosen this set to keep our options open in the future
to add additional syntax to routes, like for registering wildcard
segments and path parameters:
- ASCII digits.
- ASCII letters.
- `-_~/`.
- The internal data structure that represents a `Router` is currently a
`Vec<Route>`, meaning that resolving a request to a route is
`O(num_routes)`. Registering a route checks against each previous route
for uniqueness, meaning that constructing a router is `O(num_routes ^
2)`. There are TODO comments to use a trie, but I think this can wait,
as I expect most databases to register few routes.
- Commit 999a7c317 contains a fix to a mostly-unrelated bug where a few
bindings introduced by the SATS derive macros were unhygienic and not in
a reserved namespace, leading to name conflicts. I discovered this
'cause I tried writing an HTTP handler named `index` to serve the
index/root of a website and it broke.
## Still TODO
- [x] Resolve various TODO comments in the diff.
- [x] Documentation.
- [x] C# bindings support.
- [x] C++ bindings support.
- [x] V8 host support.
- [x] TypeScript bindings support.
# API and ABI breaking changes
New APIs, currently flagged as `unstable`, which will eventually need
stability guarantees. No (intentional) breaking changes, or changes to
existing APIs at all.
# Expected complexity level and risk
3? Changes to our HTTP routing to support the user-defined routes.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] New smoketest of the behavior!
- [x] I dunno, maybe try hosting a simple webpage and see how it works?
- [x] Build a test app with Stripe integration.
- @aasoni did this.
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Closes#4989
We were not using `arbitrary_precision` in the workspace: I only found
the workspace dependency declaration itself plus a normal
`serde_json::Number::from(3u8)` call in
`crates/cli/src/spacetime_config.rs`, which does not require the
feature.
## Summary
- remove `serde_json/arbitrary_precision` from the workspace dependency
declaration
- keep `raw_value`, which is the only serde_json feature the workspace
is clearly using here
- avoid forcing a non-additive serde_json feature onto downstream
consumers via feature unification
## Validation
- `cargo tree -e features -p serde_json`
- `cargo check -p spacetimedb-sdk -p spacetimedb-lib -p spacetimedb-sats
-p spacetimedb-schema`
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
## Summary
- add `ReducerContext::database_identity()` in Rust bindings
- deprecate `ReducerContext::identity()` and keep it as a compatibility
alias
- update reducer docs example to use `ctx.database_identity()`
- add C# reducer-context equivalent: `DatabaseIdentity` and obsolete
`Identity` alias
- update Rust/C# module test callsites to the new API name
- update C# codegen snapshots for generated `ReducerContext` API output
## Why
Issue #3201 reports user confusion between reducer `ctx.identity()`
(database/module identity) and `ctx.sender`. This change clarifies
naming while preserving compatibility.
## Validation
- `cargo check -p spacetimedb -p module-test` (Passed)
- `dotnet test crates/bindings-csharp/Codegen.Tests/Codegen.Tests.csproj
--nologo` (Passed)
- `dotnet test crates/bindings-csharp/Runtime.Tests/Runtime.Tests.csproj
--nologo` (Failed) pre-existing unrelated failure:
- `Runtime.Tests/JwtClaimsTest.cs(10,23): CS1729: 'JwtClaims' does not
contain a constructor that takes 2 arguments`
## Compatibility
- Rust: `identity()` still works but is deprecated in favor of
`database_identity()`.
- C#: `Identity` still works but is marked `[Obsolete]` in favor of
`DatabaseIdentity`.
Closes#3201
# Description of Changes
This patch aims to improve the situation for the need of a generic
function over multiple `ctxs`.
The `DbContext` was introduced for this but due to the associated type
amibguity when trying to implement it another mmethod is needed.
This pr implements a `db_read_only()` method which always results in a
`LocalReadOnly` hence Rust can infer the types of a the returned type
(which prior to this was either `__view , __query and a 3rd one i cent
remember right now 😓`.
I have chosen to be defensive by implementing it as `unstable` but i can
also remove this if that is desired.
It allows the following pattern which has been workig great in my
project so its time to contribute it :>
```rust
pub(crate) trait YourTableNameRead {
// Your methods which only need read access
fn test(&self,args);
}
// The read version is a supertrait to give access to the read methods.
pub(crate) trait YourTableNameWrite: YourTableNameRead {
// Your methods which need read-write access
}
// The read version gets implemented for every DbContext since we can always read.
impl<Db: DbContext> YourTableNameRead for Db {
fn test(&self,args) {
self.db_read_only().table_name().whatever(args);
}
// By constraining the associated type to Local we only get this for writeable ctxs.
impl<Db: DbContext<DbView = Local>> YourTableNameWrite for Db {}
```
These allow you to do on the calling site:
```rust
use YourTableNameRead;
#[view|reducer|procedure]
fn my_func(ctx: Reducer|(Anon)View|Tx, args ) {
ctx.test(args);
}
```
# API and ABI breaking changes
None
# Expected complexity level and risk
1. Minor api and qol adition which is furthermore unstable.
# Testing
- [x] Works in my project
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Make Timestamp a FilterableValue in Rust, C#, and Typescript. I'm not
sure this is changing all the places because we have the server and the
client in those 3 languages.
# API and ABI breaking changes
It's an additive change.
# Expected complexity level and risk
3. There are some designs decisions, like comparing timestamp to
strings/numbers.
# Testing
Added unit tests for the 3 languages.
# Description of Changes
Revert the following PRs that have caused some breakage:
```
a32cffa76 Finish refactoring out replay (#4850)
d639be0af Replay: some code motion & reuse `ReplayCommittedState` (#4849)
78d6b6f7d Update NativeAOT-LLVM infrastructure to current ABI (#4515)
d5c1738c1 Better module backtraces for panics and whatnot (#577)
6f23b19f3 Wait for database update to become durable (#4846)
81c9eab86 Add `spacetime lock/unlock` to prevent accidental database deletion (#4502)
809aebd7c Move field `replay_table_updated` to `ReplayCommittedState` (#4807)
21b58ef99 Update axum (#2713)
b5cadff7a Extract replay stuff out of `CommittedState`, part 1 (#4804)
```
I also updated the Python smoketests for breakage introduced in
https://github.com/clockworklabs/SpacetimeDB/pull/4502. Reverting that
PR caused conflicts, so this fix is more straightforward.
# API and ABI breaking changes
Maybe kind of, but we haven't released any of these.
# Expected complexity level and risk
1
# Testing
Ask @bfops about testing
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Support is added to Rust, C#, C++, and TS modules.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
I wasn't able to regen module bindings for sdk tests, hitting an error
while doing so.
---------
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Adds `Deserialize::validate` and all the friends necessary to make that
work.
The point of this is to enable allocation-less validation of input in
some data format, in particular BSATN, against some expected type, in
particular an `AlgebraicValue` at an `AlgebraicType`.
This is extracted from
https://github.com/clockworklabs/SpacetimeDB/pull/4311 and is used there
to convert raw BSATN to `BytesKey` without constructing the composite
`AlgebraicValue` just for validation only to throw it away immediately.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
TODO: tweak existing SATS roundtrip tests to also check validation.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Title.
Is useful to more easily reuse code.
I asked @joshua-spacetime on discord if this would be reasonable and he
said 👍 .
This mirrors the method available on `ReducerContext` for consistency.
# API and ABI breaking changes
None
# Expected complexity level and risk
1 - minimal api surface addition
# Testing
- [x] Tested in my project and works as expected.
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Adds `count` to rust's procedural view API.
This API already existed in C# and typescript. This patch just adds
tests for those module languages.
We were already tracking reads for `count`, and so all this patch does
is add the new rust module bindings.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Smoketests for rust, C#, and typescript.
# Description of Changes
Query builder views return a subset of rows from a physical table. If
that table has a primary key, then so should the view. What this means
concretely is that the view should expose the same api as the table,
specifically as it relates to the primary key column.
With that in mind, this patch commits the following changes:
1. Annotates `ViewDef` with a `primary_key`
2. Updates the return type of query builder views in the raw module def
to a special product type
3. Adds an index for the primary key on the view's backing table
4. Updates the query planner to use this index
5. Updates rust client codegen to generate `on_update` for such views
# API and ABI breaking changes
None
Old `impl Query` views compiled with an older version of SpacetimeDB
will continue to work as they did before - without a primary key.
# Expected complexity level and risk
3
# Testing
- [x] New rust sdk integration suite exercising `on_update` for PK views
and semijoin scenarios
- [x] Smoketests for PK views and semijoin scenarios
# Description of Changes
`cargo ci lint` was running `cargo fmt`, but that didn't pick up all
files. As a result, our pre-commit hook (which just runs `rustfmt` on
any changed `.rs` files) would change otherwise-untouched files in merge
commits.
This PR addresses that discrepancy by having `cargo ci lint` run
`rustfmt` on all tracked `.rs` files.
The entire diff is just `rustfmt` changes except for the changes in
`tools/ci/src/main.rs`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
2
# Testing
- [x] `cargo ci lint` fails
- [x] `cargo ci lint` passes after running `rustfmt` on everything
- [x] `cargo fmt --all` doesn't cause any diff after doing the above
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
## Summary
- `create_bytes_source()` in the host returns `BytesSourceId::INVALID`
for empty response bodies, but `HttpClient::send()` unconditionally
reads the body source via `read_bytes_source_into()`, which panics with
`"invalid source passed"`
- Any procedure HTTP request returning an empty body (e.g. HTTP 200 with
no content from a successful reducer call) triggers this panic
- Fix: check for `BytesSource::INVALID` before reading, matching the
existing pattern used for JWT payloads in `bindings-sys`
## Test plan
- [x] Verified `cargo check -p spacetimedb` passes
- [x] Trigger an HTTP request from a procedure that returns an empty
response body (e.g. calling a reducer via HTTP that returns 200 with no
body)
- [x] Verify no panic occurs and an empty `Body` is returned
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Adds a cross-language `#[spacetimedb::settings]` mechanism for declaring
module-level settings. The first supported setting is
`CASE_CONVERSION_POLICY`, which controls how identifiers are
case-converted.
### Rust
```rust
use spacetimedb::CaseConversionPolicy;
#[spacetimedb::settings]
const CASE_CONVERSION_POLICY: CaseConversionPolicy = CaseConversionPolicy::None;
```
- Proc macro validates const name against `KNOWN_SETTINGS`
- Generates `__preinit__` export for registration
- Duplicate settings produce a linker error (duplicate symbol)
### TypeScript
```ts
export const moduleSettings: ModuleSettings = {
caseConversionPolicy: CaseConversionPolicy.None,
};
```
- `ModuleSettings` interface with typed fields
- `caseConversionPolicy` section pushed in `rawModuleDefV10()`
## Stacked on
- `shub/case-conversion` (base)
- PR #4364 (internal opt-out API)
This PR adds the proper user-facing API on top of the internal mechanism
in #4364.
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Update the Default casing policy to `snake_case` for `RawModuleDefV10`.
Messy PR contains changes at different places, so that CI can pass:
Here are the main changes as follows:
- `bindings-macro` & `bindings` crate: `name` macro in Indexes for
canonical name and supply it to `RawModuleDefV10` via `ExplicitNames`.
- `bindings-typescript`:
- Changes has been reviewed through this PR -
https://github.com/clockworklabs/SpacetimeDB/pull/4308.
- `binding-csharp`: a single line change to pass `sourceName` of index
instead of null.
- `codegen`:
- Changes has been merged from branch -
https://github.com/clockworklabs/SpacetimeDB/pull/4337.
- Except a fix in rust codegen to use canonical name in Query buillder
instead of accessor.
- `lib/db/raw_def`: Extends `RawDefModuleV10` structure to support case
conversion.
- `schema` crate:
- `validate/v9` - Nothing itself should change or changes in v9
validation logic but the file contains a `CoreValidator` which is shared
with `validate/v10`. No test have t be updated to `validate/v9` which
ensures we aren't regressing it.
- `validate/v10`: This is the main meat, look at the new tests added in
bottom to understand what it does.
- Rest of the files are either test updates or module bindings.
## Testing:
1. Extensive unit tests have been added to verify generated `ModuleDef`
is correct.
2. I have done some e2e testing to verify rust codegen with rust and
typescript modules.
3. I would have like to do more testing for other codegens , I am
continue doing .
I have removed `sql.py` smoketest, as that seems to be already migated
in new framework and was headache to update.
## Expected complexity level and risk
4, It could have side-effect which aren't easily visible.
- - -
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
- Updated all markdown files that call out `#[spacetimedb::table(name`
to `#[spacetimedb::table(accessor`
- Updated the LLM details on the same change
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Small rework in documentation to use the new
# Testing
- [x] Did a small double check on name vs accessor locally
- [x] Codex tested all the LLM answers
## Description of Changes
This PR primarily affects the `bindings-macro` and `schema` crates to
review:
### Core changes
1. Replaces the `name` macro with `accessor` for **Tables, Views,
Procedures, and Reducers** in Rust modules.
2. Extends `RawModuleDefV10` with a new section for:
* case conversion policies
* explicit names
New sections are not validated in this PR so not functional.
3. Updates index behavior:
* Index names are now always **system-generated** for clients. Which
will be fixed in follow-up PR when we start validating RawModuleDef with
explicit names.
* The `accessor` name for an index is used only inside the module.
## Breaking changes (API/ABI)
1. **Rust modules**
* The `name` macro must be replaced with `accessor`.
2. **Client bindings (all languages)**
* Index names are now system-generated instead of using explicitly
provided names.
**Complexity:** 3
A follow-up PR will reintroduce explicit names with support for case
conversion.
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
# Description of Changes
Standardizes the query builder API across all three language SDKs (Rust,
TypeScript, C#) for consistency.
**Rust:**
- Rename `Query` struct to `RawQuery`, make `Query` a trait with `fn
into_sql(self) -> String`
- All builder types (`Table`, `FromWhere`, `LeftSemiJoin`,
`RightSemiJoin`) implement `Query<T>` trait
- Views can return `-> impl Query<T>` instead of specifying exact
builder types
- The `#[view]` macro auto-detects `impl Query<T>` and rewrites to
`RawQuery<T>`
- Add `Not` variant to `BoolExpr` with `.not()` method
**TypeScript:**
- Add `ne()` to `ColumnExpression`
- Refactor `BooleanExpr` to `BoolExpr` class with chainable `.and()`,
`.or()`, `.not()` methods
- Make builders valid queries directly (`.build()` deprecated but still
works)
- Deprecate `from()` wrapper — use `tables.person.where(...)` directly
- Merge `query` export into `tables` so table refs are also query
builders
- Add subscription callback form: `subscribe(ctx =>
ctx.from.person.where(...))`
- Unify `useTable` with query builder syntax; deprecate `filter.ts`
**C#:**
- Add `Not()` method to `BoolExpr<TRow>`
- Add `IQuery<TRow>` interface implemented by all builder types
(`Table`, `FromWhere`, `LeftSemiJoin`, `RightSemiJoin`, `Query`)
- Add `ToSql()` to all builder types so `.Build()` is no longer required
- Update `AddQuery` to accept `IQuery<TRow>` instead of `Query<TRow>`
# API and ABI breaking changes
- Rust: `Query<T>` is now a trait (was a struct). The struct is renamed
to `RawQuery<T>`. This is a breaking change for any code that used
`Query<T>` as a type directly.
- TypeScript: `BooleanExpr` is now a `BoolExpr` class (was a
discriminated union type). The `query` export is deprecated in favor of
`tables`.
- C#: `AddQuery` now accepts `Func<QueryBuilder, IQuery<TRow>>` instead
of `Func<QueryBuilder, Query<TRow>>`. Existing `.Build()` calls still
work since `Query<TRow>` implements `IQuery<TRow>`.
# Expected complexity level and risk
3 — Changes touch multiple language SDKs and codegen, but each
individual change is straightforward. The Rust macro rewrite for `impl
Query<T>` detection is the most complex piece. All existing
`.build()`/`.Build()` calls continue to work.
# Testing
- [x] `cargo test -p spacetimedb-query-builder` — 16/16 tests pass
- [x] `cargo check -p spacetimedb` — clean, no warnings
- [x] `cargo check` on views-query, views-sql, views-basic,
views-trapped smoketest modules — all clean
- [x] `cargo test -p spacetimedb-codegen codegen_csharp` — snapshot
updated, passes
- [x] `npm test` (TypeScript) — 101/101 tests pass
- [x] C# QueryBuilder tests — new tests for `Not()`, `IQuery<T>`
interface
- [ ] CI passes
## Summary
- Restricts `update()` across all three server-side SDKs (Rust, C#,
TypeScript) to only work on primary key columns, not all unique columns
- Calling `update()` on a non-PK unique column is semantically a
delete+insert — clients won't see it as a row update unless the primary
key stays the same
- Fresh re-implementation of #1862, extended to cover C# and TypeScript
bindings
### Design principle
Primary key is a constraint on columns, not a property of indexes. An
index is unique or not unique — whether `update()` is available is
derived from the column's attributes at the point of use, not stored on
the index.
This differs from #1862 which introduced a `Uniqueness` enum (`No |
Unique | PrimaryKey`) on the index itself. Instead, we keep `is_unique:
bool` and check the column metadata where needed.
### Rust changes
- Add `PrimaryKey` marker trait and `where Col: PrimaryKey` bound on
`UniqueColumn::update()`
- `marker_type()` receives `primary_key_column` and checks the column
directly — no `is_pk` field on the index
- `sdk-test` unique tables use `update_non_pk_by` (delete+insert)
instead of `update_by`
- Benchmarks that used `update()` on `#[unique]` columns changed to
either `#[primary_key]` or delete+insert
### C# changes
- Only emit `Update()` on `UniqueIndex` when the column has `PrimaryKey`
attrs
- Update `unique_*` test reducers in `sdk-test-cs` to use
`Delete()`+`Insert()` instead
### TypeScript changes
- `UniqueIndex` now has only `find` + `delete` (no `update`)
- `AllColumnsPrimaryKey` type-level helper checks column metadata from
the `TableDef`
- `Index` routing conditionally intersects `{ update() }` when columns
are PK
- Runtime only attaches `update` method for PK unique indexes
- `sdk-test-ts` unique tables use delete+insert instead of update
- Type-level test in `schema.test-d.ts` verifies `id.update()` compiles
(PK) and `name2.update` errors (non-PK unique)
## Test plan
- [x] C# codegen tests pass (`dotnet test
crates/bindings-csharp/Codegen.Tests`)
- [x] Rust SDK test module compiles
- [x] Benchmarks compile
- [x] TypeScript type checks pass (`npx tsc --noEmit`)
- [ ] CI passes
Closes#1862
This reverts commit fd3cdb3d0b.
# Description of Changes
Reverts #4105, We do not actually need `__describe_module_v10__`, as
original describe reducer returns `enum` - `crates:lib::RawModuleDef`
which is extended to add a variant for `V10`
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
# Testing
Existing tests should be enough
# Description of Changes
Switch Wasm module to `__describe_module_v10__` from
`__describe_module__` to expose `RawModuleDefV10`.
Note: Originally merged into the 2.0 breaking changes branch
https://github.com/clockworklabs/SpacetimeDB/pull/4115.
# API and ABI breaking changes
2.0 breaking change.
# Expected complexity level and risk
1
# Testing
Exising smoketests should be enough.
---------
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
Required in order to bump rolldown to its latest version, which I'm
planning to do afterwards. I also figure we may as well do this before
releasing a major version.
# Expected complexity level and risk
2
# Testing
n/a
# Description of Changes
Same changes as https://github.com/clockworklabs/SpacetimeDB/pull/4101
but for `connection_id`.
# API and ABI breaking changes
API Breaking
# Expected complexity level and risk
1
# Testing
Pure refactor. Tests and docs have been updated, but no new tests added.
# Description of Changes
Makes the `sender` field on `ViewContext` private and exposes a
`sender()` method. Does the same for `ReducerContext` and
`ProcedureContext`.
The purpose of this change: So that the host can determine if/when a
view invokes or reads the `sender`.
Currently, because `sender` is a field, the host assumes that it is
always read. This means views must be materialized per client, even if
the view doesn't actually depend on `sender`, resulting in data
duplication.
The initial solution for this problem was `AnonymousViewContext` which
doesn't have a `sender` field. The better solution is to make `sender` a
method so that it can call into the host and record when it's actually
invoked.
Note, this patch only updates the module API, so the current
implementation does not change. `ViewContext` views are still duplicated
across clients. Changing this requires a new host syscall and for
`sender()` to invoke that syscall. This however is backwards compatible
and can be done anytime after the module APIs for the other languages
(C#, TypeScript, C++) are updated.
Also note that `ReducerContext` and `ProcedureContext` were updated
purely for consistency. There are currently no plans to track reads of
`sender` in these contexts.
# API and ABI breaking changes
Breaks the rust module api.
# Expected complexity level and risk
1
# Testing
N/A
# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/2298.
The general idea is to avoid moving around types that are 400+ bytes or
so only to eventually pass them by reference.
Just pass them by ref as soon as possible instead.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
The goal of this PR is to:
- Make it easy to switch our backing implementation of identifiers, as
seen in the commit changing to `LeanString`.
- Improve type safety of our identifiers by having every identifier
start with `RawIdentifier` and then `Identifier` is just a wrapper with
validation on construction. `TableName` and `ReducerName` are then just
wrappers around `Identifier`.
- Reduce allocation in `InstanceEnv` by using the now clone-on-write
O(1) + SSO optimized `Identifier`.
- Reduce allocations in the query engine as a consequence of improving
`RawIdentifier` and `Identifier`.
- Have `&'static str` strings be further optimized by avoiding to
allocate even for long ones. This is supported by `impl From<&'static
str> for RawIdentifier` as well as switching to `LeanString`.
The PR results in a roughly 2k TPS improvement over master for WASM +
hash indices. This should also help V8, as this is VM-agnostic.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? Mostly just small changes in many places, but in a sea of small
changes, mistakes can happen.
# Testing
Covered by existing tests.
# Description of Changes
Patch:
1. `crates/lib/src/db/raw_def/v10.rs` - a definition as per
https://github.com/clockworklabs/SpacetimeDBPrivate/issues/2412.
2. Refactors `RawModuleDefV9` validation code to dedup some of the core
validation logic, No functionality should change there.
3. Validation logic for new `RawModuleDefV10`.
4. Modify `crates::schema::def::ModuleDef` :
- To include `raw_module_def_version`
- `visibility` field to existing `RedcuerDef` and `ProcedureDef`
5. It deprecates RLS in favour of Views, If anyoine feel otherwise, let
me know.
Lot of code is duplicated from `RawModuleDefV9` includng tests.
# API and ABI breaking changes
NA, `RawModuleDefV10` is not yet exported by modules.
# Expected complexity level and risk
3? close analyses of structure is important to ensure future
extensibility.
# Testing
For code motion in `RawModuleDefV9` validation: Existing unit tests
seems to cover any functionality change.
For `RawModuleDefV10`: V9 test has been repeated with more `assert`s to
check on `visibility`
# Description of Changes
`Uuid` addad in the last update is not filterable even though it is
`Copy` and (imo) obviously _very_ suited to filter.
Promote it to a special `FilterableValue` in the same way as `Identity`
and adjust the docs.
# API and ABI breaking changes
None
# Expected complexity level and risk
1. Only copies the same strategy as `Identity` which is also a newtype
wrapper around a integer like `Uuid`
# Testing
Tested on my project.
After the change:
<img width="586" height="298" alt="image"
src="https://github.com/user-attachments/assets/98e97c16-906b-4e66-b132-676ece40462e"
/>
Before the change:
<img width="1495" height="547" alt="image"
src="https://github.com/user-attachments/assets/da74e49a-1dbd-43ae-9b13-39dcc39c81dc"
/>
You can also test this yourself when adding:
`spacetimedb = { version = "1.11.2", git =
"https://github.com/kistz/SpacetimeDB.git", branch = "uuid-as-filter" }`
to your cargo.toml project
- [x] Tested before and after. Before it was a compiler error afterwards
not.
# Description of Changes
Client Query builder for rust, as per proposal -
https://github.com/clockworklabs/SpacetimeDBPrivate/pull/2356.
1. Pach moves query builder to its separate crate, so that it can be
shared between module and sdk.
2. Implements `TypedSubscriptionBuilder` in `sdks/rust` as mentioned in
proposal
3. Modify codegen to extend types to support query builder as mentioned
in proposal
4. a test
# API and ABI breaking changes
NA, additive changes.
# Expected complexity level and risk
2
# Testing
Added a test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
This pr reexports `PointIndex` and `PointIndexReadOnly` because
currently the new `Hash` indices dont compile when used in projects.
This is because the table macro cant find those two exports.
Maybe @Centril could take a look? :>
# API and ABI breaking changes
None
# Expected complexity level and risk
1. Only fixes a compiler error.
# Testing
- [x] Project compiles now.
Before:
<img width="1454" height="337" alt="image"
src="https://github.com/user-attachments/assets/42df164c-dd09-42b9-97df-a1df719dc3b4"
/>
After:
<img width="1404" height="172" alt="image"
src="https://github.com/user-attachments/assets/0d11158c-daae-44b6-b001-ccccfe0940fd"
/>
# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/1122.
Adds hash indices and exposes them through `#[index(hash)]` for Rust
modules,
with support for typescript and C# to come in follow ups.
On the client/sdk side, for now, any index is backed via a BTree/native
index as it is the most general.
A hash index may only be queried through point scan and never ranged
scans.
Attempting a ranged scan results in an error, with the mechanism
implemented in the previous PR
(https://github.com/clockworklabs/SpacetimeDB/pull/3974).
# API and ABI breaking changes
None
# Expected complexity level and risk
2?
# Testing
A test for ensuring that hash indices cannot be used for range scans is
added.
Tests exercising hash indices will come in the next PR.
# Description of Changes
This PR does several small things:
1. It removes the explicit `h1` tags on every page, and either uses the
side bar title directly, or puts it in the frontmatter
2. It merges what are currently called quickstarts into a single Chat
App Tutorial
3. It creates new quickstarts which just use `spacetime dev --template`
to get you up and running quickly
4. It adds a "The Zen of SpacetimeDB" page much like the Zen of Python
which goes over the 5 key principles of SpacetimeDB
5. It reorders all Tabs groups so that the ordering is `TypeScript`,
`C#`, `Rust`, `Unreal`, `C++`, `Blueprints` (order of decreasing
popularity).
6. It improves the sidebar navigation by having categories act as
overview pages, and also fixes the breadcrumbs
7. It fixes various small typos and issues
8. Closes#3610 and adds cursor rules files generally
9. It fixes general styling on the docs page by bring it inline with the
UI design:
Old:
<img width="1678" height="958" alt="image"
src="https://github.com/user-attachments/assets/f36efee6-b81a-4463-a179-da68b3a7152e"
/>
New:
<img width="1678" height="957" alt="image"
src="https://github.com/user-attachments/assets/f430f77d-0663-47f2-9727-45cbfe10e4c7"
/>
https://github.com/user-attachments/assets/adc5a78a-ada8-45b5-8078-a45cb81477a3
# API and ABI breaking changes
This PR does NOT change any old links. It does add new pages though.
# Expected complexity level and risk
3 - it's a large change. I manually tested the TypeScript Chat App
Tutorial but I have not gone through the Rust and C# quickstarts.
However, we have testing on the quickstarts and this is text only so can
be carefully reviewed.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Ran through each step of the Chat App TypeScript tutorial to
ensure it is working
- [x] Ran and tested the styles and the functionality of the side bar
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
Co-authored-by: clockworklabs-bot <clockworklabs-bot@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Closes
[#3290](https://github.com/clockworklabs/SpacetimeDB/issues/3290).
Adds a new "special" type to SATS, `UUID`, which is represented as the
product `{ __uuid__: u128 }`. Adds versions of this type to all of our
various languages' module bindings libraries and client SDKs, and
updates codegen to recognize it and output references to those named
library types. Adds methods for creating new UUIDs according to the V4
(all random) and V7 (timestamp, monotonic counter and random)
specifications.
# API and ABI breaking changes
We add a new type
# Expected complexity level and risk
2
it impacts all over the code
# Testing
- [x] Extends the Rust and Unreal SDK tests, and the associated
`module-test` modules in Rust, C# and TypeScript, with uses of UUIDs.
- [x] Extends the C# SDK regression tests with uses of UUIDs.
- [x] Extends the TypeScript test suite with tests with uses of UUIDs.
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Provides new WASM ABIs:
- `datastore_index_scan_point_bsatn`
- `datastore_delete_by_index_scan_point_bsatn`
These are then used where applicable to speed up `.find(_)` and friends.
Point scans are also used more internally where applicable.
What remains after this is use in C# module bindings and to expose this
in TS as well.
The PR makes TPS go from roughly 36k to 38k TPS on my machine and also
makes a difference in flamegraphs where the time spent in some index
scans are substantially decreased.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? This touches the datastore an how we expose it to modules.
# Testing
Some existing tests now exercise the new ABIs by changing what
`.find(_)` and friends do.
---------
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
As title.
# API and ABI breaking changes
incremental binding.
# Expected complexity level and risk
# Testing
Unit and smoketest
# Description of Changes
Fixes#3827
Adding lte and gte support for query_builder in rust.
# API and ABI breaking changes
Incremental change to bindings.
# Expected complexity level and risk
1
# Testing
unit test and smoketest
# Description of Changes
As title
# API and ABI breaking changes
NA
# Expected complexity level and risk
0
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.
The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue
So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.
Fixes#3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?
# Expected complexity level and risk
2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.
# Testing
- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
Kilian Strunz