Commit Graph

3186 Commits

Author SHA1 Message Date
Kim Altintop e2b4113ffb Async shutdown for database / durability (#3880)
Controlled shutdown of a database should drain the outstanding
transactions
queue(s) and flush them to the durability layer.

With the introduction of another queueing layer in #3868, it became
harder to
observe when or if this process is completed.

This patch thus introduces an explicit (async) shutdown method for
`RelationalDB` and below, which will wait until all submitted
transactions are
either reported durable, or an error occurs in the durability layer.

`RelationalDB` is made `!Clone`, such that shutdown can be initiated in
the
`Drop` impl. Note that this requires access to a tokio runtime, which we
thread
through via the `Persistence` services in order to allow control over
which of
the various runtimes is being used for durability-related tasks.

Also moves `RelationalDB::open` to a blocking thread when a
persistence-enabled
database is constructed by the `HostController` -- this process performs
heavy
I/O and can take a substantial amount of time, during which we don't
want to
block a worker thread.

# API and ABI breaking changes

None

# Expected complexity level and risk

3

# Testing

- [ ] some testing added
- [ ] existing tests still pass
- [ ] `impl Drop for RelationalDB` difficult to test, extra eyeballs
needed

---------

Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2025-12-17 18:28:42 +00:00
Zeke Foppa 70628fb51e cargo ci on windows (#3859)
# Description of Changes

Make `cargo ci` work properly on Windows, in preparation for
https://github.com/clockworklabs/SpacetimeDB/pull/3702.

# API and ABI breaking changes

No. CI-only.

# Expected complexity level and risk

2. Not trivial, but not complicated.

# Testing

- [x] CI output seems to be genuinely running the tests, and it's
passing on Windows
- [x] Make a change to `crates/bindings-csharp` and see that `cargo ci
test` fails
- [x] I can manually run a minimal `cargo ci smoketests` invocation on a
Windows machine

---------

Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Kasama <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-12-17 17:34:49 +00:00
Phoebe Goldman 13614d7515 Add a fault-tolerant commitlog replay mode for use in debugging (#3887)
# Description of Changes

When debugging broken commitlogs, we want to inspect the whole
commitlog, including the part after the first error.
This is in contrast with the way we want to replay in prod, where we'd
rather get a hard error than an incorrect state.

This commit adds a new flag to commitlog replay, `ErrorBehavior`. The
`core` crate passes `ErrorBehavior::FailFast`
when replaying commitlogs to reconstruct databases. Internal tooling
(not in this repository) uses `ErrorBehavior::Warn` to print the
entirety of a broken commitlog.

# API and ABI breaking changes

Changes internal APIs only.

# Expected complexity level and risk

1 - no change to behavior of SpacetimeDB.

# Testing

None.
2025-12-17 14:15:24 +00:00
Jeffrey Dallatezza a1607ce523 Add Reducer to the end of reducer arg types to avoid collisions (#3889)
# Description of Changes

This helps with the issue reported in
https://github.com/clockworklabs/SpacetimeDB/issues/3811.

Right now we have a type representing the reducer, and a type for the
reducer args, and both have the same name. This adds `Reducer` to the
end of the args type, which is similar to what we are doing for
procedure arguments or the `Row` suffix for tables.

This will still cause some potential problems, since someone could have
a type that ends in `Reducer` (or `Row`), but this will fix the majority
of issues that are currently breaking people.

This also has some changes to get the basic react example to build.

# API and ABI breaking changes

This is technically a breaking change if people are using this for type
annotations (which doesn't seem too likely), but it should be an easy
one for people to fix.

# Expected complexity level and risk

1.

# Testing

I tested the quickstart.
2025-12-17 05:43:33 +00:00
Kim Altintop a388c48458 Ensure all-ephemeral transactions don't consume a tx offset (#3884)
Views are materialized in mutable transactions, but should not increment
the transaction offset maintained in the committed state.

This fixes storing completely empty transactions in the commitlog, and
maintains that the committed state tx offset is in-sync with the
commitlog's tx offset.

# Expected complexity level and risk

2

# Testing

Added a test.

---------

Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2025-12-16 23:14:21 +00:00
Kim Altintop b1676abb82 cli: Flush stdout after printing the database tree (#3888)
`spacetime delete` may print the database tree and ask for confirmation.
We did not flush stdout, causing the tree to be interleaved with the
yes/no prompt.


# Expected complexity level and risk

1
2025-12-16 22:46:03 +00:00
Phoebe Goldman cc2ac5424c Strip sensitive info from HTTP errors (#3878)
# Description of Changes 

`reqwest` includes the full URL in its errors, including query params.
This is unfortunate, as query params can contain sensitive info like API
tokens. It's difficult for modules to clean these themselves, as they
see errors as strings, losing the structure of `reqwest::Error`.
In this commit, we strip query parts out of URLs in errors before
returning them to modules. I've also audited all of the error return
paths in the `http_request` method and left comments justifying why the
unchanged ones are safe.

# API and ABI breaking changes

Only if you consider the format of error messages part of our API, which
I don't. Procedure APIs aren't stable yet anyways.

# Expected complexity level and risk

1

# Testing

None yet - accepting input from reviewers about desired tests if we feel
that's necessary.

- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->

---------

Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Julien Lavocat <JulienLavocat@users.noreply.github.com>
2025-12-16 22:32:17 +00:00
Noa 4d23fe3e8d [TS] Fix useTable casing issues (#3853)
# Description of Changes

Fixes #3807. Not sure what the best way to model this in the API is
(another argument to `t.row()`?), but it does work.

# Expected complexity level and risk

1

# Testing

- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
2025-12-16 21:57:55 +00:00
Zeke Foppa f49945cc6a Fix CLI reference generation (#3403)
# Description of Changes

Add back the instructions for regenerating CLI docs, which were removed
in https://github.com/clockworklabs/SpacetimeDB/pull/3343. I also made a
script for it.

This also fixes the CI checking this file, which was silently broken in
the same PR.

I have **not** verified that this works in Git Bash in Windows.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing
- [x] CI passes
- [x] CI fails if I change the CLI reference
- [x] CLI reference looks visually reasonable on a local `pnpm dev`

---------

Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-12-16 20:17:51 +00:00
Jeffrey Dallatezza 4ebb31c573 Fix useReducer params (#3873)
# Description of Changes

`ParamsType` is a tuple, so we need to spread when we actually call
reducer functions.

Currently reducer arguments aren't being sent with the `useReducer`
hook.

# Expected complexity level and risk

1

# Testing

I tested this manually with the quickstart chat app.

Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
2025-12-16 20:17:27 +00:00
Julien Lavocat 178bb9d523 Fix broken SpacetimeDB logo in docs (#3886)
# Description of Changes

The `.png` logo image has been replaced by a `.svg` and needed to be
updated.

# API and ABI breaking changes

Not applicable

# Expected complexity level and risk

0.000001

# Testing

Run `pnpm dev` and check that the logo is correctly displayed in the top
left corner
2025-12-16 20:17:27 +00:00
Jeffrey Dallatezza 12fe0217b1 Tighten query types (#3866)
# Description of Changes

This improves the type safety a bit from
https://github.com/clockworklabs/SpacetimeDB/pull/3812.

The core change is that the previous version typed queries based on the
typescript type, not the spacetime type. This meant that we allowed
queries for incorrect tables, like a table that had the same column
names and types, but had a u32 instead of a u64 somewhere.

This still has an issue with allowing results from tables where the rows
are reordered, which would actually be a problem, but hopefully that is
not too common.

# API and ABI breaking changes

This is technically a breaking change, because it changes some type
parameters. I don't think people should be relying on these though, so I
don't think we should be worried about breaking them.

This would only cause new type errors for apps that are likely to error
at runtime anyway.

# Expected complexity level and risk

1.5. This should be low risk, since it is just a typing change.

# Testing

This has some type checks in `view.test-d.ts`, and I've done some manual
e2e testing.
2025-12-16 19:07:52 +00:00
Mazdak Farrokhzad 492e591845 Add WASM point scan ABIs & use them in Rust bindings (#3863)
# Description of Changes

Provides new WASM ABIs:

- `datastore_index_scan_point_bsatn`
- `datastore_delete_by_index_scan_point_bsatn`

These are then used where applicable to speed up `.find(_)` and friends.

Point scans are also used more internally where applicable.

What remains after this is use in C# module bindings and to expose this
in TS as well.

The PR makes TPS go from roughly 36k to 38k TPS on my machine and also
makes a difference in flamegraphs where the time spent in some index
scans are substantially decreased.

# API and ABI breaking changes

None

# Expected complexity level and risk

3? This touches the datastore an how we expose it to modules.

# Testing

Some existing tests now exercise the new ABIs by changing what
`.find(_)` and friends do.

---------

Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
2025-12-12 22:07:54 +00:00
Zeke Foppa 8967ea7b59 CI - Use rust-toolchain-file everywhere (#3872)
# Description of Changes

We were using `rust-toolchain` in some places, and `rust-toolchain-file`
in others. I think Rust released a new version, which made the
`rust-toolchain` parts break with:

```
info: downloading component 'rustfmt'
info: removing previous version of component 'rust-src'
info: rolling back changes
error: could not rename component file from '/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/term/terminfo/searcher' to '/root/.rustup/tmp/a4eo07uz83vsyfhk_dir/bk': Invalid cross-device link (os error 18)
Error: Process completed with exit code 1.
```

(Separately, this breakage is confusing.. we'll probably run into this
again when we roll forward our rust version..)

# API and ABI breaking changes

None. CI-only change.

# Expected complexity level and risk

1

# Testing

- [x] CI passes
- [x] There are no more instances of `rust-toolchain`:
```
$ grep -rIP 'rust-toolchain(?!-file)' .github/workflows
.github/CODEOWNERS:/rust-toolchain.toml @cloutiertyler
```
(on `master`, this finds the instances we changed)

---------

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-12-11 18:51:48 +00:00
Mazdak Farrokhzad 907d67ec1f wasmtime: pool async stack allocations on unix (#3830)
# Description of Changes

Uses `with_host_stack` to provide a `StackCreator` that pools
`FiberStack`s.
This does not use the pooling instance allocator and is limited to just
stacks.

# API and ABI breaking changes

None

# Expected complexity level and risk

3? Some unsafe and wasmtime internals relied upon.

# Testing

Covered by existing tests.
2025-12-11 07:46:47 +00:00
Roberto Pommella Alegro 5ac65739e5 add initial cargo ci (#3409)
# Description of Changes

This changes the ci runs to execute `cargo ci` instead of running
commands directly from the github workflow.

The goal here is to unify the commands under `cargo ci` so that it's
easier and more intuitive to run locally

# API and ABI breaking changes

There are no API/ABI changes.

<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->

# Expected complexity level and risk

Complexity: 1

It is not a complex change as it is mostly localized to the ci runs and
is easily reversible if something goes wrong. The biggest risk here is
to have future CI runs break, which can be remediated by reverting these
changes.

<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.

This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

- [x] run `cargo ci` and its subcommands locally
- [x] run the github workflow against this branch to check if the CI
jobs are working properly.

---------

Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Signed-off-by: Roberto Pommella Alegro <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
2025-12-10 19:18:43 +00:00
Mario Montoya cdf8391ff3 Add rng support to ProcedureContext on Rust (#3865)
# Description of Changes

As the title says, required for
https://github.com/clockworklabs/SpacetimeDB/pull/3538.

# Expected complexity level and risk

1

# Testing

Will be done on the `uuid` pr.

---------

Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
2025-12-10 19:05:21 +00:00
Mazdak Farrokhzad e5f1558ec7 Don't panic in datastore_index_scan_range_bsatn on invalid ranges (#3746)
# Description of Changes

Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3617.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

A proptest `empty_range_scans_dont_panic` is added.
2025-12-10 13:45:36 +00:00
Shubham Mishra 6c0b66ad6e filter alias for rust query builder's where method (#3856)
# Description of Changes
As title.


# API and ABI breaking changes
incremental binding.

# Expected complexity level and risk

# Testing
Unit and smoketest
2025-12-10 13:19:32 +00:00
Shubham Mishra 17b6ee1847 fix automigration bug with auto_inc (#3862)
# Description of Changes
fixes #3861.

While running prechecks for automigration, the `range` passed to
`iter_by_col_range_mut` was of type `AlgebraicValue::I128`, even though
it should have matched the column’s type.

Fix is straightforward by introducing `AlgebraicValue::from_i128`
helper.

# API and ABI breaking changes
NA

# Expected complexity level and risk
1

# Testing

- Modified smoketest to repro reported bug.

---------

Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2025-12-10 13:18:38 +00:00
Shubham Mishra d6eff1befe lte and gte operators to rust query builder (#3855)
# Description of Changes
Fixes #3827 

Adding lte and gte support for query_builder in rust.


# API and ABI breaking changes
Incremental change to bindings.

# Expected complexity level and risk
1


# Testing
unit test and smoketest
2025-12-10 12:24:34 +00:00
Jeffrey Dallatezza 0866367b07 Handle unknown transactions without erroring. (#3858)
# Description of Changes

Don't throw an error if there is no `reducerInfo`. The code was
previously trying to handle the case of an unknown reducer, but was
effectively asserting that the reducerInfo existed too soon.

Now we should be fine handling a transaction, even if we can't determine
reducer information for it.

# Expected complexity level and risk

1.

# Testing

I tested this by running the quickstart chat app, and using the CLI to
delete rows (via `spacetime sql`). Before this change, the client
errors, but now it handles it correctly.

I also tested with the repro in
https://github.com/clockworklabs/SpacetimeDB/issues/3817
2025-12-09 22:37:49 +00:00
Noa bea7a7dc68 [TS] Use 'override' when overriding class items (#3852)
# Description of Changes

This should fix part of #3503. Adds an override modifier to generated
code and fixes a warning from the angular compiler.

# Expected complexity level and risk

1

# Testing

- [x] Code works fine with the added `override` modifier.
- [ ] Perhaps we should have an angular test project?
2025-12-09 20:08:40 +00:00
Phoebe Goldman 111bc1a1f9 Rename docs dirs and files to have more digits (#3851)
# Description of Changes

This makes it possible to insert new pages between existing pages
without renaming the existing pages.

I also added a section to docs/README.md with procedure for how to name
new pages, which I'll paste here:

### Adding new pages

All of our directory and file names are prefixed with a five-digit
number which determines how they're sorted.
We started with the hundreds place as the smallest significant digit, to
allow using the tens and ones places to add new pages between.
When adding a new page in between two existing pages, choose a number
which:

- Doesn't use any more significant figures than it needs to.
- Is approximately halfway between the previous and next page.

For example, if you want to add a new page between `00300-foo` and
`00400-bar`, name it `00350-baz`. To add a new page between `00350-baz`
and `00400-bar`, prefer `00370-quux` or `00380-quux`, rather than
`00375-quux`, to avoid populating the ones place.

To add a new page after all previous pages, use the smallest multiple of
100 larger than all other pages. For example, if the highest-numbered
existing page is `01350-abc`, create `01400-def`.

# API and ABI breaking changes

N/a

# Expected complexity level and risk

1 - @JulienLavocat said this wouldn't break anything.

# Testing

None.

---------

Co-authored-by: Jason Larabie <jason@clockworklabs.io>
2025-12-09 19:11:48 +00:00
Kim Altintop 16f1b2c1fe client-api: Deny changing the parent of an existing database (#3837)
Mainly a smoketest to exercise the intended behaviour. Also return an
error if we end up delegating to the reset database endpoint, which
itself doesn't accept a `parent` parameter.
2025-12-09 18:35:48 +00:00
John Detter 7e367b38a6 Update typescript package size limits (#3854)
# Description of Changes

<!-- Please describe your change, mention any related tickets, and so on
here. -->

Updated package sizes from a failed release dry-run:
https://github.com/clockworklabs/SpacetimeDBPrivate/actions/runs/20045893743/job/57491806595

# API and ABI breaking changes

<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->

None

# Expected complexity level and risk

1 - this is just a release fix

<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.

This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

I have not tested this, it is a trivial change.
2025-12-09 11:02:33 +00:00
Kim Altintop 062649c92e client-api: Send WebSocket messages fragmented (#2931)
RFC 6455, Section 5.4 describes message fragmentation, and we can do
that with tungstenite.

It does seem to help getting control messages (ping, pong, close)
through without head-of-line blocking.

# Expected complexity level and risk

2 - Need to test with clients

# Testing

TBD - some more abstraction is needed due to the difficulty of
synthetically producing a large outgoing message.
2025-12-09 09:21:11 +00:00
Noa af5b04e949 Implement sourcemap handling (#3828)
# Description of Changes

Uses the `sourcemap` crate to map text locations in the bundle to text
locations in the original source code.

# Expected complexity level and risk

1 - essentially only related to diagnostics

# Testing

- [x] Manually tested
- [ ] Add an automated test for backtrace output
2025-12-08 22:29:54 +00:00
Shubham Mishra 5100aec3d4 Some more UI tests for Views (#3829)
# Description of Changes
As title

# API and ABI breaking changes
NA

# Expected complexity level and risk
0

---------

Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
2025-12-08 19:33:02 +00:00
joshua-spacetime 93c0015f51 fix error when dropping view in auto-migration (#3836)
# Description of Changes

Fixes the following issues:
1. When dropping a view, we deleted its row from `st_view`, but didn't
drop the backing table.
2. `delete_col_eq` returned a nonsensical error if the delete set was
empty.

# API and ABI breaking changes

None

# Expected complexity level and risk

0

# Testing

- [x] Auto-migrate smoketests
2025-12-08 07:01:21 +00:00
Noa afe169ac4a Fix the issues with scheduling procedures (#3816)
# Description of Changes

This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.

The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue

So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.

Fixes #3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?

# Expected complexity level and risk

2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.

# Testing

- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
2025-12-05 22:27:30 +00:00
Piotr Sarnacki e0dc9fb261 Remove energy multiplier from wasmtime fuel calculations (#3832)
# Description of Changes

In the past we've been converting CPU instructions into energy. We are
not doing it on the SpacetimeDB side anymore, thus we should report the
WasmTime fuel directly
2025-12-05 19:09:28 +00:00
Shubham Mishra 013b202861 Views: snapshot to skip ephermeral tables (#3720)
# Description of Changes.
fixes #3715 

The patch makes snapshots to skip ephemeral tables.

# API and ABI breaking changes
NA

# Expected complexity level and risk
1

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->

---------

Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
2025-12-05 09:20:41 +00:00
Jeffrey Dallatezza 3e7ab2ccac Add a typescript query builder for views (#3812)
# Description of Changes

This adds a way to build queries with typescript views, and it allows
views to return queries (if the return type of the query is an array).

For examples and syntax, you can look at the tests in
[crates/bindings-typescript/tests/query.test.ts](https://github.com/clockworklabs/SpacetimeDB/compare/jsdt/ts-query-builder?expand=1#diff-4fd25c191f1207085a491cf84996c601f805f5e8280d1cf2a812ebad6aa6e75a).

To play around with the syntax, you might find it easier to look in
[crates/bindings-typescript/src/server/view.test-d.ts](https://github.com/clockworklabs/SpacetimeDB/compare/jsdt/ts-query-builder?expand=1#diff-4fd25c191f1207085a491cf84996c601f805f5e8280d1cf2a812ebad6aa6e75a).

This could still use some cleanup, and there are some places where the
type safety is imperfect. I'll try to list the known limitations here:
1. This will allow the use of `eq` for columns that are product types,
even though the query engine doesn't support it. This can be fixed
later, and it would only be a breaking change for modules that have
invalid queries.
2. When we check if a view is returning a query of the correct type, we
are checking with the typescript row type. We should be checking with
the spacetime type, since this type checking will allow a couple
incorrect things to be returned:
1. A different table with any superset of the fields (for example, a
different table that has one extra field). That will fail when
executing, but will be allowed by the typescript compiler.
2. A table with the same fields, but with those fields in a different
order would also fail at runtime, but be allowed by the typescript
compiler.
4. A table with fields of a different spacetimetype that map to the same
typescript type (like `u16` and `u32`).

I can also add back functions for things like inequality once we are ok
with the rest of it.

# API and ABI breaking changes

This adds some new API surface, but does not break existing code.

# Expected complexity level and risk

2.

# Testing

For automated tests, there are unit tests to see what sql gets emitted
in `tests/query.test.ts`, and some tests of the types in
`view.test-d.ts`.

I've also run some manual tests with a typescript module with views.
2025-12-04 23:02:33 +00:00
Tyler Cloutier 46f3e07dfc Fixes issues with --delete-data=on-conflict (#3730)
# Description of Changes

Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3729

I genuinely don't know what came over me.

# API and ABI breaking changes

None

# Expected complexity level and risk

1.5 very straightforward but not strictly trivial

# Testing
Adds automated integration tests (written in Rust and run with `cargo
test`, although note this comment from @matklad about integration tests
for the future
https://internals.rust-lang.org/t/running-test-crates-in-parallel/15639/2):

- [x] Can publish an updated module if no migration is required
- [x] Can publish an updated module if auto-migration is required (with
the yes-break flag true/false)
- [x] Cannot publish if a manual migration is required
- [x] Can publish if a manual migration is required but the user
specified `--delete-data`
- [x] Can publish if a manual migration is required by the user
specified `--delete-data=on-conflict`
- [x] No data deletion occurs if no migration is required and
`--delete-data=on-conflict` is specified

---------

Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
2025-12-04 21:45:41 +00:00
Kim Altintop a959996ba7 Debug "stuck module" issue (#3813)
Adds some logging and times out lock acquisition attempts in the host
controller.

Should help debugging clockworklabs/SpacetimeDBPrivate#2337
2025-12-04 20:23:32 +00:00
Noa c2b8c7fc7f [TS] Implement TextEncoder/TextDecoder with native code (#3800)
# Description of Changes

This implements (a subset of) the TextEncoder/TextDecoder web APIs using
native functions to do the actual `Uint8Array <-> String` conversion.
This should be a good bit faster than the `fast-text-encoding` package.

# Expected complexity level and risk

2 - this introduces new kinds of JS code and host calls to the v8 host,
but they're pretty well encapsulated.

# Testing

- [x] All TS modules make use of these already for encoding/decoding
strings in BSATN - the `fast-text-encoding` polyfill we pull in only
takes effect if the classes don't already exist, so the smoketests
passing means it works.
2025-12-04 20:11:52 +00:00
Kim Altintop f458d418e5 smoketests: Adjust test_enable_disable_replication test (#3822)
This now can't rely on `replication_state` before enabling replication.
2025-12-04 18:55:38 +00:00
Mazdak Farrokhzad 55e0c99850 Store SubscriptionMetrics for Update, Subscribe, Unsubscribe in ModuleSubscriptions (#3821)
# Description of Changes

This showed up in flamegraphs, in particular for `Update`, so let's
cache the `SubscriptionMetrics`s.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

Covered by existing tests.
2025-12-04 18:36:56 +00:00
Shubham Mishra c99a6131c8 [Rust] Query builder Integration. (#3823)
# Description of Changes
Patch contain few different things, required to run query builder end to
end.
Git history seems messed up but it can be reviewed commitwise from -
69c1907b3e.

1. Decouples generic type `T` from table name, as type represents the
`struct` and not table, and there couple be multiple tables implemented
from same struct.
2. modify table macro implementaiton to generate code supporting
`traits` and types required for query builder,
3. Disable aliasing in sql queries for now, as it was causing
semantically wrong query when we were doing joins on `FromWhere` (as
original `Expr` were referencing to original table names).
4. smoketests.


# API and ABI breaking changes
Add `Query<T>` type as Views return type.


# Expected complexity level and risk
2, changes are simple but not splitted up well.

# Testing
1. Added smoketest.

TODO:
ui test.

---------

Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
2025-12-04 17:45:29 +00:00
Noa 653a2a1bad Update wasmtime to v39 (#3818)
# Description of Changes

As discussed; could possibly improve performance/in general it's good to
keep up with patches, it's been over a year since we last bumped this.

# Expected complexity level and risk

2 - large, important dependency but wasmtime is very solid,
well-engineered software.

# Testing

- [x] smoketests run on wasmtime
2025-12-04 10:53:51 +00:00
joshua-spacetime 60048ecbdf [Rust] update module bindings to use new view abi (#3819)
# Description of Changes

Updates the rust module bindings to use new view abi
(`ViewResultHeader`) and updates the return codes for the
`__call_view__` and `__call_view_anon__` module exports. This is in
preparation for `Query` builder support.

# API and ABI breaking changes

Not breaking. Existing modules will continue to use the old abi. New
modules will use the new abi. However previous host versions will not
support modules built using this version of the bindings.

# Expected complexity level and risk

2

# Testing

This is an internal refactor. All existing tests should continue to
pass. The only tests that needed updating were the negative module
compilation tests because the error messages produced by rustc changed.
2025-12-04 08:28:41 +00:00
Shubham Mishra 51ec052509 [Rust] Module-side query builder types (#3797)
# Description of Changes

Implements a query builder for rust modules according to the
[spec](https://github.com/clockworklabs/SpacetimeDBPrivate/blob/master/proposals/0030-views.md#5-query-builder-for-incremental-evaluation).

Look at the tests in `query_builder.rs`.

TODO: Extend return type for views to include `Query` and update the
`#[table]` macro to generate query builder bindings.

# API and ABI breaking changes

This is an additive change. Current modules do not need to be updated
unless they wish to use the query builder.

# Expected complexity level and risk

1.5, Will appreciate comments to better structure code.

# Testing 

[x]: unit tests.

I have also done type safety checks manually from module.

---------

Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
2025-12-04 06:53:26 +00:00
Jeffrey Dallatezza 9989416cd3 Update view ABI to support returning queries (#3685)
# Description of Changes

This adds some changes for how we return data from view functions.
Originally, we interpreted the output of a view function as a bsatn
encoding of an array of rows. Since we also want to be able to return
queries from view functions, we need to be able to return different
types too. At this point, this is effectively not a functional change,
since we don't use the new format, and we don't actually try to parse
the new format.

This introduces a new format for view returns, which is a
`ViewResultHeader`, potentially followed by additional data. For
example, if a view were returning rows directly, it would write a
`ViewResultHeader::RowData`, followed by an array of rows. Note that we
could have given that object a byte array with the rows instead of using
a header an a separate object, but that would force us create an extra
copy when encoding and when decoding.

To make this backward compatible with existing views, we have a
different way to return the new format. For v8 views, if they return a
byte array, we assume it is the old format. If they return an object, we
expect the `data` field of that object to be the actual return data.

For wasm views, we interpret a return code of 2 to mean that it uses the
new format.

On the host side, we handle this naively: we will perform the query, and
we will act as though the view has a read dependency on the tables in
the query. In follow up PRs we can make this more efficient.

# API and ABI breaking changes

This is not a breaking change, but it does make the ABI more complicated
(specifically to avoid breaking it).

# Expected complexity level and risk

1.5. This should not affect the existing return style.

# Testing

I've done manual testing of this with a version of the typescript
bindings that returns queries.
2025-12-03 19:56:41 +00:00
Jason Larabie 52d098dbfb Added and tested procedure docs for Unreal C++ & Unreal Blueprint (#3810)
# Description of Changes
Closes: #3781 
- Added and tested procedure docs for Unreal C++ & Unreal Blueprint
- Fixed a small issue with the blueprint accessors for the event status
- Fixed a bug in the C# procedure docs

# API and ABI breaking changes

N/A

# Expected complexity level and risk

1

# Testing

I built a new local test framework to do another clean end to end test
of procedures with Unreal to properly test the Blueprint work as well.

- [x] Built the Rust server alongside a new Unreal client and tested
each procedure/callback in C++ and Blueprint
- [x] Reviewed the docs locally
2025-12-03 19:47:25 +00:00
Zeke Foppa 141048cdd8 Bump versions to 1.11.0 (#3808)
# Description of Changes

Bumping versions to 1.11.0 in preparation for an upcoming release.

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

- [x] Existing CI passes

---------

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-12-02 22:45:29 +00:00
Zeke Foppa 64aa808957 CI - Skip the Unity testsuite on external PRs (#3805)
# Description of Changes

Bubbling up an `if` since we split out the unity testsuite into its own
job.

The GitHub docs say
(https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks):
> A job that is skipped will report its status as "Success". It will not
prevent a pull request from merging, even if it is a required check.

# API and ABI breaking changes

None. CI-only change.

# Expected complexity level and risk

1

# Testing

Unsure how to test this honestly 🤷

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-12-02 21:09:52 +00:00
Noa 1ef025f501 [TS] Call registerType for procedure params (#3806)
# Description of Changes

An important fix I put in #3704 that's not in master anymore because it
was reverted.

# Expected complexity level and risk

1
2025-12-02 19:19:16 +00:00
Kilian Strunz 3cead3df6a [TS] Fix development exports breaking NextJS (#3796)
# Description of Changes

NextJS 15 and 16 do not build without these development exports removed
tho they funnily enough have different errors.
<img width="961" height="262" alt="image"
src="https://github.com/user-attachments/assets/1e7ad634-d68f-4af3-a07e-302d252b41e5"
/>

Upon removing the development export everything works like a charm.
I couldn't find any benefit of having them but maybe @cloutiertyler
knows more why those are there in the first place🤷

This can be reproduced if you clone this repo i set up:
https://github.com/kistz/spacetime-next-react-hooks
Since its a build error it shouldnt matter if you have the actual module
running somewhere.

Step 1: `bun install`

Step 2: `bun dev`

Step 3: observe that the error above is appearing (not good)

Step 3: Go into `node_modules/spacetimedb/package.json` 

Step 4: Remove the `development` fields under exports like in this pr

Step 5: `bun dev`

Step 6: Profit. (should only be a connection error (aka nothing renders)
at this point since the module your trying to connect to is missing)

# API and ABI breaking changes

None

# Expected complexity level and risk

1

# Testing

Not a whole lot since i dont fully grasp the impact of this change but
without it i cant even use the `spacetimedb` package with NextJS and
removing them fixes it.
But even development scenarios are working just fine without it.

Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
2025-12-02 18:28:06 +00:00
clockwork-tien 0c82b6559d Docs: Update docs nav height to 56px (#3788)
# Description of Changes

- Update docs nav height to 56px

## Screenshots:

<img width="1256" height="55" alt="image"
src="https://github.com/user-attachments/assets/df070fb2-f3c8-4f17-ae58-6a6aa72c8033"
/>

<!-- Please describe your change, mention any related tickets, and so on
here. -->

# API and ABI breaking changes

<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->

# Expected complexity level and risk

<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.

This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->

# Testing

<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->

- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
2025-12-02 15:48:45 +00:00