# Description of Changes
This PR is a very large change to the workings of the TypeScript SDK and
as such requires a higher bar of testing than other PRs. However, it
does several important things:
1. Unifies the API of the server and client so they not only have the
same API, but they actually implement it with the same TypeScript types.
This fixes several inconsistencies between them and fixes several small
bugs as well.
2. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3365
3. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3431
4. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3435
5. Subsumes the work done in
https://github.com/clockworklabs/SpacetimeDB/pull/3447
6. Derives all type information on the client from a single
`RemoteModule` type which vastly cleans up the correctness of type
checking on the client and helped me to find several small bugs
It accomplishes this by changing code generation of TypeScript on the
client to code generation approximately what a developer would manually
write in their module. The ultimate goal would be to allow the developer
to use the types and functions that they define on in their module
directly on the client without needing to do any code generation at all,
provided they are using TypeScript on the server and client.
https://github.com/clockworklabs/SpacetimeDB/issues/3365 is resolved by
`.build()`ing the `DbConnection` inside a React `useEffect` rather than
doing it directly in line with the render of the provider. In order to
do that we needed to not expose the `DbConnection` directly to
developers by returning a different type from `useSpacetimeDB`.
`useSpacetimeDB` now returns a `ConnectionState` object which is stored
as React state and updates when any of the fields change. This change
also resolves https://github.com/clockworklabs/SpacetimeDB/issues/3431.
https://github.com/clockworklabs/SpacetimeDB/issues/3435 was the issue
that initially lead me down the rabbit hole of unifying the server and
the client because it was nearly impossible to track down all the
various type functions and how they connect to the values that we code
generate on the server. After several hours of attempting this, I
decided to clean up the types a bit to be more uniform.
Implementing the unification between the client and the server also
necessitated fully implemented parts of the API that were fully
implemented on the server, but were broken or missing on the client.
# API and ABI breaking changes
[Unification] -> Means that this is breaking behavior for the client
SDK, but that the new behavior is identical to the server's existing
behavior
## Breaking changes:
- Table accessor names and index accessor names are converted to
camelCase on the `ctx`, so `ctx.db.foo_bar` is now `ctx.db.fooBar`
- [Unification] On the client `my_table.iter()` returns
`IterableIterator` instead of an `Array`
- [Unification] `module_bindings` now export `TypeBuilder`s for all
types instead of a `type MyType` and object `MyType`, so instead of
using `MyType` as a type directly, you need to infer the type `MyType`
-> `Infer<typeof MyType>`.
- [Unification] We no longer generate and export `MyTypeVariants` for
sum types (these are now accessed by `Infer<typeof
MyType.variants.myVariant>`)
- [Unification] `MyType.getTypeScriptAlgebraicType()` has been replaced
with `MyType.algebraicType`
- `useSpacetimeDB()` no longer takes type parameters
- `useTable()` now takes a `TableDef` parameter and type params are
inferred
- `useTable()` now just returns an `Array` directly instead of a object
with `{ rows }`
- [Unification] `ctx.reducers.createPlayer(argA, argB)` ->
`ctx.reducers.createPlayer({ argA, argB })`
- [Unification] `ctx.reducers.onCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.onCreatePlayer(ctx, { argA, argB })`
- [Unification] `ctx.reducers.removeOnCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.removeOnCreatePlayer(ctx, { argA, argB })`
- [Unification] `myTable.count(): number` -> `myTable.count(): bigint`
## Additive changes:
- `Infer<>` now also does `InferTypeOfRow<>` if applicable
- Added a `useReducer()` React hook
- `module_bindings` now exports a `tables` object with references to all
the `TableDef`s
- `module_bindings` now exports a `reducers` object with references to
all the `ReducerDef`s
- Added a new `MyType.create('MyVariant', ...)` function in addition to
the `MyType.MyVariant(...)` constructors (this is private)
## Notable things that did not change:
- `MyType.serialize(writer: BinaryWriter, value: Infer<typeof MyType>)`
and `MyType.deserialize(reader: BinaryReader): Infer<typeof MyType>` are
still supported exactly as before.
- The `MyType.MyVariant(...)` constructor function on sum types is still
present, but implemented with the private `MyType.create('MyVariant',
...)`. We could choose to move away from this API later if we didn't
like the variants polluting the namespace
# Expected complexity level and risk
4 - This is a deep reaching an complex change for the SDK. For the
server, it is much less deep reaching since it reuses much of the same
machinery, although it does require thorough testing there as some of
the code was modified.
This change is fully localized to TypeScript and does not touch the host
(or other languages) at all, and therefore only impacts a beta aspect of
SpacetimeDB.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Added regression test for
https://github.com/clockworklabs/SpacetimeDB/issues/3435
- [x] Manually tested `test-app` and `test-react-router-app`
- [ ] Add test cases for camelCase-ing
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Noa <coolreader18@gmail.com>
This is a draft of the new functionality for `spacetime init`. In order
to run it with built-in templates you have to set the path to the config
file:
```
export SPACETIMEDB_CLI_TEMPLATES_FILE=crates/cli/.init-templates.json
```
In the future it will fetch the list from GH.
A few notes:
* the previous functionality of `spacetime init` does not work at the
moment
* the code needs a bit more cleanup and tests before merging
* there is a bit of a mix in how we generate empty server and client
projects. For Rust we use the existing way of generating. For TypeScript
we clone an empty project from the repo. I wanted to play with both ways
of doing things, and I'm still not sure which is better. Generation in
Rust means that the generated code will match the CLI version and not
necessarily whatever is in Git. On the other hand, for the builtin
templates we will be fetching the newest version from GH, which I guess
might also not what we want, ie. we probably want only stable templates.
More discussion is needed here
* we use `spacetimedb` directory for the server files
* I don't particularly like the inability to disable interactive mode
easily. We discussed disabling it by default if all of the required
arguments are passed, but I don't think it's feature proof. For example,
if someone relies on a non-interactive mode, and we add a new required
argument, instead of printing a message `missing --foo`, we will
automatically launch interactive mode, which is harder to debug. That's
why I think I'd prefer to implement `--non-interactive` argument
* it's kind of hard to keep the legacy behaviour. If you don't pass any
arguments, we go into interactive mode. In the legacy version, we would
print required arguments. If someone passes `--lang` or `--project-path`
explicitly, I guess we could run the legacy workflow, but not sure if
it's worth it, as the command was marked as unstable anyway
* the project path defaults to the project name, but I think we should
probably replace change whitespaces to dashes, or at least ask for the
project path with the project name being the default (or both)
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Currently based on #3361
Implements most of the TS module API (not yet a function for type
aliases).
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Extremely basic module stuff works
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
* Make sure the user provides at least one of `--rust-and-cli`,
`--typescript`, or `--csharp`, since providing none of them is a no-op
as of #3308
* Do a semver-parsing of the arg before doing anything, and use that
parsed version everywhere
* Consolidate some version strings that we were computing in a few
places
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Running `cargo bump-versions 1.5.0 --typescript --rust-and-cli
--csharp` only shows a diff in the change dates
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
* Small cleanups in `tools/check-diff.sh`
* Use `tools/check-diff.sh` wherever appropriate
* Simplify the `sdks/csharp/tools~/gen-*.sh` files after the repo merge
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI still passes
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
It turns out that cargo automatically uses the latest semver-compatible
versions of dependencies, which is not what we expected. tl;dr
specifying `1.5.0` actually means `>=1.5.0 <2.0.0`, but we actually
intend `1.5.*`.
This PR updates our `upgrade-version` tool, and re-runs it to fix the
dep versions.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] I ran `cargo bump-versions 1.5.0 --rust-and-cli` to regenerate the
other committed files.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fixes two problems introduced in #3185:
1. The `--directories` option on `find-publish-list.py` was not printing
directories
2. The `publish-crates.sh` script was using an undefined variable.
# API and ABI breaking changes
No breaking changes
# Expected complexity level and risk
1
# Testing
This was used to publish crates in the release yesterday.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fixed how the multi-line regexps worked on Windows (with `\r\n` line
endings).
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] `cargo run -- 1.4.1` works properly on Linux still
- [x] Applying the same diff and running on Windows, now updates the
license files properly.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Apparently, I missed several license files in #3002. I'm not sure what
method I was using to find them, but apparently it was insufficient.
**This replaces all empty `LICENSE` files with an explicit (symlink to)
BSL license, and all apache licenses with symlinks to the root apache
license.** This PR does not intentionally change any license terms, so
if you see one that changed, **it's a mistake**.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
```bash
$ find . -name '*LICENSE*' -type f | grep -v '\.meta$'
./crates/sqltest/standards/LICENSE # this one is an external library that we are not allowed to re-license
./LICENSE.txt # this is the root license
```
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
I updated `tools/publish-crates.sh` and `tools/find-publish-list.py` to
be more flexible to where our crates can be located (rather than
hardcoding `crates/foo`). This is in preparation for
https://github.com/clockworklabs/SpacetimeDB/pull/3181.
Now, `find-publish-list.py` loads the output of `cargo metadata` to
dynamically find the path of the crate in question. This also allows us
to "properly" determine which crates are ours, instead of the
`spacetimedb-` string prefix check that we were doing before.
It also now supports `--directories` to output directory paths, rather
than just crate names. `publish-crates.sh` now uses those output paths
rather than assuming that it knows where to find crates.
As a bonus, this approach is also much faster (~0.3s to find the crate
list, vs ~8 before to load and process all the tomls).
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
- [x] `find-publish-list.py` lists the same crates before and after:
```bash
$ ( git checkout master && python3 tools/find-publish-list.py --recursive --quiet bindings sdk cli standalone > before.txt )
$ ( git checkout bfops/flexible-publish-scripts && python3 tools/find-publish-list.py --recursive --quiet spacetimedb spacetimedb-sdk spacetimedb-cli spacetimedb-standalone > after.txt )
# the new script prints out crate names rather than directory names, so we need to tweak a bit
$ diff -U2 before.txt <(cat after.txt | sed 's/^spacetimedb-//' | sed 's/^spacetimedb$/bindings/')
--- before.txt 2025-08-20 10:18:07.323217870 -0700
+++ /dev/fd/63 2025-08-20 10:35:38.344074842 -0700
@@ -8,17 +8,17 @@
data-structures
schema
-table
-expr
-physical-plan
paths
fs-utils
commitlog
+table
durability
-execution
+expr
+physical-plan
snapshot
+execution
client-api-messages
query
-subscription
vm
+subscription
datastore
auth
```
Some lines are reordered because we find dependencies via metadata
instead of toml now - I spot-checked some of the moved lines to see
whether they were in an invalid place, and I did not find any evidence
of that.
- [x] `--directories` flag prints correct directories instead of names
```bash
$ python3 tools/find-publish-list.py --directories --quiet --recursive spacetimedb-sdk
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/memory-usage/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/primitives/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/metrics/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/bindings-macro/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/sats/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/lib/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/client-api-messages/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/data-structures/Cargo.toml
/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/crates/sdk/Cargo.toml
```
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We recently merged several repos together. This PR clarifies the license
terms for several subdirectories, as well as the relationship between
the licenses.
The licenses in our subdirectories have become symbolic links to
licenses in our toplevel `licenses` directory. For any particular
subdirectory's license file in the diff, you can click `... -> View
file` and then click on the text that says "Symbolic Link" on that page.
This will take you to the license file that it links to.
I have also updated the `tools/upgrade-version` script to update the
change date in the new `licenses/BSL.txt` file.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None. Only changes to license files.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
* Script to run perf against SpacetimeDB
* Non-controversial script improvements
* No args is fine
---------
Co-authored-by: Boppy <no-reply@boppygames.gg>
When building a release for a tag we have to build a multi arch image
ourselves. So far we tagged it with a tag `<short-sha>-full`. This is
nice to keep the history of all the tagged commits, but it's not very
usable as usually we just want the latest build for a given tag. This
commit introduces tagging the build with a git tag
Because we build our docker images in more than one job, they are also
pushed as separate images. This commit adds a script that will merge the
images into a multiplatform manifest
* Added license files for the primitives crate
* Fixed version number dependency thing
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
* Saving because I'm testing writing files
* New upgrade program working quite well
* Update license file as well
* Tool seems good to go
* Cargo check is passing, new upgrade-version is ready, old version
removed
* Updating lock file is required for CI to pass
* main.rs clippy lints
* More sensible default
---------
Co-authored-by: Boppy <no-reply@boppygames.gg>
* Script for publishing crates to crates.io
* Added sdk to the publish-crates script, renamed client-sdk to sdk
* Updated Cargo.lock and added 2 crates to the upgrade version script
* Small fix
* Actual fix
* Updated client API messages crate to be publishable
* Added LICENSE file to sdk
* Chose specific version for tokio-tungstenite
---------
Co-authored-by: Boppy <no-reply@boppygames.gg>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Tyler Cloutier