# Description of Changes
Move anything running on `spacetimedb-runner` to
`spacetimedb-new-runner`
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This commit adds several new metrics to `DB_METRICS` for tracking
procedures' HTTP requests:
- `procedure_http_request_size_bytes`.
- `procedure_http_response_size_bytes`.
- `procedure_num_http_requests`.
- `procedure_num_successful_http_requests`.
- `procedure_num_failed_http_requests`.
- `procedure_num_timeout_http_requests`.
- `procedure_num_in_progress_http_requests`.
See help strings in `crates/datastore/src/db_metrics/mod.rs` for details
on what each of these tracks.
Closes#3712 .
# API and ABI breaking changes
N/a - I don't think we count metrics as a stable API.
# Expected complexity level and risk
2, I guess? If we intend to use these for billing, some of the choices
I've made about tracking may impact our business.
# Testing
None; I don't know how to test Prometheus metrics.
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Rust SDK test suite for views
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
This patch only adds tests, it does not change functionality.
# Description of Changes
Implements the C# module bindings for Procedures (#3510)
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
- [X] Locally tested against existing C# Procedures regression test
client (minus the Transaction and HTTP portions, as those are not in
this change).
# Description of Changes
There were mentions of `hashbrown` in the repo that did not go through
`spacetimedb_data_structures::map`.
This caused compile errors on master when running certain tests locally.
These have been replaced with the proper imports.
The PR also bump hashbrown to 0.16.1 and foldhash to 0.2.0.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
Covered by existing tests.
# Description of Changes
View tables have private metadata columns that need to be dropped before
sending results to clients. Before this patch we dropped these columns
for sql queries and initial subscriptions, but we didn't drop them after
incremental update which is what this patch does.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Smoketest
# Description of Changes
Provides a fetch-alike API on `ctx.http`. I guess it could just be
`ctx.fetch()` instead of `ctx.http.fetch()`, but I'm not sure if that's
a good idea.
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Need to verify that this actually works
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
Title.
Closes#3644 .
# API and ABI breaking changes
Moves an unreleased HTTP route to make it explicitly unstable.
# Expected complexity level and risk
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Used `curl` to hit the previous, not-unstable-looking route, got a
404.
- [x] Used `curl` to hit the new, obviously-unstable route, got a proper
response.
# Description of Changes
Our automated discord posts have a more explicit call to action for the
PR author.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Did a test post and it looks fine
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Stop the CLI doc check from failing on `Cargo.lock` diffs, which it was
doing before.
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI still fails elsewhere if `Cargo.lock` needs updating
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Implements `__call_procedure__` in the TS bindings and host.
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
This reverts commit 53f692dec6 (#3568)
That PR broke the build if any of the directories used as templates are
not clean. It spammed lots of errors like:
```
error: `C:\Users\boppy\clockwork\SpacetimeDB\crates\cli/.templates/parent_parent_sdks_csharp_examples~_quickstart-chat_client/obj~/Debug/net8.0/client.pdb` wasn't a utf-8 file
--> C:\Users\boppy\clockwork\SpacetimeDB\crates\cli/.templates/parent_parent_sdks_csharp_examples~_quickstart-chat_client/obj~/Debug/net8.0/client.pdb:0:49
|
error: `C:\Users\boppy\clockwork\SpacetimeDB\crates\cli/.templates/parent_parent_sdks_csharp_examples~_quickstart-chat_client/obj~/Debug/net8.0/ref/client.dll` wasn't a utf-8 file
--> C:\Users\boppy\clockwork\SpacetimeDB\crates\cli/.templates/parent_parent_sdks_csharp_examples~_quickstart-chat_client/obj~/Debug/net8.0/ref/client.dll:0:3
|
error: `C:\Users\boppy\clockwork\SpacetimeDB\crates\cli/.templates/parent_parent_sdks_csharp_examples~_quickstart-chat_client/obj~/Debug/net8.0/refint/client.dll` wasn't a utf-8 file
--> C:\Users\boppy\clockwork\SpacetimeDB\crates\cli/.templates/parent_parent_sdks_csharp_examples~_quickstart-chat_client/obj~/Debug/net8.0/refint/client.dll:0:3
```
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] `cargo build` passes on my repo now, which it wasn't before this
commit
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We technically don't need to `git ls-files` when getting the CLI
templates because when we compile the release builds we work with a
clean git clone, thus it's unlikely any untracked files will be created
in the templates directory.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] I've manually checked that the init command still generates the
templates properly
# Description of Changes
Currently, we have a metric for reducer panics called
`spacetime_worker_wasm_instance_errors_total`. This commit adds a metric
for tracking errors returned from the module, like for example an Err
result in Rust, or throwing a SenderError in TypeScript
This will be needed for the web database overview dashboard, but it
should be useful for tracking sender errors in general.
# Expected complexity level and risk
1
# Testing
- [x] I've tested the change locally verifying the counter increments
when an instance returns an error
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
We have some metrics measuring various lower level things like index
scans, but at the moment we have no easy way to figure out which columns
might need an index. This commit introduces three new metrics that
provide that information by labeling count, latency, and number of rows
canned along with the scan type (index scan, table scan, mixed scan) and
info about unindexed columns.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
I'm honestly not sure. I don't think it's overly complex, but it adds
some overhead in the subscriptions initial query path.
# Testing
- [x] I've tested the changes locally
# Description of Changes
Precise index readsets
fixes #https://github.com/clockworklabs/SpacetimeDBPrivate/issues/2118
# API and ABI breaking changes
NA
# Expected complexity level and risk
2.5
Potential to regress performance.
# Testing
Updated smoketests.
# Description of Changes
Closes: #3535
Updated the Unreal SDK to handle procedures and procedure callbacks as
closely matched to Rust + C#.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - This adds a new testing frame that should be removed once procedures
are handled in /modules/sdk-test.
# Testing
Added a mirror of /sdks/unreal/tests/TestClient to use the new
/modules/sdk-test-procedure which adds complexity we'll want to remove.
- [x] Add Unreal client test of sdk-test-procedure
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This adds a github actions workflow for attaching artifacts to a
release. After this PR merges it can manually be invoked in the github
actions page.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - This can't really break anything other than the release artifacts
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
It worked when I tested it just now. Look at the release here:
https://github.com/clockworklabs/SpacetimeDB/releases
Plus you can check the job output here:
https://github.com/clockworklabs/SpacetimeDB/actions/runs/19578784486/job/56071106846?pr=3724
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Upgrade to version 1.9.0.
# API and ABI breaking changes
None - just a version upgrade.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that the license has been updated
- [x] The version number looks correct (1.9.0)
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes: #3533
Updated the C# SDK to handle procedures and procedure callbacks in a
similar fashion to the Rust client as well as added the codegen to
support it.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - This adds a new testing frame that should be removed once procedures
are handled with C# module bindings
# Testing
Added /sdks/csharp/examples~/regression-tests/procedure-client to match
modules/sdk-test-procedure which we can roll into the standard
regression-tests once C# supports the procedure attribute.
- [x] Add C# client test of sdk-test-procedure
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
Follow up to #3684. Moves `Error` and `Timeout` out of lib, so that we
don't have to implement `SpacetimeType` for them, and then removes the
http dependency altogether, so that `lib` can be leaner. I also got rid
of the separate `HttpValue` type, since it only really exists to mirror
the `http` crate and typescript won't make use of it.
# Expected complexity level and risk
1
# Testing
n/a - just code movement.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- This has 2 benefits:
1. `ubuntu-latest` recently changed to 24.04 instead of 22.04 so that
broke the last CLI on some systems. This will fix that issue.
2. Performance - this should build much much faster.
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
2 - if the CLI fails to build this might be why.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- [x] Tested via today's release:
https://github.com/clockworklabs/SpacetimeDB/actions/runs/19550648024/job/55980960439
# Description of Changes
I thought this was covered by the `NEUTRAL` case, but apparently
`SKIPPED` is separate.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
We'll see when it merges 🤷
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes#3517 .
With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.
Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.
I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.
This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close#3644 .
Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
rather than running it on the single-threaded database executor.
# API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.
# Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.
# Testing
- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
This modifies the environment variable that we pass in for requiring
SpacetimeAuth to publish so that we can put different issuer values for
live and staging.
# API and ABI breaking changes
This requires a new environment variable or it skips this requirement.
# Expected complexity level and risk
1
# Testing
I have not tested this change locally.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
These tests have been flaky, so we're disabling them until they're
working more consistently.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Tests are indeed skipped? 🤷
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Due to a limitation around passing headers to a WebSocket connection,
The typescript SDK rely on the endpoint `/v1/identity/websocket-token`
to get a new, short-lived token.
Currently, this endpoint strips all the other claims from the token and
only returns the following claims:
- `hex_identity`
- `sub`
- `iss`
- `aud`
- `iat`
- `exp`
This PR aims to fix this issue by introducing a new member field `extra`
to `SpacetimeIdentityClaims` and `TokenClaims` and letting serde do its
job.
# API and ABI breaking changes
None
# Expected complexity level and risk
2 - The change is trivial (1) but I'm not 100% familiar with all the
places where we would be signing a token (1).
# Testing
1. `curl` the endpoint and checking that the token returned contains all
the expected claims
2. Check that that the endpoint `v1/identity` still correctly issues and
identity and token
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Change from `tokio::spawn` task to blocking thread for Views cleanup
tasks.
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
# Description of Changes
1. Updates the Replication Tests in
`sdks/csharp/examples~/regression-tests` to include better coverage of
Views
2. Added missing linkage for __call_view__ and __call_view_anon__
3. Updated *ViewDispatcher Invoke to transform BSATN.ValueOption<> into
BSATN.List<>
4. Fixed issues with the indexing of views to match correctly during
__call_view__ and __call_view_anon__
# API and ABI breaking changes
No
# Expected complexity level and risk
2
# Testing
- [x] Running `run-regression-tests.sh` passes.
---------
Signed-off-by: rekhoff <r.ekhoff@clockworklabs.io>
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
A background task to cleanup unsubscribed views.
fixes#3587
# API and ABI breaking changes
NA
# Expected complexity level and risk
2
# Testing
Added a test
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
There are two purposes for introducing the changes in this commit:
1. Decrease the cardinality of the metric, as the caller id and
connection id cwould result in way too combiations
2. Add a database_identity label, which is much more useful for the
metric
# API and ABI breaking changes
I'm honestly not sure if metrics labels change is an API breaking
change, I suppose not? For what it's worth, I don't think that anyone
was relying on the labels I've removed.
# Expected complexity level and risk
1
# Testing
- [x] I've tested the change locally
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Make View backing tables and related St tables not persistent.
1. Modifies `CommittedState` to hold set of ephemeral tables.
2. Update `TxData` to contain a subset of ephemeral tables which has
been modified in current transaction.
`do_durability` filter those table out before writting the transaction
to commitlog.
depends on: https://github.com/clockworklabs/SpacetimeDB/pull/3651
# API and ABI breaking changes
NA
# Expected complexity level and risk
2.5.
looks simple but changes comes in the hotpath, I ensured we don't do
unneccessary heap allocations but patch has the potential to regress
perfomance.
# Testing
- unit test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
We had a few issues with `smoketests --list`. I wanted to improve this
behavior in preparation for using it (or similar logic) to parallelize
the unit tests.
First of all, the `--list` logic ran _after_ our calls to `cargo build`
and `spacetime login`, so it could take a while to run tests _and_ you
would need a server running just to list tests.
Also, if any tests failed to import, it would give a cryptic error and
exit in the middle of the list:
```
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/__main__.py", line 169, in <module>
main()
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/__main__.py", line 151, in main
for test in itertools.chain(*itertools.chain(*tests)):
TypeError: '_FailedTest' object is not iterable
```
Now, it completes the entire list regardless, and prints a much better
clearer message:
```
Failed to construct unittest.loader._FailedTest.pg_wire:
ImportError: Failed to import test module: pg_wire
Traceback (most recent call last):
File "/usr/lib/python3.12/unittest/loader.py", line 137, in loadTestsFromName
module = __import__(module_name)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/tests/pg_wire.py", line 5, in <module>
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
```
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Existing CI still passes
- [x] Running `python -m smoketests --list` no longer tries to build
SpacetimeDB or log in to a server
- [x] Running `python -m smoketests --list` has a (much) more
descriptive error if there are broken imports
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes: #3658
With the work on Views we ran into trouble with the Return Type and
certain C# code layout where the types are defined inside the Module
class. This fixes the bug in our current approach but needs
consideration to possibly be changed entirely.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Minor change
# Testing
- [x] Retested with the linked code with and without the namespace
- [x] Re-ran regression tests
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
We have added one too many python dependencies. I'm putting them in a
shared file, which will also ease installation for devs running them
locally.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
Existing CI still passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- This just moves the typescript tests onto the custom runner.
|Test|Before|After|% Change|
|---|---|---|---|
|Typescript - Lint|1m|<1m|No change|
|Typescript - Build + Test|14m|5m|64%|
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - if the typescript tests become flaky this could be the reason.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- [x] The typescript CI still passes and is faster.
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
The `spacetime dev` and `spacetime generate` commands always overwrite
the `.env.local` and `module_bindings` files, even if the contents are
identical. This can be a nuisance if your client is watching for changes
and reloading, as `spacetime dev` will modify both of those at different
times in the process.
This change will first read the contents of the files and see if a write
is necessary.
For writing `.env.local` as part of `spacetime dev`, the file contents
were already read so no real impact should exist.
For `module_bindings`, I've done some initial profiling with 181
generated typescript files and separately 161 rust files and noticed no
performance impact.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
I've ran this code on a project that generates 181 source files for
typescript and 161 for rust. It correctly writes files when changes have
been made and leaves the files unmodified if no changes are made.
For review testing, I would recommend running both with and without my
change on a much more complex project if it's available. I would think
not writing the contents of each file would mitigate the cost to read
most of the time, but perhaps in certain project makeup scenarios and
storage performance characteristics, this could possible be marginally
slower sometimes, though I have not observed that to be the case.
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Adds `ProcedureContext::{with_tx, try_with_tx}`.
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3515.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
An integration test `test_calling_with_tx` is added.
# Description of Changes
Changes with cache hit:
CI / Smoketests (spacetimedb-runner): 17m -> 10m
CI / Smoketests (windows-latest): 39m -> 31m
CI / Lints: 3m30s -> 2m
unity-testsuite: 23m -> 19m
This included creating a stripped-down `Dockerfile` that only runs
SpacetimeDB (as opposed to e.g. building tracy, flamegraphs, etc.). That
shaves off about 4-5m on the linux smoketests job.
# API and ABI breaking changes
No. CI only.
# Expected complexity level and risk
2
# Testing
Ran existing CI and noted the change in times in the description above
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Fixes client codegen for views.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
I'm not sure what tests we have for C++.
- [x] Updated client snapshots for rust, C#, and typescript
- [ ] Rust sdk test
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
This PR modifies the `--delete-data` flag on `spacetime publish` and
adds the `--delete-data` flag on `spacetime dev`.
In particular instead of `--delete-data` being a boolean, it is now a an
enum:
- `always` -> corresponds to the old value of `true`
- `never` -> corresponds to the old value of `false`
- `on-conflict` -> clears the database, but only if publishing would
have required a manual migration
This flag does NOT change any behavior about prompting users to confirm
if they want to delete the data. Users will still be prompted to confirm
UNLESS they pass the separate `--yes` flag.
`spacetime dev` gets the same `--delete-data` flag. The default value of
`never` is equivalent to the existing behavior. `spacetime dev`
continues to publish with `--yes` just as before. This behavior is
unchanged.
# API and ABI breaking changes
Adds the flags specified above. This is NOT a breaking change to the
CLI. Passing `--delete-data` is the equivalent of
`--delete-data=always`.
This IS technically a breaking change to the `pre_publish` route. As far
as I'm aware this is *only* used by our CLI however.
> IMPORTANT SIDE NOTE: I would argue that `--break-clients` should
really be renamed to `--yes-break-clients` because it actually behaves
like the `--yes` force flag, but only for a subset of the user prompts.
I have not made this change because it would be a breaking change, but
if the reviewers agree, I will make this change.
# Expected complexity level and risk
2, Very small change, but if we get it wrong users could accidentally
lose data. I would ask reviewers to think about ways that users might
accidentally pass `--delete-data --yes`.
# Testing
- [ ] I have not yet tested manually.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Apparently, when `spacetime generate` passes files to a linter, the
files are passed in as relative paths, which did not play properly with
`dotnet format` (so we effectively borked it).
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] Unity testsuite no longer fails on expected diffs
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This PR is a very large change to the workings of the TypeScript SDK and
as such requires a higher bar of testing than other PRs. However, it
does several important things:
1. Unifies the API of the server and client so they not only have the
same API, but they actually implement it with the same TypeScript types.
This fixes several inconsistencies between them and fixes several small
bugs as well.
2. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3365
3. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3431
4. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3435
5. Subsumes the work done in
https://github.com/clockworklabs/SpacetimeDB/pull/3447
6. Derives all type information on the client from a single
`RemoteModule` type which vastly cleans up the correctness of type
checking on the client and helped me to find several small bugs
It accomplishes this by changing code generation of TypeScript on the
client to code generation approximately what a developer would manually
write in their module. The ultimate goal would be to allow the developer
to use the types and functions that they define on in their module
directly on the client without needing to do any code generation at all,
provided they are using TypeScript on the server and client.
https://github.com/clockworklabs/SpacetimeDB/issues/3365 is resolved by
`.build()`ing the `DbConnection` inside a React `useEffect` rather than
doing it directly in line with the render of the provider. In order to
do that we needed to not expose the `DbConnection` directly to
developers by returning a different type from `useSpacetimeDB`.
`useSpacetimeDB` now returns a `ConnectionState` object which is stored
as React state and updates when any of the fields change. This change
also resolves https://github.com/clockworklabs/SpacetimeDB/issues/3431.
https://github.com/clockworklabs/SpacetimeDB/issues/3435 was the issue
that initially lead me down the rabbit hole of unifying the server and
the client because it was nearly impossible to track down all the
various type functions and how they connect to the values that we code
generate on the server. After several hours of attempting this, I
decided to clean up the types a bit to be more uniform.
Implementing the unification between the client and the server also
necessitated fully implemented parts of the API that were fully
implemented on the server, but were broken or missing on the client.
# API and ABI breaking changes
[Unification] -> Means that this is breaking behavior for the client
SDK, but that the new behavior is identical to the server's existing
behavior
## Breaking changes:
- Table accessor names and index accessor names are converted to
camelCase on the `ctx`, so `ctx.db.foo_bar` is now `ctx.db.fooBar`
- [Unification] On the client `my_table.iter()` returns
`IterableIterator` instead of an `Array`
- [Unification] `module_bindings` now export `TypeBuilder`s for all
types instead of a `type MyType` and object `MyType`, so instead of
using `MyType` as a type directly, you need to infer the type `MyType`
-> `Infer<typeof MyType>`.
- [Unification] We no longer generate and export `MyTypeVariants` for
sum types (these are now accessed by `Infer<typeof
MyType.variants.myVariant>`)
- [Unification] `MyType.getTypeScriptAlgebraicType()` has been replaced
with `MyType.algebraicType`
- `useSpacetimeDB()` no longer takes type parameters
- `useTable()` now takes a `TableDef` parameter and type params are
inferred
- `useTable()` now just returns an `Array` directly instead of a object
with `{ rows }`
- [Unification] `ctx.reducers.createPlayer(argA, argB)` ->
`ctx.reducers.createPlayer({ argA, argB })`
- [Unification] `ctx.reducers.onCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.onCreatePlayer(ctx, { argA, argB })`
- [Unification] `ctx.reducers.removeOnCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.removeOnCreatePlayer(ctx, { argA, argB })`
- [Unification] `myTable.count(): number` -> `myTable.count(): bigint`
## Additive changes:
- `Infer<>` now also does `InferTypeOfRow<>` if applicable
- Added a `useReducer()` React hook
- `module_bindings` now exports a `tables` object with references to all
the `TableDef`s
- `module_bindings` now exports a `reducers` object with references to
all the `ReducerDef`s
- Added a new `MyType.create('MyVariant', ...)` function in addition to
the `MyType.MyVariant(...)` constructors (this is private)
## Notable things that did not change:
- `MyType.serialize(writer: BinaryWriter, value: Infer<typeof MyType>)`
and `MyType.deserialize(reader: BinaryReader): Infer<typeof MyType>` are
still supported exactly as before.
- The `MyType.MyVariant(...)` constructor function on sum types is still
present, but implemented with the private `MyType.create('MyVariant',
...)`. We could choose to move away from this API later if we didn't
like the variants polluting the namespace
# Expected complexity level and risk
4 - This is a deep reaching an complex change for the SDK. For the
server, it is much less deep reaching since it reuses much of the same
machinery, although it does require thorough testing there as some of
the code was modified.
This change is fully localized to TypeScript and does not touch the host
(or other languages) at all, and therefore only impacts a beta aspect of
SpacetimeDB.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Added regression test for
https://github.com/clockworklabs/SpacetimeDB/issues/3435
- [x] Manually tested `test-app` and `test-react-router-app`
- [ ] Add test cases for camelCase-ing
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Fixes a subscription plan caching bug related to client-specific views.
Before this fix, you could define a client-specific view:
```rust
fn my_view(ctx: &ViewContext) -> Option<Player> {
ctx.db.player().identity().find(ctx.sender)
}
```
And subscribe to it as follows:
```sql
SELECT * FROM my_view
```
Note this view is implicitly parameterized by `:sender`, however when
generating a query hash for this subscription, this fact would not be
taken into account which would result in this query being cached and
reused for all callers.
After this fix, a query hash is generated for this subscription as
though it were given as:
```sql
SELECT * FROM my_view WHERE identity = :sender
```
# API and ABI breaking changes
None
**Note for CLI code owners:**
I had to touch the `subscribe` cli command file. No updates to the api.
It just needed to be updated to look for views in the module def.
# Expected complexity level and risk
1
# Testing
- [x] Added a regression smoketest
# Description of Changes
When the author isn't in the lookup table, we were getting `null`
instead of the empty string. This fixes that.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
None
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
`spacetime generate` calls `dotnet format`.. but it did so with
`--folder`, without going to any particular directory. The result would
be that `dotnet format` tries to format _whatever directory it was run
from.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] `python3 -m smoketests namespaces` no longer hangs on my machine
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
It is almost always wrong / undesirable to pack more than one
transaction in a commit, so adjust the default accordingly.
This also avoids surprises when using `#[serde(default)]` with nested
structs -- serde evaluates the default depth-first, so overriding a
single field in a nested struct will not consider any
`#[serde(default = "custom_default")]` annotations on the parent.
Zeke Foppa