# Description of Changes
These tests have been flaky, so we're disabling them until they're
working more consistently.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Tests are indeed skipped? 🤷
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Due to a limitation around passing headers to a WebSocket connection,
The typescript SDK rely on the endpoint `/v1/identity/websocket-token`
to get a new, short-lived token.
Currently, this endpoint strips all the other claims from the token and
only returns the following claims:
- `hex_identity`
- `sub`
- `iss`
- `aud`
- `iat`
- `exp`
This PR aims to fix this issue by introducing a new member field `extra`
to `SpacetimeIdentityClaims` and `TokenClaims` and letting serde do its
job.
# API and ABI breaking changes
None
# Expected complexity level and risk
2 - The change is trivial (1) but I'm not 100% familiar with all the
places where we would be signing a token (1).
# Testing
1. `curl` the endpoint and checking that the token returned contains all
the expected claims
2. Check that that the endpoint `v1/identity` still correctly issues and
identity and token
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Change from `tokio::spawn` task to blocking thread for Views cleanup
tasks.
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
# Description of Changes
1. Updates the Replication Tests in
`sdks/csharp/examples~/regression-tests` to include better coverage of
Views
2. Added missing linkage for __call_view__ and __call_view_anon__
3. Updated *ViewDispatcher Invoke to transform BSATN.ValueOption<> into
BSATN.List<>
4. Fixed issues with the indexing of views to match correctly during
__call_view__ and __call_view_anon__
# API and ABI breaking changes
No
# Expected complexity level and risk
2
# Testing
- [x] Running `run-regression-tests.sh` passes.
---------
Signed-off-by: rekhoff <r.ekhoff@clockworklabs.io>
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
A background task to cleanup unsubscribed views.
fixes#3587
# API and ABI breaking changes
NA
# Expected complexity level and risk
2
# Testing
Added a test
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
There are two purposes for introducing the changes in this commit:
1. Decrease the cardinality of the metric, as the caller id and
connection id cwould result in way too combiations
2. Add a database_identity label, which is much more useful for the
metric
# API and ABI breaking changes
I'm honestly not sure if metrics labels change is an API breaking
change, I suppose not? For what it's worth, I don't think that anyone
was relying on the labels I've removed.
# Expected complexity level and risk
1
# Testing
- [x] I've tested the change locally
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Make View backing tables and related St tables not persistent.
1. Modifies `CommittedState` to hold set of ephemeral tables.
2. Update `TxData` to contain a subset of ephemeral tables which has
been modified in current transaction.
`do_durability` filter those table out before writting the transaction
to commitlog.
depends on: https://github.com/clockworklabs/SpacetimeDB/pull/3651
# API and ABI breaking changes
NA
# Expected complexity level and risk
2.5.
looks simple but changes comes in the hotpath, I ensured we don't do
unneccessary heap allocations but patch has the potential to regress
perfomance.
# Testing
- unit test.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
We had a few issues with `smoketests --list`. I wanted to improve this
behavior in preparation for using it (or similar logic) to parallelize
the unit tests.
First of all, the `--list` logic ran _after_ our calls to `cargo build`
and `spacetime login`, so it could take a while to run tests _and_ you
would need a server running just to list tests.
Also, if any tests failed to import, it would give a cryptic error and
exit in the middle of the list:
```
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/__main__.py", line 169, in <module>
main()
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/__main__.py", line 151, in main
for test in itertools.chain(*itertools.chain(*tests)):
TypeError: '_FailedTest' object is not iterable
```
Now, it completes the entire list regardless, and prints a much better
clearer message:
```
Failed to construct unittest.loader._FailedTest.pg_wire:
ImportError: Failed to import test module: pg_wire
Traceback (most recent call last):
File "/usr/lib/python3.12/unittest/loader.py", line 137, in loadTestsFromName
module = __import__(module_name)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/tests/pg_wire.py", line 5, in <module>
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
```
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Existing CI still passes
- [x] Running `python -m smoketests --list` no longer tries to build
SpacetimeDB or log in to a server
- [x] Running `python -m smoketests --list` has a (much) more
descriptive error if there are broken imports
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes: #3658
With the work on Views we ran into trouble with the Return Type and
certain C# code layout where the types are defined inside the Module
class. This fixes the bug in our current approach but needs
consideration to possibly be changed entirely.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Minor change
# Testing
- [x] Retested with the linked code with and without the namespace
- [x] Re-ran regression tests
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
We have added one too many python dependencies. I'm putting them in a
shared file, which will also ease installation for devs running them
locally.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
Existing CI still passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- This just moves the typescript tests onto the custom runner.
|Test|Before|After|% Change|
|---|---|---|---|
|Typescript - Lint|1m|<1m|No change|
|Typescript - Build + Test|14m|5m|64%|
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - if the typescript tests become flaky this could be the reason.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
- [x] The typescript CI still passes and is faster.
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
The `spacetime dev` and `spacetime generate` commands always overwrite
the `.env.local` and `module_bindings` files, even if the contents are
identical. This can be a nuisance if your client is watching for changes
and reloading, as `spacetime dev` will modify both of those at different
times in the process.
This change will first read the contents of the files and see if a write
is necessary.
For writing `.env.local` as part of `spacetime dev`, the file contents
were already read so no real impact should exist.
For `module_bindings`, I've done some initial profiling with 181
generated typescript files and separately 161 rust files and noticed no
performance impact.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
I've ran this code on a project that generates 181 source files for
typescript and 161 for rust. It correctly writes files when changes have
been made and leaves the files unmodified if no changes are made.
For review testing, I would recommend running both with and without my
change on a much more complex project if it's available. I would think
not writing the contents of each file would mitigate the cost to read
most of the time, but perhaps in certain project makeup scenarios and
storage performance characteristics, this could possible be marginally
slower sometimes, though I have not observed that to be the case.
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Adds `ProcedureContext::{with_tx, try_with_tx}`.
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3515.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
An integration test `test_calling_with_tx` is added.
# Description of Changes
Changes with cache hit:
CI / Smoketests (spacetimedb-runner): 17m -> 10m
CI / Smoketests (windows-latest): 39m -> 31m
CI / Lints: 3m30s -> 2m
unity-testsuite: 23m -> 19m
This included creating a stripped-down `Dockerfile` that only runs
SpacetimeDB (as opposed to e.g. building tracy, flamegraphs, etc.). That
shaves off about 4-5m on the linux smoketests job.
# API and ABI breaking changes
No. CI only.
# Expected complexity level and risk
2
# Testing
Ran existing CI and noted the change in times in the description above
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Fixes client codegen for views.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
I'm not sure what tests we have for C++.
- [x] Updated client snapshots for rust, C#, and typescript
- [ ] Rust sdk test
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
This PR modifies the `--delete-data` flag on `spacetime publish` and
adds the `--delete-data` flag on `spacetime dev`.
In particular instead of `--delete-data` being a boolean, it is now a an
enum:
- `always` -> corresponds to the old value of `true`
- `never` -> corresponds to the old value of `false`
- `on-conflict` -> clears the database, but only if publishing would
have required a manual migration
This flag does NOT change any behavior about prompting users to confirm
if they want to delete the data. Users will still be prompted to confirm
UNLESS they pass the separate `--yes` flag.
`spacetime dev` gets the same `--delete-data` flag. The default value of
`never` is equivalent to the existing behavior. `spacetime dev`
continues to publish with `--yes` just as before. This behavior is
unchanged.
# API and ABI breaking changes
Adds the flags specified above. This is NOT a breaking change to the
CLI. Passing `--delete-data` is the equivalent of
`--delete-data=always`.
This IS technically a breaking change to the `pre_publish` route. As far
as I'm aware this is *only* used by our CLI however.
> IMPORTANT SIDE NOTE: I would argue that `--break-clients` should
really be renamed to `--yes-break-clients` because it actually behaves
like the `--yes` force flag, but only for a subset of the user prompts.
I have not made this change because it would be a breaking change, but
if the reviewers agree, I will make this change.
# Expected complexity level and risk
2, Very small change, but if we get it wrong users could accidentally
lose data. I would ask reviewers to think about ways that users might
accidentally pass `--delete-data --yes`.
# Testing
- [ ] I have not yet tested manually.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Apparently, when `spacetime generate` passes files to a linter, the
files are passed in as relative paths, which did not play properly with
`dotnet format` (so we effectively borked it).
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] Unity testsuite no longer fails on expected diffs
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This PR is a very large change to the workings of the TypeScript SDK and
as such requires a higher bar of testing than other PRs. However, it
does several important things:
1. Unifies the API of the server and client so they not only have the
same API, but they actually implement it with the same TypeScript types.
This fixes several inconsistencies between them and fixes several small
bugs as well.
2. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3365
3. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3431
4. Closes https://github.com/clockworklabs/SpacetimeDB/issues/3435
5. Subsumes the work done in
https://github.com/clockworklabs/SpacetimeDB/pull/3447
6. Derives all type information on the client from a single
`RemoteModule` type which vastly cleans up the correctness of type
checking on the client and helped me to find several small bugs
It accomplishes this by changing code generation of TypeScript on the
client to code generation approximately what a developer would manually
write in their module. The ultimate goal would be to allow the developer
to use the types and functions that they define on in their module
directly on the client without needing to do any code generation at all,
provided they are using TypeScript on the server and client.
https://github.com/clockworklabs/SpacetimeDB/issues/3365 is resolved by
`.build()`ing the `DbConnection` inside a React `useEffect` rather than
doing it directly in line with the render of the provider. In order to
do that we needed to not expose the `DbConnection` directly to
developers by returning a different type from `useSpacetimeDB`.
`useSpacetimeDB` now returns a `ConnectionState` object which is stored
as React state and updates when any of the fields change. This change
also resolves https://github.com/clockworklabs/SpacetimeDB/issues/3431.
https://github.com/clockworklabs/SpacetimeDB/issues/3435 was the issue
that initially lead me down the rabbit hole of unifying the server and
the client because it was nearly impossible to track down all the
various type functions and how they connect to the values that we code
generate on the server. After several hours of attempting this, I
decided to clean up the types a bit to be more uniform.
Implementing the unification between the client and the server also
necessitated fully implemented parts of the API that were fully
implemented on the server, but were broken or missing on the client.
# API and ABI breaking changes
[Unification] -> Means that this is breaking behavior for the client
SDK, but that the new behavior is identical to the server's existing
behavior
## Breaking changes:
- Table accessor names and index accessor names are converted to
camelCase on the `ctx`, so `ctx.db.foo_bar` is now `ctx.db.fooBar`
- [Unification] On the client `my_table.iter()` returns
`IterableIterator` instead of an `Array`
- [Unification] `module_bindings` now export `TypeBuilder`s for all
types instead of a `type MyType` and object `MyType`, so instead of
using `MyType` as a type directly, you need to infer the type `MyType`
-> `Infer<typeof MyType>`.
- [Unification] We no longer generate and export `MyTypeVariants` for
sum types (these are now accessed by `Infer<typeof
MyType.variants.myVariant>`)
- [Unification] `MyType.getTypeScriptAlgebraicType()` has been replaced
with `MyType.algebraicType`
- `useSpacetimeDB()` no longer takes type parameters
- `useTable()` now takes a `TableDef` parameter and type params are
inferred
- `useTable()` now just returns an `Array` directly instead of a object
with `{ rows }`
- [Unification] `ctx.reducers.createPlayer(argA, argB)` ->
`ctx.reducers.createPlayer({ argA, argB })`
- [Unification] `ctx.reducers.onCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.onCreatePlayer(ctx, { argA, argB })`
- [Unification] `ctx.reducers.removeOnCreatePlayer(ctx, argA, argB)` ->
`ctx.reducers.removeOnCreatePlayer(ctx, { argA, argB })`
- [Unification] `myTable.count(): number` -> `myTable.count(): bigint`
## Additive changes:
- `Infer<>` now also does `InferTypeOfRow<>` if applicable
- Added a `useReducer()` React hook
- `module_bindings` now exports a `tables` object with references to all
the `TableDef`s
- `module_bindings` now exports a `reducers` object with references to
all the `ReducerDef`s
- Added a new `MyType.create('MyVariant', ...)` function in addition to
the `MyType.MyVariant(...)` constructors (this is private)
## Notable things that did not change:
- `MyType.serialize(writer: BinaryWriter, value: Infer<typeof MyType>)`
and `MyType.deserialize(reader: BinaryReader): Infer<typeof MyType>` are
still supported exactly as before.
- The `MyType.MyVariant(...)` constructor function on sum types is still
present, but implemented with the private `MyType.create('MyVariant',
...)`. We could choose to move away from this API later if we didn't
like the variants polluting the namespace
# Expected complexity level and risk
4 - This is a deep reaching an complex change for the SDK. For the
server, it is much less deep reaching since it reuses much of the same
machinery, although it does require thorough testing there as some of
the code was modified.
This change is fully localized to TypeScript and does not touch the host
(or other languages) at all, and therefore only impacts a beta aspect of
SpacetimeDB.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Added regression test for
https://github.com/clockworklabs/SpacetimeDB/issues/3435
- [x] Manually tested `test-app` and `test-react-router-app`
- [ ] Add test cases for camelCase-ing
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Fixes a subscription plan caching bug related to client-specific views.
Before this fix, you could define a client-specific view:
```rust
fn my_view(ctx: &ViewContext) -> Option<Player> {
ctx.db.player().identity().find(ctx.sender)
}
```
And subscribe to it as follows:
```sql
SELECT * FROM my_view
```
Note this view is implicitly parameterized by `:sender`, however when
generating a query hash for this subscription, this fact would not be
taken into account which would result in this query being cached and
reused for all callers.
After this fix, a query hash is generated for this subscription as
though it were given as:
```sql
SELECT * FROM my_view WHERE identity = :sender
```
# API and ABI breaking changes
None
**Note for CLI code owners:**
I had to touch the `subscribe` cli command file. No updates to the api.
It just needed to be updated to look for views in the module def.
# Expected complexity level and risk
1
# Testing
- [x] Added a regression smoketest
# Description of Changes
When the author isn't in the lookup table, we were getting `null`
instead of the empty string. This fixes that.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
None
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
`spacetime generate` calls `dotnet format`.. but it did so with
`--folder`, without going to any particular directory. The result would
be that `dotnet format` tries to format _whatever directory it was run
from.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] `python3 -m smoketests namespaces` no longer hangs on my machine
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
It is almost always wrong / undesirable to pack more than one
transaction in a commit, so adjust the default accordingly.
This also avoids surprises when using `#[serde(default)]` with nested
structs -- serde evaluates the default depth-first, so overriding a
single field in a nested struct will not consider any
`#[serde(default = "custom_default")]` annotations on the parent.
# Description of Changes
The `config` added `maincloud` in the constructors so the test need
update.
# Expected complexity level and risk
1
# Testing
- [x] Re-run the test until pass
Allows the SpacetimeDBClient to be configured with or without confirmed
reads. Like for [TypeScript], the parameter is optional -- the server
chooses the default if not set explicitly.
[TypeScript]: https://github.com/clockworklabs/SpacetimeDB/pull/3247
# Expected complexity level and risk
1
# Testing
I don't actually know what I'm doing 😅
# Description of Changes
This template was never properly tested, and it contains wrong API usage
that can't even compile.
The only problem is that in order to make it work the person table has
to be public, and I'm not sure if we want to make that change.
Alternatively we could make basic-rust code not subscribe to anything
and just connect to the database.
# Expected complexity level and risk
2
# Testing
- [x] I compiled the template and checked that it works against
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
`Internal Tests` invoke another workflow. If the calling workflow is
cancelled, now we also cancel the invoked workflow.
# API and ABI breaking changes
None.
# Expected complexity level and risk
2
# Testing
- [x] Push multiple commits to this PR, see that the invoked job is
cancelled
- [x] Manually cancel the `Internal Tests` job, see that the invoked job
is cancelled
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Speed improvements:
|CI Job|Before|After|Change|
|---|---|---|---|
|CI/Unreal Engine|29m|19m|34% faster|
|CI/Testsuite|21m|11m|47% faster|
|C#/Unity|20m|16m|20% faster|
|CI/Build and test wasm bindings|10m|3m|70% faster|
|CI/Smoketests (linux)|17m|12m|29% faster|
CI/Unreal Engine - switch to serial test execution
CI/Testsuite - partially switch to serial test execution
I also haven't seen an unexpected `Operation cancelled`, so I think that
problem might be behind us.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
No user facing changes
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
2 - this touches a lot of our CI. It has been solid so far but if we
start getting random CI failures this could be the cause.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] All CI passes
(ok as of this update the internal tests are failing but that isn't
related to this PR 😅 )
# Future Work
There are 2 remaining tests that could be optimized by moving them to
faster runners:
1. (~45m) The windows smoketests (difficult because it requires
Windows). @bfops has brought up the idea of only running this test on
PRs that are in the merge queue and I think that would be the best 80/20
here. There really is not a great reason to be running this on every
commit anyway.
2. (36m) The internal tests (easier because it's on Linux)
I would also consider moving from instantiating container images to
instantiating VM snapshots. This would allow us a lot more flexibility
and we wouldn't have to have as much container config in our workflow
files. Based on guides I've seen online this is also somewhat common.
Also managing docker within docker has been kind of a nightmare so it
would be easier to run containers inside of VMs to remove a layer of
containerization there.
## Testsuite inconsistencies
We have at least 2 different jobs that are inconsistent/flaky: the
"testsuite" and the "unreal engine tests".
I'd like @JasonAtClockwork to make the unreal engine tests run
sequentially so that we have less random failures. I know this will
increase the testing time but due to this PR we should have extra
headroom there.
I've also pinged Joshua about one of the tests that I've seen sometimes
fails randomly. This is either an issue in the test or an issue in
SpacetimeDB itself, I'm not sure yet.
These inconsistencies are present both in our custom runner and in the
Github runners, so merging this PR will not make this situation any
worse than it already is.
# Description of Changes
This makes us go from 3 threads to 2.
The next step is to core pin the V8 worker thread.
# API and ABI breaking changes
None
# Expected complexity level and risk
4
# Testing
Existing tests should cover this.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
Historically, controldb reads could be treated as just a datastructure,
but that became a lie when reconnections were introduced.
Some tricks were employed to enter the async context when needed, but
those always bear the risk of introducing a deadlock somewhere.
So, just make it async.
# Expected complexity level and risk
2
It may or may not be safe to use sled in an async context. We did
already for
the write path, but if it's a problem it'll show now.
# Testing
Not a functional change.
# Description of Changes
fixes: #2882
Schedule repeated reducers from their last execution timestamp instead
of `Timestamp::now`
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
# Testing
Existing test should be enough to cover any regression.
# Description of Changes
Our `Check that packages are publishable` CI check was silently failing
due to a subshell error not getting propagated upward.
This PR both fixes that error-swallowing behavior, and fixes the errors
that were present.
# API and ABI breaking changes
No. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Check fails if the script to find crates fails
(https://github.com/clockworklabs/SpacetimeDB/actions/runs/19349995837/job/55359458732?pr=3660)
- [x] Check is now passing
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Filter out the discord notification job itself, as well as any neutral
results.
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
Unsure how to test this without merging the PR
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Switch the `Internal Tests` job to re-use the existing CI workflow
instead of a separate one.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Internal Tests passes on this PR
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Introduces a "routes struct" for the `/identity` endpoints, much like
the `DatabaseRoutes`.
This is useful for overriding individual handlers.
See companion for motivation.
Depends-on: #3525
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- This upgrades the versions of all SDKs, the CLI, etc. to 1.8.0
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I verified that all versions seem to be updated including the BSL
license update <!-- maybe a test you want to do -->
We have 1 `1.7.0` that didn't get upgraded automatically because it is
part of the module bindings for a template:
```
crates/cli/.templates/parent_parent_crates_bindings-typescript_examples_quickstart-chat/src/module_bindings/index.ts: cliVersion: '1.7.0',
```
A case could possibly be made for bumping the template but it shouldn't
cause any issues as the module bindings directory should just get
regenerated by the user. @cloutiertyler should we be bumping module
bindings for templates when we upgrade versions?
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Improves auto-migration support for views by minimizing the cases where
we must disconnect clients.
Before this patch, any schema compatible view update and even no view
updates at all would require us to disconnect clients, because we had to
assume that the view was modified thereby rendering its currently
materialized result set stale and out of date.
Patch adds a code to re-evaluate all views whose entry is in
`st_view_sub` and make `update_database` to start using
`commit_and_broadcast_event` so that now we only have to disconnect
clients for incompatible view updates or dropped views.
# API and ABI breaking changes
NA
# Expected complexity level and risk
2
# Testing
Added smoketests.
# Description of Changes
Includes Rust, C#, and TypeScript.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Just docs.
# Description of Changes
Fixes the issue described in
https://github.com/clockworklabs/SpacetimeDB/issues/3558.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Smoketests are now successful after running `cargo bump-versions
1.8.0 --typescript`.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Permissions for evaluating SQL/DML are not generally "actions", but more
a set of permissions that are checked during evaluation.
To make this work with the teams feature, this patch extends `AuthCtx`
to allow checking a set of permissions as mandated by the spec. This set
is a bit more fine-grained than "is owner", so as to avoid baking in the
concept of teams/collaborators, or assumptions about what a role might
entail. Both are likely to evolve in the future, so evaluation of
permissions / capabilities should be confined to the impl of the
`Authorization` trait.
Unlike "actions", the `AuthCtx` must be able to evaluate permission
checks quickly and without side-effects, nor can it enter an `async`
context. In that sense, it is precomputed (if you will), and stored as a
closure in the `AuthCtx` for external authorization.
A challenge posed is how to thread through the constructed `AuthCtx` for
subscriptions.
A tempting approach would have been to equip the `HostController` with
the ability to summon an `AuthCtx`. That, however, would have created a
gnarly circular dependency, because the `HostController` also controls
the controldb, which itself demands an `AuthCtx`.
Instead, the `AuthCtx` is obtained in the endpoint handler and passed to
each method call that requires one. That's less pretty, but more
effective.
---------
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
Updates C# bindings to allow module authors to define `View`s and
`AnonymousView`s using the pattern:
```
[SpacetimeDB.View]
public static ExampleData? GetExampleDataById(ViewContext ctx, uint id)
{
return ctx.Db.ExampleData.Id.Find(id);
}
[SpacetimeDB.View]
public static ExampleData? GetAnonymousExampleDataById(AnonymousViewContext ctx, uint id)
{
return ctx.Db.ExampleData.Id.Find(id);
}
```
During publishing of a C# module, the views data will be converted into
`RawViewDefV9`
# API and ABI breaking changes
No known breaking changes
# Expected complexity level and risk
2
# Testing
- [X] Locally tested locally adding the above sample pattern to the
`sdks\csharp\examples~\regression-tests\server` test and performing a
`dotnet build` and a `spacetime publish test`, both of which succeeded.
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
This adds authorization to the relevant API endpoints, updates the CLI
commands and adds smoketests for the teams feature.
**Note**: Authorizing SQL (incl. subscriptions) is a bit more involved,
and submitted as a separate PR in the series.
Depends-on: https://github.com/clockworklabs/SpacetimeDB/pull/3519
# Description of Changes
This patch tests calling, updating, and materialization of views through
the SQL api.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5
# Testing
Smoketests
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
So far, the `--clear-database` option to `publish` has simply dropped
and then re-created the database (if it did exist).
This will no longer work when databases can have "children": because
dropping and re-creating is not atomic, children would either become
orphans, or be dropped as well.
To solve this, `reset_database` is introduced as a separate action that:
- shuts down all replicas
- if a `program_bytes` is supplied, replaces the database's initial
program
- if a `host_type` is supplied, replaces the database's host type
- starts `num_replicas` or the previous number of replicas, which
initialize themselves as normal
As this could be its own CLI command, the action is provided as its own
API endpoint (undocumented).
However, since `publish` has no way of knowing whether the database it
operates on actually exists, the `publish_database` handler will just
invoke the `reset_database` handler if `clear=true` and the database
exists, and return its result. This is to avoid starting the transfer of
the program in the request body, only to receive a redirect.
Some refactoring was necessary to dissect and understand the flow.
# API and ABI breaking changes
Introduces a new, undocumented API endpoint.
We may want to nest it under `/unstable`.
# Expected complexity level and risk
2
# Testing
From the outside, the observed behavior should be as before,
so smoketests should cover it.
# Description of Changes
Disconnects clients when adding, removing, or updating RLS rules. Client
are not disconnected if an auto-migration does not touch any RLS rules.
> Why do we need to disconnect clients at all?
Subscriptions cache RLS rules. By disconnecting clients, we forcibly
evict stale invalid RLS rules from the cache.
# API and ABI breaking changes
Bug fix. Previously subscriptions would keep stale RLS rules in cache.
# Expected complexity level and risk
1
# Testing
- [x] Adding unit and smoke tests
- [x] Manual inspection with `spacetime subscribe`
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Zeke Foppa