# Description of Changes
Closes: #3895
- Added third tier expanding folder
- Merged most into intro + core concepts
- Collapsed modules + database into one
- Moved several documents to how-to to streamline the core concepts
- Added warning to RLS to use views
- refactored all links
- Moved How To and References into Developer Resources
- Fixed TypeScript View code
Sample: (Getting Started will start expanded due to being the initial
page)
<img width="293" height="746" alt="image"
src="https://github.com/user-attachments/assets/6abc3a0d-1ae2-4886-bda7-11f5565afecf"
/>
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
- [X] Updated the quickstarts.py and generate-cli tooling
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
As title.
# API and ABI breaking changes
incremental binding.
# Expected complexity level and risk
# Testing
Unit and smoketest
# Description of Changes
fixes#3861.
While running prechecks for automigration, the `range` passed to
`iter_by_col_range_mut` was of type `AlgebraicValue::I128`, even though
it should have matched the column’s type.
Fix is straightforward by introducing `AlgebraicValue::from_i128`
helper.
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
# Testing
- Modified smoketest to repro reported bug.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
Fixes#3827
Adding lte and gte support for query_builder in rust.
# API and ABI breaking changes
Incremental change to bindings.
# Expected complexity level and risk
1
# Testing
unit test and smoketest
# Description of Changes
This makes it possible to insert new pages between existing pages
without renaming the existing pages.
I also added a section to docs/README.md with procedure for how to name
new pages, which I'll paste here:
### Adding new pages
All of our directory and file names are prefixed with a five-digit
number which determines how they're sorted.
We started with the hundreds place as the smallest significant digit, to
allow using the tens and ones places to add new pages between.
When adding a new page in between two existing pages, choose a number
which:
- Doesn't use any more significant figures than it needs to.
- Is approximately halfway between the previous and next page.
For example, if you want to add a new page between `00300-foo` and
`00400-bar`, name it `00350-baz`. To add a new page between `00350-baz`
and `00400-bar`, prefer `00370-quux` or `00380-quux`, rather than
`00375-quux`, to avoid populating the ones place.
To add a new page after all previous pages, use the smallest multiple of
100 larger than all other pages. For example, if the highest-numbered
existing page is `01350-abc`, create `01400-def`.
# API and ABI breaking changes
N/a
# Expected complexity level and risk
1 - @JulienLavocat said this wouldn't break anything.
# Testing
None.
---------
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Mainly a smoketest to exercise the intended behaviour. Also return an
error if we end up delegating to the reset database endpoint, which
itself doesn't accept a `parent` parameter.
# Description of Changes
Fixes the following issues:
1. When dropping a view, we deleted its row from `st_view`, but didn't
drop the backing table.
2. `delete_col_eq` returned a nonsensical error if the delete set was
empty.
# API and ABI breaking changes
None
# Expected complexity level and risk
0
# Testing
- [x] Auto-migrate smoketests
# Description of Changes
This reapplies the patch from #3704, and fixes the issues that were
causing it to deadlock.
The reason it was deadlocking was that it allowed for the following
sequence of events:
* `SchedulerActor::handle_queued()` begins mutable tx
* `ModuleHost::disconnect_client()` submits call to `call_reducer(tx:
None)`
* scheduler submits call to `call_reducer(tx: Some)`
* `WasmModuleInstance::disconnect_client` now has to try to take tx
lock, but the scheduler's call_reducer already holds it and is behind it
in the queue
So, I moved most of the logic from `handle_queued` back to being
executed in the module worker thread, but kept the code in
`scheduler.rs` so that it can all be reasoned about locally.
Fixes#3645. Should I uncomment the implementation of
`ExportFunctionForScheduledTable for F: Procedure` now?
# Expected complexity level and risk
2 - there's a chance that this patch hasn't fully fixed the deadlock
issue from #3704, but I'm quite confident.
# Testing
- [x] Manually verified that deadlock no longer occurs - previously,
`while true; do python -m smoketests schedule_reducer -k
test_scheduled_table_subscription; done` would freeze up in only 2 or 3
iterations, but now it can run for 10 minutes without issues.
# Description of Changes
Patch contain few different things, required to run query builder end to
end.
Git history seems messed up but it can be reviewed commitwise from -
69c1907b3e.
1. Decouples generic type `T` from table name, as type represents the
`struct` and not table, and there couple be multiple tables implemented
from same struct.
2. modify table macro implementaiton to generate code supporting
`traits` and types required for query builder,
3. Disable aliasing in sql queries for now, as it was causing
semantically wrong query when we were doing joins on `FromWhere` (as
original `Expr` were referencing to original table names).
4. smoketests.
# API and ABI breaking changes
Add `Query<T>` type as Views return type.
# Expected complexity level and risk
2, changes are simple but not splitted up well.
# Testing
1. Added smoketest.
TODO:
ui test.
---------
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Small cleanup of duplicate logic on smoke tests
# Expected complexity level and risk
1
# Testing
- [x] Run smoke tests
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
This PR is the first in a series of PRs which are an effort the
reorganize the docs and make it easier for people to get started using
SpacetimeDB.
This PR:
1. Renames all folders to be `kebab-case`. This is because `docIds` are
set based on folder and file names, and there is no way to override that
within a file (all you can change with `id: my-id` is the last component
of the id. Because we want to be able to access docs in react components
we need stable predictable ids.
- This also necessitates setting up a few `_category_.json` files so
that the categories do not show up as kebab case in the sidebar.
2. Adds a few React components with images so it's clearer on how to get
started
3. Reorganizes the quickstarts to be at the top of the side bar and
simplifies the server and client into a single document named for the
language
4. Sets up redirections so the old links to the old documents now point
to the new documents
5. Copies the entirety of `typography.css` from the website into this
repo for the purpose of keeping them identical
6. Makes minor fixes and improvements to the contents of the docs
In a future PR, I intend to:
1. Add a tutorial section to the docs which will include two tutorials:
Blackholio and Chat
2. Turn the chat quickstart documents into the Chat tutorial and
consolidate it into a single file with different language switchers (if
this is feasible)
3. Create a new quickstart section, which has a quickstart guide for
each platform that we support so far, namely:
- Vanilla TypeScript
- Vanilla JavaScript
- Vanilla Rust
- Vanilla C#
- React
- Unity
- Unreal
4. Update the quickstart guides to do the following (in the future I
will add more platforms with this scheme):
- Use `spacetime dev --template <framework-template>` to create a new
project for the given framework
- Insert data into the database
- Add a new `reducer` to the template
- Add a new `table` to the template
- Add a new `view` to the template
- Call the reducer from the client
- Subscribe to the `view` from the template
5. Break up the current per language module reference files into files
about SpacetimeDB concepts with a `TabGroup` for each language, namely:
- Modules
- Tables
- Reducers
- Procedures
- Code generation
- DbConnection (and the SDK API)
- Auth
- etc.
The idea being that we should explain our platform by concept, not by
language. This will also allow us to reuse a lot of prose that is
repeated in the different language documents.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Tested the documents locally to ensure that they render properly
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
View tables have private metadata columns that need to be dropped before
sending results to clients. Before this patch we dropped these columns
for sql queries and initial subscriptions, but we didn't drop them after
incremental update which is what this patch does.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Smoketest
# Description of Changes
Precise index readsets
fixes #https://github.com/clockworklabs/SpacetimeDBPrivate/issues/2118
# API and ABI breaking changes
NA
# Expected complexity level and risk
2.5
Potential to regress performance.
# Testing
Updated smoketests.
# Description of Changes
We had a few issues with `smoketests --list`. I wanted to improve this
behavior in preparation for using it (or similar logic) to parallelize
the unit tests.
First of all, the `--list` logic ran _after_ our calls to `cargo build`
and `spacetime login`, so it could take a while to run tests _and_ you
would need a server running just to list tests.
Also, if any tests failed to import, it would give a cryptic error and
exit in the middle of the list:
```
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/__main__.py", line 169, in <module>
main()
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/__main__.py", line 151, in main
for test in itertools.chain(*itertools.chain(*tests)):
TypeError: '_FailedTest' object is not iterable
```
Now, it completes the entire list regardless, and prints a much better
clearer message:
```
Failed to construct unittest.loader._FailedTest.pg_wire:
ImportError: Failed to import test module: pg_wire
Traceback (most recent call last):
File "/usr/lib/python3.12/unittest/loader.py", line 137, in loadTestsFromName
module = __import__(module_name)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lead/work/clockwork-localhd/SpacetimeDBPrivate/public/smoketests/tests/pg_wire.py", line 5, in <module>
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
```
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Existing CI still passes
- [x] Running `python -m smoketests --list` no longer tries to build
SpacetimeDB or log in to a server
- [x] Running `python -m smoketests --list` has a (much) more
descriptive error if there are broken imports
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We have added one too many python dependencies. I'm putting them in a
shared file, which will also ease installation for devs running them
locally.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
Existing CI still passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Changes with cache hit:
CI / Smoketests (spacetimedb-runner): 17m -> 10m
CI / Smoketests (windows-latest): 39m -> 31m
CI / Lints: 3m30s -> 2m
unity-testsuite: 23m -> 19m
This included creating a stripped-down `Dockerfile` that only runs
SpacetimeDB (as opposed to e.g. building tracy, flamegraphs, etc.). That
shaves off about 4-5m on the linux smoketests job.
# API and ABI breaking changes
No. CI only.
# Expected complexity level and risk
2
# Testing
Ran existing CI and noted the change in times in the description above
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This PR modifies the `--delete-data` flag on `spacetime publish` and
adds the `--delete-data` flag on `spacetime dev`.
In particular instead of `--delete-data` being a boolean, it is now a an
enum:
- `always` -> corresponds to the old value of `true`
- `never` -> corresponds to the old value of `false`
- `on-conflict` -> clears the database, but only if publishing would
have required a manual migration
This flag does NOT change any behavior about prompting users to confirm
if they want to delete the data. Users will still be prompted to confirm
UNLESS they pass the separate `--yes` flag.
`spacetime dev` gets the same `--delete-data` flag. The default value of
`never` is equivalent to the existing behavior. `spacetime dev`
continues to publish with `--yes` just as before. This behavior is
unchanged.
# API and ABI breaking changes
Adds the flags specified above. This is NOT a breaking change to the
CLI. Passing `--delete-data` is the equivalent of
`--delete-data=always`.
This IS technically a breaking change to the `pre_publish` route. As far
as I'm aware this is *only* used by our CLI however.
> IMPORTANT SIDE NOTE: I would argue that `--break-clients` should
really be renamed to `--yes-break-clients` because it actually behaves
like the `--yes` force flag, but only for a subset of the user prompts.
I have not made this change because it would be a breaking change, but
if the reviewers agree, I will make this change.
# Expected complexity level and risk
2, Very small change, but if we get it wrong users could accidentally
lose data. I would ask reviewers to think about ways that users might
accidentally pass `--delete-data --yes`.
# Testing
- [ ] I have not yet tested manually.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Fixes a subscription plan caching bug related to client-specific views.
Before this fix, you could define a client-specific view:
```rust
fn my_view(ctx: &ViewContext) -> Option<Player> {
ctx.db.player().identity().find(ctx.sender)
}
```
And subscribe to it as follows:
```sql
SELECT * FROM my_view
```
Note this view is implicitly parameterized by `:sender`, however when
generating a query hash for this subscription, this fact would not be
taken into account which would result in this query being cached and
reused for all callers.
After this fix, a query hash is generated for this subscription as
though it were given as:
```sql
SELECT * FROM my_view WHERE identity = :sender
```
# API and ABI breaking changes
None
**Note for CLI code owners:**
I had to touch the `subscribe` cli command file. No updates to the api.
It just needed to be updated to look for views in the module def.
# Expected complexity level and risk
1
# Testing
- [x] Added a regression smoketest
# Description of Changes
Improves auto-migration support for views by minimizing the cases where
we must disconnect clients.
Before this patch, any schema compatible view update and even no view
updates at all would require us to disconnect clients, because we had to
assume that the view was modified thereby rendering its currently
materialized result set stale and out of date.
Patch adds a code to re-evaluate all views whose entry is in
`st_view_sub` and make `update_database` to start using
`commit_and_broadcast_event` so that now we only have to disconnect
clients for incompatible view updates or dropped views.
# API and ABI breaking changes
NA
# Expected complexity level and risk
2
# Testing
Added smoketests.
# Description of Changes
Fixes the issue described in
https://github.com/clockworklabs/SpacetimeDB/issues/3558.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Smoketests are now successful after running `cargo bump-versions
1.8.0 --typescript`.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Permissions for evaluating SQL/DML are not generally "actions", but more
a set of permissions that are checked during evaluation.
To make this work with the teams feature, this patch extends `AuthCtx`
to allow checking a set of permissions as mandated by the spec. This set
is a bit more fine-grained than "is owner", so as to avoid baking in the
concept of teams/collaborators, or assumptions about what a role might
entail. Both are likely to evolve in the future, so evaluation of
permissions / capabilities should be confined to the impl of the
`Authorization` trait.
Unlike "actions", the `AuthCtx` must be able to evaluate permission
checks quickly and without side-effects, nor can it enter an `async`
context. In that sense, it is precomputed (if you will), and stored as a
closure in the `AuthCtx` for external authorization.
A challenge posed is how to thread through the constructed `AuthCtx` for
subscriptions.
A tempting approach would have been to equip the `HostController` with
the ability to summon an `AuthCtx`. That, however, would have created a
gnarly circular dependency, because the `HostController` also controls
the controldb, which itself demands an `AuthCtx`.
Instead, the `AuthCtx` is obtained in the endpoint handler and passed to
each method call that requires one. That's less pretty, but more
effective.
---------
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
This adds authorization to the relevant API endpoints, updates the CLI
commands and adds smoketests for the teams feature.
**Note**: Authorizing SQL (incl. subscriptions) is a bit more involved,
and submitted as a separate PR in the series.
Depends-on: https://github.com/clockworklabs/SpacetimeDB/pull/3519
# Description of Changes
This patch tests calling, updating, and materialization of views through
the SQL api.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5
# Testing
Smoketests
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Disconnects clients when adding, removing, or updating RLS rules. Client
are not disconnected if an auto-migration does not touch any RLS rules.
> Why do we need to disconnect clients at all?
Subscriptions cache RLS rules. By disconnecting clients, we forcibly
evict stale invalid RLS rules from the cache.
# API and ABI breaking changes
Bug fix. Previously subscriptions would keep stale RLS rules in cache.
# Expected complexity level and risk
1
# Testing
- [x] Adding unit and smoke tests
- [x] Manual inspection with `spacetime subscribe`
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Disables parameterized views in rust until we can pass arguments from
sql. Note parameters are already disabled in C# and typescript.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Needed to update some tests. Will revert these changes once we re-enable
parameters.
# Description of Changes
Now running `python -m smoketests` no longer updates
`smoketests/config.toml` with a new identity, and also `replication`
doesn't fail when docker isn't available.
# Expected complexity level and risk
1
# Testing
- [x] those things no longer happen
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Based on #3461, which fixes a bug encountered in the quickstart.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
n/a
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Yes
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
This is a draft of the new functionality for `spacetime init`. In order
to run it with built-in templates you have to set the path to the config
file:
```
export SPACETIMEDB_CLI_TEMPLATES_FILE=crates/cli/.init-templates.json
```
In the future it will fetch the list from GH.
A few notes:
* the previous functionality of `spacetime init` does not work at the
moment
* the code needs a bit more cleanup and tests before merging
* there is a bit of a mix in how we generate empty server and client
projects. For Rust we use the existing way of generating. For TypeScript
we clone an empty project from the repo. I wanted to play with both ways
of doing things, and I'm still not sure which is better. Generation in
Rust means that the generated code will match the CLI version and not
necessarily whatever is in Git. On the other hand, for the builtin
templates we will be fetching the newest version from GH, which I guess
might also not what we want, ie. we probably want only stable templates.
More discussion is needed here
* we use `spacetimedb` directory for the server files
* I don't particularly like the inability to disable interactive mode
easily. We discussed disabling it by default if all of the required
arguments are passed, but I don't think it's feature proof. For example,
if someone relies on a non-interactive mode, and we add a new required
argument, instead of printing a message `missing --foo`, we will
automatically launch interactive mode, which is harder to debug. That's
why I think I'd prefer to implement `--non-interactive` argument
* it's kind of hard to keep the legacy behaviour. If you don't pass any
arguments, we go into interactive mode. In the legacy version, we would
print required arguments. If someone passes `--lang` or `--project-path`
explicitly, I guess we could run the legacy workflow, but not sure if
it's worth it, as the command was marked as unstable anyway
* the project path defaults to the project name, but I think we should
probably replace change whitespaces to dashes, or at least ask for the
project path with the project name being the default (or both)
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Makes `name` a required parameter of the `#[view]` macro. This is to be
consistent with the `#[table]` macro since views should expose the same
interface as tables.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Added new compile tests
- [x] Updated existing compile tests
## Description of Changes
- Migrate the documentation site to **Docusaurus**.
- Add a **GitHub Action workflow** that runs `pnpm build` on pull
requests targeting `master`.
This command checks for broken links and images and ensures the site
builds correctly.
- Add a **GitHub Action workflow** to publish the docs automatically
when a merge occurs on the `docs/release` branch.
**Important**: The workflows haven't been tested yet due to the fact
that they need to be merged first before being able to run. It's a
fairly basic workflow I don't expect this to block anyone but anyways I
will make another PR with the complete changes when I can actually run
it.
---
## API and ABI Breaking Changes
No breaking changes for **SpacetimeDB** users.
Documentation contributors should experience minimal disruption —
content is still written in Markdown.
---
## Expected Complexity Level and Risk
**Complexity:** 3/5
**Reasons:**
1. Several adjustments were made to fit Docusaurus conventions:
- Replaced our custom `:::server-rust` blocks with [Docusaurus `<Tabs/>`
components](https://docusaurus.io/docs/markdown-features/tabs).
- Converted “Note” callouts using
[admonitions](https://docusaurus.io/docs/markdown-features/admonitions)
instead of Markdown blockquotes.
- Updated all images (including Unity tutorial ones, which were
previously commented out).
- Moved images from the DO bucket into the repository at
`docs/static/images`.
2. Customized Docusaurus CSS and one component to apply our theme and
color palette.
---
## Testing
Tested locally.
---
## Review Notes
This PR involves many file changes — a full code review is likely not
productive.
Instead, please focus on verifying that the **documentation content**
was migrated correctly:
- Review sections you’re familiar with to confirm accuracy.
- Ensure formatting, links, and images render as expected.
You can follow the updated **README** for instructions on running
Docusaurus locally.
---------
Signed-off-by: Julien Lavocat <JulienLavocat@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
# Description of Changes
The quickstart smoketest was not correctly fudging our NuGet
dependencies to use the local versions of the packages, so it was
pulling them from NuGet. This ended up causing issues when we tried to
modify local packages and then use them in a way that affected the
quickstart (e.g.
https://github.com/clockworklabs/SpacetimeDB/pull/3386).
We had a few issues: we weren't ensuring that the local packages were
built, we weren't using the right directory for those packages, and we
weren't adding a "package source mapping" that forced those packages to
be used from the local directory.
# API and ABI breaking changes
None. CI-only.
# Expected complexity level and risk
2
# Testing
- [x] CI passes when
https://github.com/clockworklabs/SpacetimeDB/pull/3386 is merged
together with this PR (it wasn't before).
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
`auto_migrate_database` should not take table schema as input because
each step is ordered and may depend on changes made by a prior step,
this approach can lead to incorrect assumptions (e.g., adding a column
changes its table id).
Migration steps now retrieve the current table schema when they run.
This adds a small performance cost, but it simplifies the logic and code
is not under hotpath.
fixes#3441
# API and ABI breaking changes
NA
# Expected complexity level and risk
1
# Testing
Update smoketest to add failed case.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Adds the `#[view]` procedural macro and module describers for views.
```rust
#[view(public)]
fn player(ctx: &ViewContext) -> Vec<Player> {
ctx.db.player().identity().find(ctx.sender).into_iter().collect()
}
#[view(public)]
fn player(ctx: &AnonymousViewContext, level: u32) -> Vec<Player> {
ctx.db.player().level().filter(level).collect()
}
```
Note, this deviates from the proposal in that views may only return
`Vec<T>` or `Option<T>`. They can't return an arbitrary `SpacetimeType`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
This technically isn't a breaking change, but it's worth mentioning that
this patch refactors `ReducerInfo` so that we can use it for views as
well.
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Negative compile tests
- [x] Negative publish (module validation) tests
- [x] Test system tables are updated accurately
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
The `#[table]` macro now generates read-only table and index handles.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] positive and negative test cases using
`ReducerContext::as_read_only()`
# Description of Changes
Updates smoketests to check for following.
- Normal auto migration subscription should not disconnect subcriber.
- Add table columns migration should disconnect subscribers.
Solves #1957 .
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
PR contains:
* CLI changes for the `pre_publish` endpoint when publishing a module.
* The regular `--yes` flag will not bypasses the *break clients* warning
prompt — an extra confirmation is now required. For CI, a hidden flag
`--break-clients` is added.
* Added smoketest.
* Some trivial naming changes in `client-api-*` crates for consistency
reasons.
* `pre_publish` route to accept similar Body size limit as `publish`
route.
# API and ABI breaking changes
an additive API change, does not break anything.
# Expected complexity level and risk
2
# Testing
- Existing smoketests passing for backward compatibility.
- New smoketest for add columns
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Finish the work started in #3227, which switched a hardcoded `127.0.0.1`
to a dynamic server address.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] Smoketests still pass
- [ ] `Internal Tests` pass under this PR (they don't on master)
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes
[#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686).
Add support for listening using the [PG wire
protocol](https://www.postgresql.org/docs/current/protocol.html) so `pg`
clients could be used against the database.
# API and ABI breaking changes
The output of `duration` is changed to `rfc3339`, instead of the way is
made with `sats` because is what is done in `pg`, see note below.
# Expected complexity level and risk
2
~~There is open questions that are in the [ticket
#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686). Also
the crate used here require `RustTls`, so it could be good idea to
decide if~~:
* ~~Rewrite a big chunk of code to use `OpenSSL`~~
* ~~Move to `RustTls`
https://github.com/clockworklabs/SpacetimeDB/pull/1700~~
* ~~Pay for the extra compilation cost~~.
I open another port(`5433`) to listen for `pg` connections using `ssl`.
Need to be decided if this is the way or instead try to multi-plex the
current port for both protocols.
# Testing
Only manual testing so far. Solving the above questions allow me to
implement some unit tests. Also, not yet integrated into cloud for the
same reasons.
- [x] Adding some test for the binary encoding of special and primitive
types
- [x] Smoke test using `psql` that connect to the db instance and run
some queries
- [x] Manually inspect using a UI database explorer how infer the types,
some of this tools generate special widgets when displaying `json,
duration, etc`
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Implements [subscribing to durable
commits](https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1594).
The setting works on a per-connection level, and essentially just delays
sending transaction updates until the transaction is reported as durable
by the database.
For connectionless SQL operations, the setting works per-request. No SQL
syntax is provided by this patch to toggle the configuration.
After some deliberation, I opted to obtain the offset when a transaction
commits (as opposed to when it starts). This creates some mild
inconvenience, because we prevent the transaction from committing until
the corresponding subscription updates are enqueued.
The strategy is, however, more correct should we ever support weaker
isolation levels, and it is easier to document.
Follow-ups include:
- Provide SQL syntax (`SET synchronous_commit = ON` or something)
- C# and TypeScript SDKs
- Reference docs?
# API and ABI breaking changes
Not breaking, but adds a parameter to the subscribe and sql endpoints.
# Expected complexity level and risk
4
To the author's understanding, ordering of outbound messages is not
changed by this patch, even if there are messages that don't have a
transaction offset (such as error messages). I.e. while waiting for the
transaction offset of a message to become durable, no message enqueued
after that message will be delivered. This may not be desirable in some
cases.
The patch may contain concurrency bugs, e.g. awaiting futures that may
never resolve.
# Testing
- [x] Implemented a new test in the `module_subscription_actor` module
- [x] Added unit tests for the core logic in `ClientConnectionReceiver`
It would be desirable to also have integration-level tests, but I'm
currently unsure how to write those without being able to control if and
when the database reports an offset as durable.
---------
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
Closes
https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1998#event-19488677475
Now the smoke test parse the `config.toml` for the correct address for
connecting.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
- [x] Run the smoke both on public / private
# Description of Changes
Closes#2100
Replay the `quickstart-chat` guide steps for the `Rust/C#` client &
server, and confirm the connection and `set_name`, `send_message`
commands.
# Expected complexity level and risk
1
# Testing
- [x] Added a smoke test
# Description of Changes
First commit reverts disabling adding enum variants
(https://github.com/clockworklabs/SpacetimeDB/pull/3178).
Second commit makes replay recognize inserts / deletes to `st_column`
and triggers a refresh of the in-memory table that was referenced in the
`st_column` change.
# API and ABI breaking changes
None
# Expected complexity level and risk
3, small code bug "deep".
# Testing
I've confirmed manually that a module that couldn't be restarted before
this PR can now be restarted.
We should probably follow up this PR with a smoketest.
Mario Alejandro Montoya Cortes