# Description of Changes
Updates smoketests to check for following.
- Normal auto migration subscription should not disconnect subcriber.
- Add table columns migration should disconnect subscribers.
Solves #1957 .
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
PR contains:
* CLI changes for the `pre_publish` endpoint when publishing a module.
* The regular `--yes` flag will not bypasses the *break clients* warning
prompt — an extra confirmation is now required. For CI, a hidden flag
`--break-clients` is added.
* Added smoketest.
* Some trivial naming changes in `client-api-*` crates for consistency
reasons.
* `pre_publish` route to accept similar Body size limit as `publish`
route.
# API and ABI breaking changes
an additive API change, does not break anything.
# Expected complexity level and risk
2
# Testing
- Existing smoketests passing for backward compatibility.
- New smoketest for add columns
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
Closes: https://github.com/clockworklabs/SpacetimeDBPrivate/issues/2058
- Updated the generation code to setup basic initialization to
properties for Unreal Blueprints
- Added new Blueprint library to interact with FContextBase to allow
access to inherited properties from all contexts in Blueprint
# API and ABI breaking changes
No breaking changes
# Expected complexity level and risk
2 - Updates the generation code
# Testing
- [x] Ran Unreal tests to confirm no breaking changes
# Description of Changes
Add documentation for SpacetimeAuth
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
Checked if everything was working using a local instance of the website
# Description of Changes
As the title says
# Expected complexity level and risk
1
# Testing
- [x] Manually create the projects shown as examples and run them
# Description of Changes
The `AutoMigrateStep::DisconnectAllUsers` step is implemented as
follows:
1. The `spacetimedb::db::update::update_database` function returns a
response of type
`UpdateDatabaseResult::UpdatePerformedWithClientDisconnect`.
2. Upon receiving this response, the `host_controller::update_module`
proceeds to drop the `watch::Sender<ModuleHost>` field within the
`core::host_controller::Host` and disconnect clients.
# API and ABI breaking changes
NA
# Expected complexity level and risk
3.
Diff code is simple but It depends on the subcription logic to behave
correctly.
# Testing
Manually.
---------
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Report more metrics about snapshot compression, namely:
- time to compress a single snapshot (histogram)
- for each compression pass:
- number of snapshots found to be already compressed (gauge)
- number of snapshots compressed (gauge)
- cumulative number of objects compressed (gauge)
- cumulative number of objects hardlinked (gauge)
Those metrics are collected from the `spacetimedb-snapshot` crate
without imposing a prometheus dependency on it, i.e. they can be
observed by the caller as ordinary Rust types.
This is exploited to avoid scanning the entire snapshot repository on
each pass -- only the range `(last_compressed + 1)..newest_snapshot` is
visited (note that the `compress_snapshots` method now short-circuits on
errors).
Lastly, the snapshot worker can now be configured to disable
compression. This greatly simplifies implementation of alternative
post-processing strategies, e.g. involving archival, for which a more
coarse-grained compression strategy may be more appropriate.
Subcribers are notified of a new snapshot _after_ compression, such that
any filesystem locks should be released.
# Expected complexity level and risk
2
# Testing
May need some, I'm pondering.
The `DurabilityProvider` trait was introduced to enable the
`HostController` to procure an alternative `Durability` impl from an
external source.
It is also useful to be able to instantiate a `SnapshotWorker`
externally, in order to subscribe to snapshot creation events without
access to the `RelationalDB` instance it is operating on.
At a later stage, we may also use it to control the snapshot frequency
externally.
This patch thus reframes the trait as `PersistenceProvider`, whose job
is to provide persistence-related services.
Also separates snapshot creation and compression of older snapshots, and
adds instrumentation to gather timing information for both.
# Description of Changes
Re-submit of #3281 (reverted by #3293), with only the intended changes.
# Expected complexity level and risk
1.5
# Testing
No functional changes.
# Description of Changes
As the title says
# Expected complexity level and risk
1
# Testing
- [x] Manually check the error launching a local pg instance
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
I chose to regenerate all of the bindings in
https://github.com/clockworklabs/SpacetimeDB/pull/3310, but that caused
the snapshots to change. This PR follows up to update them.
I also updated the outdated script `tools~/gen-quickstart-chat.sh`
(which was generating from a different module than the CI was), and
updated the CI to use it instead of manually running a similar command.
# API and ABI breaking changes
Test-only changes.
# Expected complexity level and risk
1
# Testing
- [x] unity-testsuite passes on this PR (which it does not on `master`).
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We had places that weren't passing `-y`. I assume these were only
working because they were in environments where the things were already
installed, so there wasn't any confirmation dialog. For some reason, the
arm runner now newly needs to install packages, so CI invocations began
failing.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [ ] the "Test spacetimedb-update" flow passes on arm, which is not
currently true elsewhere.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Move the unity testsuite to `spacetimedb-runner` since it otherwise runs
out of disk space.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Unity CI passes now.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Move the global.json file to the repo root, which should make `dotnet`
resolve the correct SDK regardless of the directory it is run in.
Where a copy of the global.json file is needed, symlink it to the one at
the root.
Should fix "C#/Unity - Test Suite".
# Description of Changes
As the title says.
Add `--pg-port NUM` to the `start` command
# API and ABI breaking changes
Before this, it was set to `5432` unconditionally.
Docs updated at https://github.com/clockworklabs/SpacetimeDB/pull/3302.
# Expected complexity level and risk
1
# Testing
- [x] Run smoke tests
# Description of Changes
Each reducer gets its arguments through an `ArgSource`, a Unix-file-like
abstraction for streams of bytes. Prior to this commit, we had an ABI
designed as if it could support other args sources, but it actually
hardcoded the ID of the reducer args source, and errored elsewhere.
This commit extends the `BytesSource` infrastructure to support other
bytes sources. This will be useful for exposing JWT payloads and HTTP
responses. No other `BytesSource` uses are actually included in this
commit, only the infrastructure.
This commit also defines a new host call,
`bytes_source_remaining_length`. This is intended to allow callers to
pre-allocate a buffer correctly sized to read the entire `BytesSource`
all at once. The new host function is added to a new ABI minor version,
10.1, so that old SpacetimeDB hosts can detect and reject too-new
compiled modules. I have added uses of this new function to
`__call_reducer__` in both Rust and C#, even though it's not strictly
necessary,
and I haven't removed the loop which repeatedly calls
`bytes_source_read` and grows the buffer.
# API and ABI breaking changes
Adds a new ABI minor version, `spacetime_10.1`. This means that old
SpacetimeDB hosts will reject new compiled modules.
# Expected complexity level and risk
2-ish? WASM ABI code is always fiddly, but this is a pretty simple case.
# Testing
- [x] New behavior and new host function are both hit through existing
tests that instantiate modules and call reducers against them, so I
believe automated testing is sufficient.
---------
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
Implements `Serialize` and `Deserialize` for tuples.
Extracted from https://github.com/clockworklabs/SpacetimeDB/pull/3276.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
A test `roundtrip_tuples_in_different_data_formats` is added.
This reverts commit 2b61190d4d.
An accident happened, and the patch contains changes that were intended
for a separate PR.
Perhaps better to start over.
The `DurabilityProvider` trait was introduced to enable the
`HostController` to procure an alternative `Durability` impl from an
external source.
It is also useful to be able to instantiate a `SnapshotWorker`
externally, in order to subscribe to snapshot creation events without
access to the `RelationalDB` instance it is operating on.
At a later stage, we may also use it to control the snapshot frequency
externally.
This patch thus reframes the trait as `PersistenceProvider`, whose job
is to provide persistence-related services.
Also separates snapshot creation and compression of older snapshots, and
adds instrumentation to gather timing information for both.
# Expected complexity level and risk
1.5
# Testing
Not a functional change, existing tests should cover that.
# Description of Changes
Adds a way to configure the DbConnection to use confirmed reads.
# Expected complexity level and risk
1
# Testing
Still figuring out...
# Description of Changes
Inspired by a question @Shubham8287 asked me: how long does it take to
replay real production databases, e.g. a BitCraft region module? I did
not have a good answer, and didn't know how I could get an answer.
Add 5 new metrics:
- `replay_total_time_seconds(db: Identity)`, total time to read and
restore snapshot and replay commitlog.
- `replay_snapshot_read_time_seconds(db: Identity)`, time to read
snapshot from disk.
- `replay_snapshot_restore_time_seconds(db: Identity)`, time to restore
snapshot once it's already in memory.
- `replay_commitlog_time_seconds(db: Identity)`, time to replay
commitlog once snapshot is restored.
- `replay_commitlog_num_commits(db: Identity)`, number of commits
visited during commitlog replay, i.e. the length of the suffix after the
most recent usable snapshot.
All of these are `Gauge`s (or `IntGauge` for the `num_commits` metric),
as we expect observations to be very infrequent
(approx. once per label value per SpacetimeDB process).
# API and ABI breaking changes
N/a
# Expected complexity level and risk
1
# Testing
I remain unsure how to test metrics. This won't break anything, though,
so I'm not worried.
# Description of Changes
In environments without dotnet installed, these tests were the only
thing preventing `cargo test --all -- --skip csharp` from completing
successfully.
I also included `rust` in the name, though running tests in an
environment without cargo/rustc installed seems less likely to work.
Extracted from #3263 at request of @Centril .
# API and ABI breaking changes
N/a
# Expected complexity level and risk
1
# Testing
N/a
# Description of Changes
Finish the work started in #3227, which switched a hardcoded `127.0.0.1`
to a dynamic server address.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] Smoketests still pass
- [ ] `Internal Tests` pass under this PR (they don't on master)
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
- `add_columns_to_table` api to handle `AutoMigrateStep::AddColumns` --
Look at doc comment for more Info.
depends on: https://github.com/clockworklabs/SpacetimeDB/pull/3261
TODO: handle `AutoMigrateStep::DisconnectAllUsers`.
# API and ABI breaking changes
N/A.
# Expected complexity level and risk
2? Changes are not in hotpath but a bug in migration can corrupt tables.
# Testing
a test to exercise the API.
---------
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
Extends log records with a new field, `function`, which stores the name
of the reducer being executed when the log was produced. I have chosen
to name this field `function` rather than `reducer` because we will soon
be adding procedures, which are not reducers but will also be valid
values for this field.
While making this change, I noticed inconsistent values for injected
logs. Previously, we injected logs in three places, with different
values for the record fields:
1. `SystemLogger` (used when publishing and updating) set `filename:
Some("spacetimedb")` and `target: None`.
2. `log_reducer_error` (used for reducer error returns) set `filename:
None` and `target: Some(reducer)`, with `reducer` being the name of the
reducer.
3. `ModuleHost::inject_logs` (used for calls to nonexistent reducers and
calls with ill-typed arguments) set `filename: Some("external")` and
`target: None`.
With this change, I have decided that injected logs universally have
`filename: Some("__spacetimedb__")` and `target:
Some("__spacetimedb__")`. I have chosen the double-underscore convention
for reserved names to avoid confusion should a user name a source file
or reducer `spacetimedb`. I am not terribly attached to using
`spacetimedb` here, and would be happy to change to some other string,
like `internal` or `system`.
I have further decided that `log_reducer_error` and `inject_logs` both
have access to a sensible non-`None` value for `function` and so should
set it to the name of the reducer. The `target` field is not used for
this. Note that in cases where a client attempts to call a non-existent
reducer, the `function` field of the log record will be set to that
client-supplied non-existent reducer name.
# API and ABI breaking changes
Changes our log record format. This is at least backwards-compatible (as
in, newer consumers can read older logs) because the new field is
optional. I am unsure if it is forwards-compatible (as in, older
consumers reading newer logs) because I don't know if `serde_json` will
ignore unknown fields or will error.
# Expected complexity level and risk
1
# Testing
- [x] Manually ran `quickstart-chat` (modified to log messages and name
changes) and saw sensible output.
# Description of Changes
Adds support for truncation in the datastore.
The commitlog format already supported it.
A table is considered truncated in a transaction when there are deletes
and when the row count post-deletes is 0.
# API and ABI breaking changes
None
# Expected complexity level and risk
3?
# Testing
Some existing datastore tests have been amended to check whether
`TxData` contain truncation or not.
---------
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Shubham Mishra <shubham@clockworklabs.io>
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
Aternative to and closes
https://github.com/clockworklabs/SpacetimeDB/pull/3210.
This version relies on `pending_schema_changes`.
The first commit adds `clear_table` to the datastore that's efficient
and can be exposed to the module ABI in a follow up.
The second commit fixes `drop_table`.
# API and ABI breaking changes
None
# Expected complexity level and risk
3?
# Testing
`test_drop_table_is_transactional` is amended to check `TxData`.
---------
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Shubham Mishra <shubham@clockworklabs.io>
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
Bumping remaining files so that everything is at 1.4.0.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This PR removes the `@clockworklabs/typescript-sdk` from the repository
and retains only `spacetimedb` in the `crates/bindings-typescript`
directory. Some files are migrated to `spacetimedb`. I have also updated
the appropriate READMEs.
In addition I have symlinked the old `sdks/typescript` directory to
point to `crates/bindings-typescript`.
# API and ABI breaking changes
This is not technically a breaking change of any kind, although it does
orphan and deprecate the
[@clockworklabs/spacetimedb-sdk](https://www.npmjs.com/package/@clockworklabs/spacetimedb-sdk)
npm package. This package will no longer work with SpacetimeDB.
Users should now install and use the `spacetimedb` package.
# Expected complexity level and risk
2, it's a straightforward change but affects many files.
# Testing
- [ ] I ran `pnpm test` in the `spacetimedb` package
- [ ] I ran the quickstart app
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes: https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1925
This adds Unreal Blackholio to the demo/Blackholio folder matching the
Unity version using server-rust as the default. Includes:
- Leaderboard
- Circle Material - wavy border
- Split/Sucide/Input Lock
- Parallax Background
- Camera movement based on total mass
# API and ABI breaking changes
No changes
# Expected complexity level and risk
1 - Only adds the demo has no impact elsewise
# Testing
I've tested this backwards and forwards against the server-rust version
and played alongside the Unity client.
- [ ] Run it locally for review
---------
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This adds `react` as an optional peer dependency. If the TypeScript SDK
is imported into a library that uses React (of the appropriate version)
then they will have access to two new React hooks:
```ts
useSpacetimeDB<DbConnection>();
useTable<DbConnection, MyTable>('my_table');
```
This PR also updates the TypeScript `quickstart-chat` tutorial in the
docs to use the new React hooks. I will split that tutorial into
separate `React` and `TypeScript` tutorials in a future PR.
# API and ABI breaking changes
This is a purely additive change to the SDK
# Expected complexity level and risk
2 because it changes how the SDK is built a bit. Namely it makes the
`spacetimedb` library an external dependency.
# Testing
TODO: I am not through testing yet.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Julien Lavocat <JulienLavocat@users.noreply.github.com>
# Description of Changes
Closes#3219
This adds the Unreal SDK, the new Unreal test cases, updates the test
runner to handle Unreal, codegen updates for Unreal, and a QuickStart
Chat.
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
2 - This impacts the subcommand generate.rs to include unrealcpp and
crates/testing to expand for Unreal
# Testing
- [x] Run the new Unreal tests
- [x] Run any previous automation testing - with all the changes to
generate/testing I'm uncertain if there is an impact
- [x] Review the new CLI generate documentation changes
---------
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This PR moves most of the contents of `@clockworklabs/spacetimedb-sdk`
into the `spacetimedb` module. The `spacetimedb` module now exports
`sdk` and `server` as separate subpaths where `sdk` contains the code
which was previously in `@clockworklabs/spacetimedb-sdk`.
In particular it makes the following moves:
- `/sdks/typescript/packages/sdk` -> `/sdks/typescript`
- most of the contents of `/sdks/typescript/packages/sdk` ->
`crates/bindings-typescript`
- `/sdks/typescript/packages/test-app` ->
`crates/bindings-typescript/test-app`
The following packags was NOT moved:
`/sdks/typescript/examples/quickstart-chat`
## Motivation
In accordance with
https://github.com/clockworklabs/SpacetimeDB/issues/3250, we would like
to consolidate `@clockworklabs/spacetimedb-sdk` into a single
`spacetimedb` package so that users can import the different things they
need from a single package.
### Pros:
- allow users to install a single package with subpaths `spacetimedb`,
`spacetimedb/react`, `spacetimedb/sdk`, `spacetimedb/server`, etc.
- Is much simpler for bundling, etc.
- Is backwards compatible with `@clockworklabs/spacetimedb-sdk` which
now becomes a thin wrapper
- eventually allow us to break up the `spacetimedb` package into other
packages if we want to split them up (e.g. `@spacetimedb/lib`,
`@spacetimedb/sdk`, etc.) and we can solve the build complexity that
introduces when we get to it
- eventually allow us to move `bindings-csharp` out of the crates
directory where it probably doesn't belong anyway
- organizes all TypeScript packages into the packages directory where
you'd normally expect them, with the possible exception of
`/sdks/typescript` if we wanted to leave that separate
### Cons:
- The `sdk` directory is now a bit of a ruse as to where the code
actually lives since it's just a thin wrapper. If it eventually becomes
its own independent package, we'll also have to break up spacetimedb
into `@spacetimedb/lib` and `@spacetimedb/server` so that
`@clockworklabs/spacetimedb-sdk` can depend on `@spacetimedb/lib` while
being a dependency of `spacetimedb`.
Ideally this change would have been made later, however, it became
necessary for the following **heinously disastrous chain of forcing
moves**:
1. Adding `react` support necessitated shipping react as an optional
peer dependency under `@clockworklabs/spacetimedb-sdk/react`
2. This required adding a new build target/export/bundle
3. Previously `@clockworklabs/spacetimedb-sdk` was configured to have
`noExternal` for `spacetimedb` meaning it would collect the library into
the sdk bundle. I attempted to continue this for react support but...
4. Creating a new `react` bundle which also pulled in `spacetimedb`
caused their to be nominal type conflicts between classes in the
duplicate `spacetimedb` bundles.
5. Changing `spacetimedb` to be included as `external` caused compile
errors because `@clockworklabs/spacetimedb-sdk` is configured in
`tsconfig.json` to fail on unused variables and it was now including the
source of `spacetimedb` which is not configured to error on unused
variables and has "unused" private variables which are actually used by
us secretly, but not exposed to the clients.
> SIDE NOTE: The unused variables settings cannot be turned off on a
line by line basis, so it has to be turned off entirely, but in order to
maintain the linting checks we had I used `eslint` to enforce the rule
so that we could disable it line by line. (This caused me to discover
quite a lot of things that were broken that were caught by `eslint`
being applied to the entire project. `eslint` was previously only
applied to the `quickstart-chat` and the `crates/bindings-typescript`
library)
6. Changing the build to be external, now requires `spacetimedb` to also
be published to npm as its own module which
`@clockworklabs/spacetimedb-sdk` now imports, which requires that we add
`tsup` config to `spacetimedb` to publish a built version of the
library.
7. The only way to avoid that is to move the `sdk` and `react` code from
`@clockworklabs/spacetimedb-sdk` into the existing `spacetimedb` package
to avoid the duplicate import problem on step 4 and change
`@clockworklabs/spacetimedb-sdk` back to again use `noExternal` for its
`spacetimedb` dependency.
And here we are. I chose not to move `/crates/bindings-typescript` even
though that's probably not a great place long term. It would be better
to have it in `/packages/spacetimedb` or `/npm-packages/spacetimedb` or
`/ts-packages/spacetimedb` or something, and move all our TypeScript
packages in there. But that is a different matter.
The net result however is that we have a new `spacetimedb` package which
exports the different parts of the API under:
- `spacetimedb`
- `spacetimedb/server`
- `spacetimedb/sdk`
- `spacetimedb/react`
while still not breaking the existing deploy process, nor any
users/developers who are currently using
`@clockworklabs/spacetimedb-sdk`.
I think long term should we ever decide to split `spacetimedb` up into
multiple packages or if we have additional unrelated packages, we should
publish them to the `@spacetimedb` org which I reserved for us here:
https://www.npmjs.com/org/spacetimedb
> NOTE: `spacetimedb` is a package and `@spacetimedb` is an org.
`spacetimedb/sdk` is not a separate package, it's a subpath export of
the `spacetimedb` package, whereas `@spacetimedb/sdk` would be (and
would need to be) it's own separate package. You can certainly have both
`spacetimedb/sdk` and `@spacetimedb/sdk`. We could for example host the
code for the sdk at `@spacetimedb/sdk` and just reexport it from
`spacetimedb` under the `spacetimedb/sdk` subpath.
# API and ABI breaking changes
This should not change or modify the API or ABI in any way. If it does
so accidentally it is a bug, although I carefully went through the
exports.
# Expected complexity level and risk
3 because it changes how the SDK is built a bit and rearranges a lot of
paths.
# Testing
- [x] All of the CI passes
- [x] I also ran quickstart-chat to confirm that it is not broken
- [x] I also ran test-app
# Description of Changes
Fixes#3149
- Fix for MoveAllPlayers calculation to force the circles apart
- Fix for PlayerSplit to properly calculate Scheduled_At for the
recombine timer
# API and ABI breaking changes
None
# Expected complexity level and risk
1 - Trivial
# Testing
I've loaded this up against the Unreal Blackholio comparing the
server-rust vs server-csharp.
- [x] Confirm split + recombine work correctly in the Unity client
*Note*: The demo/Blackholio/client-unity project is currently generated
with the server-rust bindings
# Description of Changes
This adds a new system table to store the jwt payloads of connected
clients. I'm planning to use this system table to expose client claims
to modules in subsequent PRs.
The new table is called `st_connection_credentials`. It is a **private**
system table which stores a mapping from `connection_id` to
`jwt_payload`. Note that a jwt payload is a json representation of the
clients claims, not a fully signed token.
The times when we need to insert and delete these rows closely mirrors
that of the existing `st_client` table, with 1.5 exceptions:
1. We weren't previously inserting to `st_client` until after the
`OnConnect` reducer ran (even though it was in the same transaction). We
want `st_connection_credentials` to be populated before calling the
reducer, so that the reducer can use it get the credentials, so I made a
change to insert to `st_client` and `st_connection_credentials` before
calling the reducer.
2. This difference has not actualized, but when clients start sending
refresh tokens, we will probably need to update the credentials stored
in this table.
This also enforces uniqueness of connection ids. A duplicate connection
id will now make the on-connect reducer fail (since it will violate
uniqueness when trying to insert to `st_connection_credentials`).
# Expected complexity level and risk
2.5
Adding a system table is a bit risky. This is almost rollback safe, with
one annoying case that is worth calling out:
If a database is created with this system table, opening it with an
older version of spacetimedb will only work if there is a snapshot of
the database. If we try to load a table without a snapshot, replaying
will fail on the first row for that table. This is because we don't
write the table schema information to the commit log when creating a
database. In practice, this is unlikely to be an issue, because new
databases asynchronously trigger a snapshot immediately after creation.
Migrating existing databases will be fine. On startup this will detect
that there is a missing system table, and add it in a way that writes it
to the commit log. Since it is in the commit log, we can open the
database with an older version and still understand the data for that
table.
# Testing
There are unit tests that cover opening a database created with an older
version (which doesn't have this table).
I manually tested opening a migrated database with an older version of
spacetimedb.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
In order to facilitate commitlog and snapshot archival, this patch adds
channel params to `snapshot_watching_commitlog_compressor`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
The method is no longer used.
Note that this patch must merge **after** the private companion.
# API and ABI breaking changes
Internal API change.
# Expected complexity level and risk
1
# Testing
In private.
# Description of Changes
This just constrains the types of rows from `any` to `Record<string,
any>`.
# API and ABI breaking changes
Not breaking, purely internal.
# Expected complexity level and risk
1
# Testing
- [x] Just the automated tests
=