# Description of Changes
This reverts the version bump, since it seems to be causing test
flakiness somehow.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
See tin.
This helps out with throughput for the V8 reply rx/tx channel.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
Host side changes for Wasm to work with `RawModuleDefV10` along with
prior versions.
Based on #https://github.com/clockworklabs/SpacetimeDB/pull/4098
# API and ABI breaking changes
NA
# Expected complexity level and risk
1.5
# Testing
Existing test will cover for any regression.
Not yet functional for `RawModuleDefV10`, so can not be tested now.
# Description of Changes
Patch:
1. `crates/lib/src/db/raw_def/v10.rs` - a definition as per
https://github.com/clockworklabs/SpacetimeDBPrivate/issues/2412.
2. Refactors `RawModuleDefV9` validation code to dedup some of the core
validation logic, No functionality should change there.
3. Validation logic for new `RawModuleDefV10`.
4. Modify `crates::schema::def::ModuleDef` :
- To include `raw_module_def_version`
- `visibility` field to existing `RedcuerDef` and `ProcedureDef`
5. It deprecates RLS in favour of Views, If anyoine feel otherwise, let
me know.
Lot of code is duplicated from `RawModuleDefV9` includng tests.
# API and ABI breaking changes
NA, `RawModuleDefV10` is not yet exported by modules.
# Expected complexity level and risk
3? close analyses of structure is important to ensure future
extensibility.
# Testing
For code motion in `RawModuleDefV9` validation: Existing unit tests
seems to cover any functionality change.
For `RawModuleDefV10`: V9 test has been repeated with more `assert`s to
check on `visibility`
# Description of Changes
This PR implements the C# client-side typed query builder, as assigned
in https://github.com/clockworklabs/SpacetimeDB/issues/3759.
Key pieces:
* Added a small C# runtime query-builder surface in the client SDK
(`sdks/csharp/src/QueryBuilder.cs`):
* `Query` (wraps the generated SQL string)
* `Table<TRow, TCols, TIxCols>` (entry point for `All()` / `Where(...)`)
* `Col<TRow, TValue>` and `IxCol<TRow, TValue>` (typed column
references)
* `BoolExpr` (typed boolean expression composition)
* SQL identifier quoting + literal formatting helpers (`SqlFormat`)
* `Join(...)` with `WhereLeft(...)` / `WhereRight(...)`
* `LeftSemijoin(...)` / `RightSemijoin(...)` with `Where(...)` chaining
* Extended C# client bindings codegen (`crates/codegen/src/csharp.rs`)
to generate:
* Per-table/view `*Cols` and `*IxCols` helper classes used by the typed
query builder.
* A generated per-module `QueryBuilder` with a `From` accessor for each
table/view, producing `Table<...>` values.
* A generated `TypedSubscriptionBuilder` which collects
`Query<TRow>.Sql` values and calls the existing subscription API.
* An `AddQuery(Func<QueryBuilder, Query> build)` entry point off
`SubscriptionBuilder`, mirroring the proposal’s Rust API.
* Fixed a codegen naming collision found during regression testing:
* `*Cols`/`*IxCols` helpers are now named after the table/view accessor
name (PascalCase) instead of the row type, since multiple tables/views
can share the same row type (e.g. alias tables / views returning an
existing product type).
* Kept `Cols`/`IxCols` off the public surface:
* `Table.Cols` and `Table.IxCols` are internal, so consumers only access
columns via the `Where(...)`/join predicate lambdas.
C# usage examples (mirroring the proposal’s Rust examples)
1) Typed subscription flow (no raw SQL)
```csharp
void Subscribe(SpacetimeDB.Types.DbConnection conn)
{
conn.SubscriptionBuilder()
.OnApplied(ctx => { /* ... */ })
.OnError((ctx, err) => { /* ... */ })
.AddQuery(qb => qb.From.Users().Build())
.AddQuery(qb => qb.From.Players().Build())
.Subscribe();
}
```
2) Typed `WHERE` filters and boolean composition
```csharp
conn.SubscriptionBuilder()
.OnApplied(ctx => { /* ... */ })
.OnError((ctx, err) => { /* ... */ })
.AddQuery(qb => qb.From.Players().Where(p => p.Name.Eq("alice").And(p.IsOnline.Eq(true))).Build())
.Subscribe();
```
3) “Admin can see all, otherwise only self” (proposal’s “player” view
logic, but client-side)
```csharp
Identity self = /* ... */;
conn.SubscriptionBuilder()
.AddQuery(qb =>
qb.From.Players().Where(p =>
p.Identity.Eq(self)
)
)
.Subscribe();
```
4) Index-column access for query construction (IxCols)
```csharp
conn.SubscriptionBuilder()
.AddQuery(qb =>
qb.From.Players().Where(
qb.From.Players().IxCols.Identity.Eq(self)
)
)
.Subscribe();
```
# API and ABI breaking changes
None.
* Additive client SDK runtime types.
* Additive client bindings codegen output.
* No wire-format changes.
# Expected complexity level and risk
2 - Low to moderate
* Mostly additive code + codegen.
* The main risk is correctness/compat of generated SQL strings and
name/casing conventions across languages; this is mitigated by targeted
unit tests + full C# regression test runs.
# Testing
- [X] Ran run-regression-tests.sh successfully after regenerating C#
bindings.
- [X] Ran C# unit tests using `dotnet test
sdks/csharp/tests~/tests.csproj -c Release`
- [X] Added a new unit test suite
(`sdks/csharp/tests~/QueryBuilderTests.cs`) validating:
* Identifier quoting / escaping
* Literal formatting (including `Identity`/`ConnectionId`/`Uuid` hex
literals; `U128` integer literal)
* null + enum unsupported behavior throws
* Boolean expression parenthesization (`And`/`Or`/`Not`)
* `Where(...)` overloads including `IxCols`-based predicates
* left/right semijoin SQL formatting and predicate chaining
# Description of Changes
The first commit defines a type `TableName` that is used in e.g.,
`TxData` and where determined profitable and necessary to do this
change.
`TableName` is backed by
[`ecow::EcoString`](https://docs.rs/ecow/0.2.6/ecow/string/struct.EcoString.html)
which affords O(1) clones and 15 bytes of inline storage and
`mem::size_of::<EcoString>() == 16`.
The second commit does the same for `ReducerName`. This is also used in
reducer execution.
Together, these commits increase TPS by around 5-7k TPS.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
Two documentation improvements:
1. **Reducers documentation**: Clarified that using global/static
variables in reducers is **undefined behavior**, not just "values won't
persist". Added six specific reasons why this is undefined:
- Fresh execution environments
- Module updates
- Concurrent execution
- Crash recovery
- Non-transactional updates
- Replay safety
2. **Access permissions documentation**: Replaced the "Combining Both
Techniques" example that used indexes on Option fields (which
SpacetimeDB doesn't support) with a working example that filters by a
required `department` field instead.
# API and ABI breaking changes
None. Documentation only.
# Expected complexity level and risk
1 - Documentation changes only.
# Testing
- [ ] Verify the reducers warning is clear and accurate
- [ ] Verify the access permissions example compiles and makes sense
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Rework `TxData` to:
- store all information for a table together in a single type
`TxDataTableEntry` rather than having several different maps.
- be constructed as fast as possible.
- fit every entry within a single cache line, i.e., `(TableId,
TxDataTableEntry)` takes up 64 bytes.
- fit a single table entry inline to optimize for small transactions.
- expose methods `{inserts, deletes}_for_table` to make `DeltaTx`
faster.
Rework `DatabaseUpdate` to:
- store a single `DatabaseTableUpdate` inline
- make `from_writes` profit from the changes to `TxData`, avoiding the
temporary hash map and allocating the necessary capacity from the start.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? Fairly simple changes, but in important places.
# Testing
Existing tests are changed to match the changes to `TxData`.
# Description of Changes
Uses the xoroshiro128+ implementation from the `pure-rand` package.
# Expected complexity level and risk
1 - this API is "userspace" only.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Verified that the pcg32 implementation is correct and matches the
rust version it's based off of
- [ ] Add a test using `ctx.random()`
- Add call_anon method for anonymous reducer calls
- Add describe and describe_anon methods
- Add test_call: verify anyone can call standard reducers
- Add test_describe: verify anyone can describe any database
- Add test_logs: verify non-owners cannot view logs
- Add test_publish: verify cannot publish to database you do not own
- Add test_replace_names: verify cannot replace names of database you do not own
- Add api_call_json method for JSON API calls with Content-Type header
- Add Authorization header with Bearer token to all API calls
- Add test_replace_names test from domains.py (tests PUT /v1/database/{name}/names)
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Version upgrade to `v1.12.0`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this is just a version upgrade
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
The testsuite failures are fixed by
https://github.com/clockworklabs/SpacetimeDB/pull/4120
- [x] License has been properly updated including version number and
date
- [x] CI passes
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
- Convert basic-rs template from crates.io deps to workspace deps
(spacetime init will convert them back for users)
- Add basic-rs template to workspace members in root Cargo.toml
- Rename basic-rs package to avoid collision with sdk-test-connect-disconnect
- Simplify default_module_clippy test to run both templates in place
- Add missing st_client table verification to client_disconnected test
Use sql_confirmed() instead of sql() for the SQL INSERT in the
confirmed subscription test. The confirmed subscription only sends
updates after the transaction is durable, so using sql_confirmed()
ensures the INSERT is durable before we start collecting updates.
Replace individual autoinc modules (u32, u64, i32, i64) with two
consolidated modules that test all 10 integer types (u8, u16, u32,
u64, u128, i8, i16, i32, i64, i128), matching the Python tests.
- Create autoinc-basic module with macro for all types
- Create autoinc-unique module with macro for all types
- Update test to iterate over all types in a single test
- Remove individual type-specific modules
Update module discovery to respect CARGO_TARGET_DIR env var when
looking for precompiled WASM modules. CI sets this for the main
workspace, so the modules get built there instead of in the modules
workspace's own target directory.
# Description of Changes
Defines `SmallHashMap` in `spacetimedb_data_structures`.
The data structure is a hybrid map that is backed by a vector and linear
scan for the first `M` elements.
Then it falls back into a normal hash map.
For the first `N` elements, where `N < M`, the vector is stored inline.
That is, the vector is a `SmallVec<[(K, V); N>`.
The structure is optimized for access patterns where the keys and values
are accessed together,
therefore, keys and values are not stored separately.
# API and ABI breaking changes
None, just additive.
# Expected complexity level and risk
1
# Testing
A comprehensive proptest suite is added for the datastructure.
Smoketests require pre-built binaries and have their own dedicated
command (cargo ci smoketests) that builds binaries first. Exclude
them from cargo test --all to prevent failures in CI.
# Description of Changes
Skip more work in `MutTxId::view_for_update` when there are no views.
This is a gain of a few kTPS and results in `view_for_update`
disappearing from flamegraphs.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing tests.
# Description of Changes
Comment out tracing for `eval_updates_sequential`.
This alone results in a net gain of a 9k TPS.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
No semantic changes.
# Description of Changes
Exploit `(ty: RowTypeLayout).layout.fixed` to avoid some work involving
var-len stuff:
- skip `required_var_len_granules_for_row` for only-fixed-len layouts.
- skip `write_large_blobs` for only-fixed-len layouts.
- add a fast path in `has_space_for_row` for only-fixed-len layouts.
This resulted in these methods disappearing in flamegraphs.
# API and ABI breaking changes
None
# Expected complexity level and risk
2, core datastore, but fairly trivial.
# Testing
Covered by existing tests.
# Description of Changes
(jdetter): Github marked Ryan's PR as merged because I rebased it
without updating the base branch first - my bad.
Original PR: https://github.com/clockworklabs/SpacetimeDB/pull/4120
This PR fixes C#-related CI failures when testing against `v1.12.0`
before the corresponding NuGet packages are available on nuget.org.
* CI: Ensure local C# NuGet packages are used consistently
* Pack `crates/bindings-csharp/{BSATN.Runtime,Runtime}` with `-c
Release` so the configured local package sources (`bin/Release`)
actually contain the `.nupkg`s.
* Run `./sdks/csharp/tools~/write-nuget-config.sh` in CI to generate a
`NuGet.Config` that maps `SpacetimeDB.BSATN.Runtime` /
`SpacetimeDB.Runtime` to those local sources (with `nuget.org` as
fallback for non-SpacetimeDB dependencies).
* Add an explicit `dotnet restore --configfile NuGet.Config` step before
tests, and `run dotnet test --no-restore`, so restore always uses the
intended config and does not “accidentally” restore from `nuget.org`.
* `write-nuget-config.sh`: Make NuGet config discovery + mapping
deterministic
* Write `NuGet.Config` (capitalized) and include `<clear />` + an
explicit `nuget.org` source to avoid inherited sources and to keep
`PackageSourceMapping` functional.
* Also write a repo-root `NuGet.Config` so `dotnet publish/pack` invoked
from templates (outside `sdks/csharp/`) still discovers the same
override behavior.
* Smoketests quickstart: avoid `PackageSourceMapping` restore failures
* Ensure smoketest helper packing uses `Release`, and when repo-root
`NuGet.Config` exists, perform `dotnet restore --configfile ...` +
`dotnet pack --no-restore` so pack/restore are evaluated with the same
sources/mapping.
* When editing configs, prefer `NuGet.Config` casing for Linux discovery
and ensure `nuget.org` exists as a source for the fallback mapping.
# API and ABI breaking changes
No changes
# Expected complexity level and risk
1
# Testing
- [X] Locally tested concept after simulating local repro.
- [x] Confirmed CI passes in this branch
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Enable running Rust smoketests against a remote server instead of
spawning local servers, similar to Python smoketests --remote-server.
- Add SPACETIME_REMOTE_SERVER env var support to skip local server spawn
- Add --server CLI option to cargo smoketest
- Add skip_if_remote!() macro for tests requiring local server control
- Mark restart tests with skip_if_remote!() since they need local server
Scans the target directory for smoketest_module_*.wasm files and
derives module names from filenames (underscores become hyphens).
This eliminates the need to manually maintain the registry.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This fixes 2 issues with the upgrade version tool:
1. The typescript bindings need to be updated otherwise the typescript
test in CI will fail
2. The snapshots need to be updated
When the version upgrade tool check in CI runs, snapshot changes are
accepted automatically via the `--accept-snapshots` cli argument. When
you are running this tool locally without `--accept-snapshots` you will
be asked to manually review the snapshot changes before doing a final
test to make sure the snapshots are correct.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1 - this just updates the version upgrade tool
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Version bump 1.12.0 worked:
https://github.com/clockworklabs/SpacetimeDB/pull/4084
- [x] CI passes
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Zeke Foppa