# Description of Changes
Makes the keynote benchmark job reusable so that it can be invoked and
run in other CI environments.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2
# Testing
Refactor. Relies on existing coverage.
# Description of Changes
Uses `jsonwebtoken v10.4.0` instead. Important changes include:
**1. Token serialization**
Old tokens with `"exp": null` are still accepted, but new no-expiry
tokens now omit `exp` instead of serializing it as `"exp": null`.
**2. OIDC/JWKS validation**
Issuer extraction now uses `jsonwebtoken::dangerous::insecure_decode`
for key discovery only, not validation. And the old `spacetimedb-jwks`
crate required every JWK to have a `kid`, but this patch does not
preserve that limitation.
# API and ABI breaking changes
I don't believe this is considered breaking, but it bears repeating that
new no-expiry tokens now serialize without `exp` instead of `"exp":
null`.
# Expected complexity level and risk
2
# Testing
- [x] Verify a legacy no-expiry token serialized as `"exp": null` still
validates.
- [x] Verify a token with an expired `exp` is still rejected.
- [x] Verify OIDC/JWKS validation works when the JWKS keys omit the
optional `kid` field.
# Description of Changes
Adds a merge-queue fast path for CI when the synthetic merge-group
commit has the same tree as the queued PR head.
The new `merge_queue_noop` job parses the PR number from the merge-group
ref, resolves the PR head SHA, and compares that tree to `GITHUB_SHA`.
If there is no diff, the expensive CI jobs are skipped as duplicate
work. Matrix jobs with required per-matrix check names get lightweight
no-op counterparts so branch protection still sees the expected
successful check names.
# API and ABI breaking changes
None.
# Expected complexity level and risk
2. This is limited to GitHub Actions wiring, but it interacts with merge
queue semantics and required check names. The implementation
intentionally falls back to normal CI if it cannot parse the PR number
or resolve the PR head.
# Testing
- [x] Parsed `.github/workflows/ci.yml` with Ruby YAML.
- [x] Ran `git diff --check`.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This removes the `spacetimedb-update` check specifically on arm. This
test doesn't have a whole lot of value because we're already covering
Linux + Windows on x86 and then macOS on aarch64. Removing this will
allow us to decom the arm runner.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None - just a CI change
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- I have not tested this but me and Zeke sync'd on this and we think it
makes sense.
# Description of Changes
Moves `RelationalDB` and related database code into a new
`spacetimedb-engine` crate.
The main motivation is to tighten dependency control around the engine
layer and isolate `RelationalDB`
behind a crate boundary.
- Majority of this PR is code-motion.
- Removes direct production dependence on `tokio` from
`spacetimedb-engine`.
- Keeps `tokio` only as a dev-dependency for test-only code in
`spacetimedb-engine`.
- This is intended to be a structural refactor only and should not
result in any functional change in
production.
- Adds a CI check to ensure `spacetimedb-engine` continues to compile in
simulation mode
# API and ABI breaking changes
NA
# Expected complexity level and risk
1.5.
# Testing
Existing tests should be enough.
---------
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Moves `test_index_scans` to its own job that uses the same runner as the
keynote benchmark.
This test had several issues:
1. It was a performance regression test that didn't run in an isolated
environment because it was just a test.
2. It measured timings by search the module log for `ns`(nanosecond) and
`us`(microsecond) suffixes
As a result it would occasionally flake.
Now it runs on dedicated hardware in an isolated environment, so we
shouldn't see anymore flakes.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1
# Testing
N/A
# Description of Changes
(Moving this to a tools repo)
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Set `RUST_BACKTRACE=full` for the CI jobs.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
- Completes the Blackholio demo with username selection, leader board,
split mechanic, improved visuals...
- Add Godot tests
# API and ABI breaking changes
- Nothing
# Expected complexity level and risk
1. It just updates the Blackholio demo project
# Testing
- [X] Play the updated demo
- [X] Run the tests locally
# Description of Changes
Moving `Internal Tests` to its own workflow so it can be canceled and
re-run independently from the rest of CI.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
- [x] `Internal Tests` succeed on this PR and appear to have run
properly
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
## Summary
- Add a `dorny/paths-filter` step to the `internal-tests` job that
checks for any changed file outside `docs/`.
- Gate the private-repo dispatch and the wait-for-completion steps on
that filter, so a docs-only PR no longer fires off the private CI
workflow.
- The job itself still runs and completes successfully, so any branch
protection requiring the `Internal Tests` check continues to be
satisfied.
## Test plan
- [ ] Open a docs-only follow-up PR and confirm `Internal Tests` reports
success without dispatching a private run.
- [ ] Open a PR touching `crates/` (or anything non-docs) and confirm
the private dispatch still fires as before.
# Description of Changes
Adds a new required ci check for keynote-2 benchmark regressions. The
test runs for 60s and fails if throughput < 300K TPS.
Note, this check will be flaky as long as it's running concurrently with
other CI jobs. It may need a dedicated runner/host machine. Although it
may be sufficient to only schedule one runner/VM to a single host
machine at a time. I'll need to sync with @jdetter to determine the best
way forward here.
UPDATE: We're using a dedicated runner. See the **Testing** section.
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2
Mainly copy-paste from the other CI workflows.
# Testing
This job now uses `spacetimedb-benchmark-runner` which is entirely
dedicated to this one CI job. I've tested this at different times of the
day when the CI runners are under load and not. The performance is
consistent and the test isn't flaky. It has passed every time.
# Description of Changes
Very small additions to the C# SDK specific for Godot.
I wrote the Godot Blackholio tutorial and updated the Unity one.
I added the image assets necessary for the tutorial.
I added the files for the Godot demo.
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
1. There's really no risk for current systems or projects, it's all new
additions to support nicely(-ish) Godot, new Blackholio demo for Godot
and updates to the tutorials.
# Testing
- [X] Follow the tutorial and verify everything works.
- [X] Build for Windows and verify it works.
- [x] Build for Linux and verify it works.
- [x] Build SDK
- [x] Publish to Nuget
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Due to the relatively frequent supply chain attacks on especially npm
packages, we're instituting a minimum package age in the whole repo.
- Globally set a minimum npm package age in CI
- Best-effort set npm package age using `.npmrc` beside any
`package.json`
- Add CI checks that pnpm version and minimum package age values are the
same everywhere
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI passes
- [x] if I remove a `.npmrc` then `cargo ci lint` fails
- [x] if I change a value in `.npmrc` then `cargo ci lint` fails
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
LLM benchmark infrastructure improvements and new benchmark tasks.
**Runner & scoring:**
- Add retry logic with backoff for LLM API calls (rate limits,
502/503/504, timeouts)
- Fix `generation_duration_ms` to only time the successful attempt, not
retries+sleep delays
- Add `--dry-run` flag to run benchmarks without saving results
- Add OpenRouter client as unified fallback when direct vendor keys
aren't set
- Add web search mode via OpenRouter `:online` suffix
- Extract shared OpenAI-compatible response types into `oa_compat.rs`
- Add `ReducerCallBothScorer` for calling reducers on both golden and
LLM databases
- Set `max_tokens` on OpenRouter and Meta clients to prevent silent
truncation
**Model routing:**
- Add `ModelRoute` with display name, vendor, API model, and OpenRouter
model ID
- Support ad-hoc model IDs via `--models vendor:model` without static
registration
- Add model name normalization (OpenRouter IDs, case variants →
canonical display names)
**Context modes:**
- Add `guidelines`, `cursor_rules`, `search`, `no_context` modes with
`is_empty_context_mode()` helper
- Add mode-specific prompt preambles
- Consolidate mode alias normalization (`none`/`no_guidelines` →
`no_context`)
**CI workflows:**
- Add `llm-benchmark-periodic.yml` for scheduled nightly runs with
per-language failure tracking
- **Note**: The periodic workflow requires `OPENROUTER_API_KEY`,
`LLM_BENCHMARK_UPLOAD_URL`, and `LLM_BENCHMARK_API_KEY` as GitHub
secrets.
- Add `llm-benchmark-validate-goldens.yml` for validating golden answers
still compile
**Results & summary:**
- Add `cmd_status` to show incomplete benchmark combinations with rerun
commands
- Add `cmd_analyze` for LLM-powered failure analysis
- Split `normalize_details_file` from `write_summary_from_details_file`
- Derive task categories from filesystem for summary generation
- Add timestamp tracking (`started_at`/`finished_at`) and token usage
**New benchmark tasks:**
- 30 new tasks across auth, data_modeling, queries, basics, and schema
categories
- Updated/fixed existing task prompts and golden answers
# API and ABI breaking changes
None. Internal tooling only.
# Expected complexity level and risk
2 — Changes are scoped to the LLM benchmark CLI tool
(`xtask-llm-benchmark`) and CI workflows. No impact on SpacetimeDB core.
# Testing
- [x] `cargo check -p xtask-llm-benchmark` — zero errors, zero warnings
- [x] Dry run: `llm_benchmark run --lang typescript --modes no_context
--tasks t_001 --models openai:gpt-5-mini --dry-run` — ran end-to-end,
confirmed no results saved to disk
- [ ] Verify periodic workflow runs successfully on next scheduled
trigger
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
CI was running `gen-quickstart.sh` and then checking for a diff.. but it
was checking in the wrong directory.
I have also regenerated the files because the fixed check was failing.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [x] updated CI failed without the changes to the other files
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Merged the `upgrade-version-check.yml` into `ci.yml`, and moved the
business logic under `cargo ci`.
I would also be very open to just removing this test until we choose to
define a better suite of tests for `cargo bump-version`.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Ran it locally. It made a diff
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
https://github.com/clockworklabs/SpacetimeDB/pull/4231 changed our CI to
always pass a parameter corresponding to the PR number, which.. broke on
`master` commits since they don't have a PR number.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1.
# Testing
I think I don't know how to test this. But it's basically the old
behavior on `master` commits, so it should work fine? One hopes?
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Remove the Python smoketests and the CI check that tests for edits.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [ ] All CI passes
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Flipping on some new inputs to `Internal Tests` to get new
functionality.
I'll follow-up with a more detailed description in discord.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
2
# Testing
- [x] public PR without private PR just uses master
- [x] public PR with private PR uses that one
- [x] fails if private PR is not approved
- [x] fails if private PR does not pass its CI
- [x] passes if private PR is ready to go
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Revert the following PRs that have caused some breakage:
```
a32cffa76 Finish refactoring out replay (#4850)
d639be0af Replay: some code motion & reuse `ReplayCommittedState` (#4849)
78d6b6f7d Update NativeAOT-LLVM infrastructure to current ABI (#4515)
d5c1738c1 Better module backtraces for panics and whatnot (#577)
6f23b19f3 Wait for database update to become durable (#4846)
81c9eab86 Add `spacetime lock/unlock` to prevent accidental database deletion (#4502)
809aebd7c Move field `replay_table_updated` to `ReplayCommittedState` (#4807)
21b58ef99 Update axum (#2713)
b5cadff7a Extract replay stuff out of `CommittedState`, part 1 (#4804)
```
I also updated the Python smoketests for breakage introduced in
https://github.com/clockworklabs/SpacetimeDB/pull/4502. Reverting that
PR caused conflicts, so this fix is more straightforward.
# API and ABI breaking changes
Maybe kind of, but we haven't released any of these.
# Expected complexity level and risk
1
# Testing
Ask @bfops about testing
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Migrate these checks into `cargo ci`:
- Check that packages are publishable
- Docs test
- TypeScript - Tests
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
2
# Testing
- [ ] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Removed some "if we're on windows" checks in the CI so that we're always
running through `cargo ci update-flow`.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1.
# Testing
- [x] Upgrade flow tests pass
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Merged `typescript-test.yml` and `docs-test.yml` into `ci.yml`.
Note: The required checks will need to be updated when this PR is ready
to merge.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Merge the `TypeScript - Lint` CI job into `cargo ci lint`.
Note that this removes the custom caching for the pnpm store, but we're
planning to overhaul our CI cache approach anyway.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Lint step passes on this PR
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
## Summary
- Update the experimental NativeAOT-LLVM build path
(`EXPERIMENTAL_WASM_AOT=1`) to include all host function imports from
ABI versions 10.0 through 10.4
- Fix the compiler package reference to work on both Windows x64 and
Linux x64 (was hardcoded to Windows only)
- Add a CI smoketest to verify AOT builds work on Linux x64
## Context
See #4514 for the full writeup on the C# AOT situation. The
`wasi-experimental` workload that all C# module builds depend on is
deprecated and removed from .NET 9+. NativeAOT-LLVM is the recommended
path forward for ahead-of-time compilation of C# to WebAssembly.
The existing NativeAOT-LLVM support (added by RReverser in #713) was
stale: missing imports added since then and a Windows-only package
reference.
## Changes
**`SpacetimeDB.Runtime.targets`:**
- Add 10 missing `WasmImport` declarations across spacetime_10.0 through
10.4
- Replace `runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM` with
`runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM`
so it resolves correctly on Linux x64 as well
- Use explicit version strings instead of the `$(SpacetimeNamespace)`
variable
**`ci.yml`:**
- Add AOT build smoketest step in the `csharp-testsuite` job
## Test plan
- [x] CI smoketest passes: `EXPERIMENTAL_WASM_AOT=1 dotnet publish -c
Release` builds successfully on Linux x64
- [ ] Existing C# tests continue to pass (no changes to the default
interpreter path)
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
To make `cargo ci test` more properly include the full test logic.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
honestly none
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
To make `cargo ci` more properly the full test logic.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
honestly none
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This enables the use of the self hosted windows runners.
|CI Job|Before|After|Change|
|---|---|---|---|
|CI/Smoketests (Windows)|79m|31m|154% faster|
This is *before* we do any work with caching.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None, this is just a CI change.
# Expected complexity level and risk
1 - this is just a CI change.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Ran the windows smoketest several times. Each time it is passing,
although several runs I was making performance changes so some runs are
faster than others.
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
This PR adapts the Rust SDK test suite to work with the wasm version
added in https://github.com/clockworklabs/SpacetimeDB/pull/4089 (which
I've closed in favor of this PR).
Most of the changes revolve around wasm's different async semantics -
everything runs in one thread, so things that relied on background
threads didn't work directly. Several tests would lock up because
something in them blocked synchronously, which blocked any background
work from progressing.
We moved the test-clients contents into a `test_handlers.rs` so that it
could be called from both `main` (for native tests) and `lib` (for wasm
tests). To show what actually changed, use:
```bash
git diff --no-index -- <(git show origin/master:sdks/rust/tests/procedure-client/src/main.rs) sdks/rust/tests/procedure-client/src/test_handlers.rs
```
(or similar for other test-clients)
# API and ABI breaking changes
None, I think/hope.
# Expected complexity level and risk
2
# Testing
- [x] I've augmented the CI to also run the test suite with the `web`
feature
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Thales R <thlsrmsdev@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
These have been running alongside the Rust smoketests for a while, and
we feel fairly confident that the Rust ones are doing a good job.
(However, I leave them in the repo because we still use them elsewhere).
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
None
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes
Smoketests were running twice because `tests/foo.rs` automatically gets
turned into a test binary, but we also had a `mod.rs` that listed each
file, so they were getting tested via that as well.
This was first broken in
https://github.com/clockworklabs/SpacetimeDB/pull/4184.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [x] If I run `cargo ci smoketests` locally, I still see the CLI tests
running
- [x] If I run `cargo ci smoketests` locally, I do not see a particular
CLI test name appearing more than once (e.g.
`cli_cannot_publish_breaking_change_without_flag`)
- [x] `cargo ci smoketests check-mod-list` passes
- [x] `cargo ci smoketests check-mod-list` fails if I `touch
crates/smoketests/tests/smoketests/foo.rs`
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
# Description of Changes
We had an issue where the install failed but did not fail the step:
```
Run choco install psql -y --no-progress
Chocolatey v2.6.0
Installing the following packages:
psql
By installing, you accept licenses for the packages.
Failed to fetch results from V2 feed at 'https://community.chocolatey.org/api/v2/Packages(Id='psql',Version='16.2.0')' with following message : Response status code does not indicate success: 504 (Gateway Time-out).
Need to add specific handling for exception type NuGetResolverInputException
Unable to find package 'psql'. Existing packages must be restored before performing an install or update.
```
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None - this is just CI
# Expected complexity level and risk
None - this is just CI
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Windows smoketests are able to use `psql` properly
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
`Cargo.lock` was getting stale in our smoketests modules, because we
don't check it anywhere. I added a check.
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
- [x] new CI step fails without the `Cargo.lock` changes
(<https://github.com/clockworklabs/SpacetimeDB/actions/runs/21960035403/job/63435284426?pr=4282>)
- [x] new CI step passes with the `Cargo.lock` changes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This adds the v2 websocket protocol and adds support on the server side.
For context on many of the changes/decisions, you can look at the
discussion on https://github.com/clockworklabs/SpacetimeDB/pull/4023.
To restate some of the key changes:
- The reducer event information is no longer sent with transaction
updates (because we don't want to broadcast reducer call information
anymore).
- If a client calls a reducer, they are sent a `ReducerResult` which
includes the outcome of the reducer call and and related row updates for
queries that the client is subscribed to.
- We no longer dedupe queries that appear in multiple query sets for the
same client. This is because we are moving toward per-query storage.
- Related to that, Unsubscribe requests have an option to send the
related rows. We need this for now, since clients don't have per-query
storage implemented yet.
- We don't have the json format in v2.
Notes for reviewers:
- This moves around the messages in
`crates/client-api-messages/src/websocket` (into `common`, `v1`, and
`v2`), and this renaming of existing messages adds a lot of noise to the
PR.
- In many places, I chose to duplicate a lot of code to have a v1
version and a v2 version. I went with this to make it easier to remove
the v1 version in the future (hopefully we can just fully delete most of
the v1 functions).
- `module_subscription_manager.rs` has probably has the biggest changes,
since we now track queries by query_set_id, and we get to remove some
complexity of v1's FormatSwitch.
<!-- Please describe your change, mention any related tickets, and so on
here. -->
# API and ABI breaking changes
The v1 protocol still works, though we won't send the reducer event info
for v10 modules.
# Expected complexity level and risk
4. This touches a lot of places.
# Testing
Unit testing is pretty minimal for the new code paths. I've done some
manual e2e testing with the typescript quickstart, and this has been
tested with a different branch implementing the v2 rust client.
---------
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Version upgrade to 2.0.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - this is just a version bump
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] License file has been updated
- [x] CI passing
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Apparently we ran too many unity testsuite jobs today and we got this:
```
Unable to find image 'unityci/editor:ubuntu-2022.3.32f1-linux-il2cpp-3' locally
ubuntu-2022.3.32f1-linux-il2cpp-3: Pulling from unityci/editor
docker: error from registry: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
```
So now we login before pulling to increase our limit.
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1 - this just logs us into docker before running the test
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI still passes
# Description of Changes
- Updated the chat tutorial to include blocks for C++ server code
- Updated the Python smoketests to allow C++ with the quickstart check
# API and ABI breaking changes
N/A
# Expected complexity level and risk
1 - Small changes to get the C++ modules to be tested through quickstart
# Testing
- [x] Ran the tests locally
# Description of Changes
Blocks procedures from requesting private ip ranges after dns
resolution.
Adds a new cargo feature to `spacetimedb-standalone` permitting loopback
http requests in test environments only.
# API and ABI breaking changes
None
# Expected complexity level and risk
2. I may have missed a range.
# Testing
- [x] Unit tests for IP address matching
- [x] Smoketests for blocking a private IP address
# Description of Changes
CI is currently failing in a bunch of places since we tweaked the Rust
versions available in our CI runner, and it revealed that many of our
workflows don't set the default rust toolchain.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI passes in this PR
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This adds C++ server bindings (/crate/bindings-cpp) to allow writing C++
20 modules.
- Emscripten WASM build system integration with CMake
- Macro-based code generation (SPACETIMEDB_TABLE, SPACETIMEDB_REDUCER,
etc)
- All SpacetimeDB types supported (primitives, Timestamp, Identity,
Uuid, etc)
- Product types via SPACETIMEDB_STRUCT
- Sum types via SPACETIMEDB_ENUM
- Constraints marked with FIELD* macros
# API and ABI breaking changes
None
# Expected complexity level and risk
2 - Doesn't heavily impact any other areas but is complex macro C++
structure to support a similar developer experience, did have a small
impact on init command
# Testing
- [x] modules/module-test-cpp - heavily tested every reducer
- [x] modules/benchmarks-cpp - tested through the standalone (~6x faster
than C#, ~6x slower than Rust)
- [x] modules/sdk-test-cpp
- [x] modules/sdk-test-procedure-cpp
- [x] modules/sdk-test-view-cpp
- [x] Wrote several test modules myself
- [x] Quickstart smoketest [Currently in progress]
- [ ] Write Blackholio C++ server module
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
https://github.com/clockworklabs/SpacetimeDB/pull/4209 changed our v8
patch to fix release builds.. but it turns out that the csharp-testsuite
uses both debug _and_ release builds.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
I dunno mannnn
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Required in order to bump rolldown to its latest version, which I'm
planning to do afterwards. I also figure we may as well do this before
releasing a major version.
# Expected complexity level and risk
2
# Testing
n/a
# Description of Changes
We were using `cargo install` but doing our hacky v8 dance for debug
build outputs. `cargo install` runs in release mode though.
This came up in https://github.com/clockworklabs/SpacetimeDB/pull/4205.
# API and ABI breaking changes
None. CI only
# Expected complexity level and risk
1
# Testing
🤷
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This PR translates all of our Python smoketests into Rust tests which
can be run from `cargo run`
## Motivation
The purpose of this fivefold:
1. All developers on the team are familiar with Rust
2. It simplifies our devops because we can drop Python as a dependency
to run the tests
3. You can now run all tests in the repo through the single `cargo test`
interface
4. Because we use the `SpacetimeDbGuard` and `cargo test`/`cargo
nextest` we can easily parallelize the smoke tests
5. The smoketests can now use machinery imported from SpacetimeDB crates
(e.g. `bsatn` etc.)
IMPORTANT NOTE!
There are several ways to implement the smoke tests in Rust (none are
great):
1. A separate xtask specifically for the smoke tests
- This doesn't solve the problem of the CLI tests which also use the
`guard` crate
- Idiosyncratic way to run the smoke tests as opposed to cargo test
- Does NOT resolve the cargo within cargo problem because we still have
to build the test modules with cargo
2. A `build.rs` script in `guard` which first builds the executables as
a compile step for compiling guard
- Deadlocks on a cargo lock file conflict (Outer cargo compiles guard →
runs build.rs, inner cargo tries to acquire the build directory lock,
outer cargo holds the directory lock, deadlock)
- If you fix the deadlock by using different target dirs, it still looks
stuck on building guard because it's actually compiling all of
spacetimedb-standalone and spacetimedb-cli.
- Still technically runs cargo inside of cargo.
3. Add `spacetimedb-cli` and `spacetimedb-standalone` as an artifact
dependency of the guard crate
- Has good and clear output but requires +nightly when running the
smoketests and CLI tests, otherwise won't do the right thing. See
https://github.com/rust-lang/cargo/issues/9096
4. Compile the executables at runtime during the tests themselves where
the first test takes a lock while the executables are building using
cargo within cargo
- Makes the tests look like they're taking a long time when they're just
waiting for the build to complete
- Requires relatively complex locking machinery across
binaries/tests/processes
5. A two step solution where the developer has to build the binaries
before calling the smoke tests
- Very error prone
None of these are good. `xtask` is not bad, but doesn't enable us to run
other integration tests in other crates (e.g. the CLI)
(3) is the correct solution and has the best user experience, but it
requires nightly and I don't want to introduce that for all of our
tests.
I have chosen to do a combination of (1) and (4). You will now run the
smoketests with `cargo smoketest`. If you run `cargo test --all` (or use
`guard`) without doing `cargo smoketest` it will fall back to (4) which
compiles the executables at runtime. Running `cargo build` is the **only
way** to ensure that the executables are not stale because of the
internal fingerprint checking. Everything else is fragile not robust.
NOTE! There is no way to avoid cargo within cargo and have the smoke
tests be run as cargo tests because the modules under test must be
compiled with cargo.
# API and ABI breaking changes
Note that this is a BREAKING CHANGE to `cargo test --all`. The
smoketests are now part of `cargo test --all` unless you specifically
exclude them.
# Expected complexity level and risk
3, this is partially AI translated. We need to carefully review to
ensure the semantics have not regressed.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
This moves a bunch of stuff from `lib` back into `server` and `sdk`, and
removes all but one global variable from the server sdk in preparation
for export-based reducer definition.
# Expected complexity level and risk
2 - a pretty big refactor, but it's mostly just code movement.
# Testing
- [x] Refactor, so automated tests are sufficient.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This workflow change migrates us from docker container runners to the
new system which uses virtual machines via qemu. This solves a lot of
complexities around docker-in-docker and also allows us to easily run
windows VMs in the future.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
0 - this only updates our workflow and has no impact on the code.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI is passing
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This has been popping up in more places, so I've added this in more
places.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
🤷
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
joshua-spacetime