# Description of Changes
Introduce a new **LLM benchmarking app** and supporting code.
* **CLI:** `llm` with subcommands `run`, `routes list`, `diff`,
`ci-check`.
* **Runner:** executes globally numbered tasks; filters by `--lang`,
`--categories`, `--tasks`, `--providers`, `--models`.
* **Providers/clients:** route layer (`provider:model`) with HTTP LLM
Vendor clients; env-driven keys/base URLs.
* **Evaluation:** deterministic scorers (hash/equality, JSON
shape/count, light schema/reducer parity) with clear failure messages.
* **Results:** stable JSON schema; single-file HTML viewer to
inspect/filter/export CSV.
* **Build & guards:** build script for compile-time setup;
* **Docs:** `DEVELOP.md` includes `cargo llm …` usage.
This PR is the initial addition of the app and its modules (runner,
config, routes, prompt/segmentation, scorers, schema/types,
defaults/constants/paths/hashing/combine, publishers, spacetime guard,
HTML stats viewer).
### How it works
1. **Pick what to run**
* Choose tasks (`--tasks 0,7,12`), or a language (`--lang rust|csharp`),
or categories (`--categories basics,schema`).
* Optionally limit vendors/models (`--providers …`, `--models …`).
2. **Resolve routes**
* Read env (API keys + base URLs) and build the active set (e.g.,
`openai:gpt-5`).
3. **Build context**
* Start Spacetime
* Publish golden answer modules
* Prepare prompts and send to LLM model
* Attempt to publish LLM module
4. **Execute calls**
* Run the selected tasks within each test against selected models and
languages.
5. **Score outputs**
* Apply deterministic scorers (hash/equality, JSON shape/count, simple
schema/reducer checks).
* Record the score and any short failure reason.
6. **Update results file**
* Write/update the single results JSON with task/route outcomes,
timings, and summaries.
# API and ABI breaking changes
None. New application and modules; no existing public APIs/ABIs altered.
# Expected complexity level and risk
**4/5.** New CLI, routing, evaluation, and artifact format.
* External model APIs may rate-limit/timeout; concurrency tunable via
`LLM_BENCH_CONCURRENCY` / `LLM_BENCH_ROUTE_CONCURRENCY`.
# Testing
I ran the full test matrix and generated results for every task against
every vendor, model, and language (rust + C#). I also tested the CI
check locally using [act](https://github.com/nektos/act).
**Please verify**
* [ ] `llm run --tasks 0,1,2` (explicit `run`)
* [ ] `llm run --lang rust --categories basics` (filters)
* [ ] `llm run --categories basics,schema` (multiple categories)
* [ ] `llm run --lang csharp` (language switch)
* [ ] `llm run --providers openai,anthropic --models "openai:gpt-5
anthropic:claude-sonnet-4-5"` (provider/model limits)
* [ ] `llm run --hash-only` (dry integrity)
* [ ] `llm run --goldens-only` (test goldens only)
* [ ] `llm run --force` (skip hash check)
* [ ] `llm ci-check`
* [ ] Stats viewer loads the JSON; filtering and CSV export work
* [ ] CI works as intended
---------
Signed-off-by: bradleyshep <148254416+bradleyshep@users.noreply.github.com>
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: spacetimedb-bot <spacetimedb-bot@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Introduce a hacky workaround to our `csharp-testsuite` to address
missing `librusty_v8.a`: manually check for that file and manually build
the package if it's missing.
# API and ABI breaking changes
CI-only change
# Expected complexity level and risk
1
# Testing
- [x] Locally tested removing the `librusty_v8.a` and then running
`cargo clean -p v8 && cargo build -p v8`, and this does seem to repair
it.
- [x] The CI has run with a cache that is "broken", but successfully
passes `csharp-testsuite`
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Disable `cache-on-failure` as we think that it's contributing to
mysterious `rusty_v8` linker issues.
I bumped the prefix key so that PRs with this change don't share caches
with PRs missing this change.
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
None. We'll have to see if we stop having issues once this is merged.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Make `cargo ci` work properly on Windows, in preparation for
https://github.com/clockworklabs/SpacetimeDB/pull/3702.
# API and ABI breaking changes
No. CI-only.
# Expected complexity level and risk
2. Not trivial, but not complicated.
# Testing
- [x] CI output seems to be genuinely running the tests, and it's
passing on Windows
- [x] Make a change to `crates/bindings-csharp` and see that `cargo ci
test` fails
- [x] I can manually run a minimal `cargo ci smoketests` invocation on a
Windows machine
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Kasama <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We were using `rust-toolchain` in some places, and `rust-toolchain-file`
in others. I think Rust released a new version, which made the
`rust-toolchain` parts break with:
```
info: downloading component 'rustfmt'
info: removing previous version of component 'rust-src'
info: rolling back changes
error: could not rename component file from '/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/term/terminfo/searcher' to '/root/.rustup/tmp/a4eo07uz83vsyfhk_dir/bk': Invalid cross-device link (os error 18)
Error: Process completed with exit code 1.
```
(Separately, this breakage is confusing.. we'll probably run into this
again when we roll forward our rust version..)
# API and ABI breaking changes
None. CI-only change.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [x] There are no more instances of `rust-toolchain`:
```
$ grep -rIP 'rust-toolchain(?!-file)' .github/workflows
.github/CODEOWNERS:/rust-toolchain.toml @cloutiertyler
```
(on `master`, this finds the instances we changed)
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This changes the ci runs to execute `cargo ci` instead of running
commands directly from the github workflow.
The goal here is to unify the commands under `cargo ci` so that it's
easier and more intuitive to run locally
# API and ABI breaking changes
There are no API/ABI changes.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
Complexity: 1
It is not a complex change as it is mostly localized to the ci runs and
is easily reversible if something goes wrong. The biggest risk here is
to have future CI runs break, which can be remediated by reverting these
changes.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] run `cargo ci` and its subcommands locally
- [x] run the github workflow against this branch to check if the CI
jobs are working properly.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Signed-off-by: Roberto Pommella Alegro <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This has 2 benefits:
1. If the Unity test fails because of a license issue then we don't have
to re-run the C# tests again as part of this flow. Re-running the Unity
tests will be much faster if that's the only thing the job is doing.
2. These tests will run faster because they will now run in parallel as
separate CI jobs.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Both tests pass
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Sets a 2 hour timeout for the smoketests. If the smoketests are taking
more than 2 hours they likely will never finish and we should just time
them out.
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x]
https://github.com/clockworklabs/SpacetimeDB/actions/runs/19697510980/job/56425630374?pr=3772
# Description of Changes
Move anything running on `spacetimedb-runner` to
`spacetimedb-new-runner`
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Stop the CLI doc check from failing on `Cargo.lock` diffs, which it was
doing before.
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI still fails elsewhere if `Cargo.lock` needs updating
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
These tests have been flaky, so we're disabling them until they're
working more consistently.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Tests are indeed skipped? 🤷
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We have added one too many python dependencies. I'm putting them in a
shared file, which will also ease installation for devs running them
locally.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
Existing CI still passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Changes with cache hit:
CI / Smoketests (spacetimedb-runner): 17m -> 10m
CI / Smoketests (windows-latest): 39m -> 31m
CI / Lints: 3m30s -> 2m
unity-testsuite: 23m -> 19m
This included creating a stripped-down `Dockerfile` that only runs
SpacetimeDB (as opposed to e.g. building tracy, flamegraphs, etc.). That
shaves off about 4-5m on the linux smoketests job.
# API and ABI breaking changes
No. CI only.
# Expected complexity level and risk
2
# Testing
Ran existing CI and noted the change in times in the description above
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Speed improvements:
|CI Job|Before|After|Change|
|---|---|---|---|
|CI/Unreal Engine|29m|19m|34% faster|
|CI/Testsuite|21m|11m|47% faster|
|C#/Unity|20m|16m|20% faster|
|CI/Build and test wasm bindings|10m|3m|70% faster|
|CI/Smoketests (linux)|17m|12m|29% faster|
CI/Unreal Engine - switch to serial test execution
CI/Testsuite - partially switch to serial test execution
I also haven't seen an unexpected `Operation cancelled`, so I think that
problem might be behind us.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
No user facing changes
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
2 - this touches a lot of our CI. It has been solid so far but if we
start getting random CI failures this could be the cause.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] All CI passes
(ok as of this update the internal tests are failing but that isn't
related to this PR 😅 )
# Future Work
There are 2 remaining tests that could be optimized by moving them to
faster runners:
1. (~45m) The windows smoketests (difficult because it requires
Windows). @bfops has brought up the idea of only running this test on
PRs that are in the merge queue and I think that would be the best 80/20
here. There really is not a great reason to be running this on every
commit anyway.
2. (36m) The internal tests (easier because it's on Linux)
I would also consider moving from instantiating container images to
instantiating VM snapshots. This would allow us a lot more flexibility
and we wouldn't have to have as much container config in our workflow
files. Based on guides I've seen online this is also somewhat common.
Also managing docker within docker has been kind of a nightmare so it
would be easier to run containers inside of VMs to remove a layer of
containerization there.
## Testsuite inconsistencies
We have at least 2 different jobs that are inconsistent/flaky: the
"testsuite" and the "unreal engine tests".
I'd like @JasonAtClockwork to make the unreal engine tests run
sequentially so that we have less random failures. I know this will
increase the testing time but due to this PR we should have extra
headroom there.
I've also pinged Joshua about one of the tests that I've seen sometimes
fails randomly. This is either an issue in the test or an issue in
SpacetimeDB itself, I'm not sure yet.
These inconsistencies are present both in our custom runner and in the
Github runners, so merging this PR will not make this situation any
worse than it already is.
# Description of Changes
Our `Check that packages are publishable` CI check was silently failing
due to a subshell error not getting propagated upward.
This PR both fixes that error-swallowing behavior, and fixes the errors
that were present.
# API and ABI breaking changes
No. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Check fails if the script to find crates fails
(https://github.com/clockworklabs/SpacetimeDB/actions/runs/19349995837/job/55359458732?pr=3660)
- [x] Check is now passing
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
This adds authorization to the relevant API endpoints, updates the CLI
commands and adds smoketests for the teams feature.
**Note**: Authorizing SQL (incl. subscriptions) is a bit more involved,
and submitted as a separate PR in the series.
Depends-on: https://github.com/clockworklabs/SpacetimeDB/pull/3519
# Description of Changes
I allowed chatgpt to mislead me about how to do these format strings.
Apparently this is the wrong syntax. I've now verified the correct
syntax:
https://docs.github.com/en/actions/reference/workflows-and-actions/expressions#format
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
I honestly don't know how to check what concurrency group something is
running in..
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
When a new commitlog segment is created, allocate disk space for it up
to the maximum segment size. Also do this when resuming writes to an
existing segment, such that segments created without preallocation will
allocate as well when the database is opened.
Preallocation is gated behind the feature "fallocate", because it is not
always desirable to preallocate, e.g. for local `standalone` users.
The feature can only be enabled on Linux targets, because allocation is
done using the Linux-specific `fallocate(2)` system call.
Unlike `ftruncate(2)` or the portable `posix_fallocate(3)`,
`fallocate(2)`
supports allocating disk space without zeroing. This is currently
required, because the commitlog format does not handle padding bytes.
If not enough space can be allocated, the commitlog refuses writes. For
commitlogs that were created without preallocation, this means that the
commitlog cannot even be opened in this situation.
The local durability impl will crash if it detects that the commitlog is
unable to allocate enough space.
This means that a database will eventually crash and be unable to start
in
an out-of-space situation.
Allocated space is not included in the reported size of the commitlog.
Instead, allocated blocks are reported separately.
# Expected complexity level and risk
3 - Disk size monitoring may need to be adjusted.
# Testing
- [x] Adds a test that demonstrates the crash behavior of
[`spacetimedb_durability::Local`]
when there is insufficient space. The test performs I/O against a loop
device.
- [x] Modified the `repo::Memory` impl so that it can run out of space.
No test currently
utilizes this, but existing tests assuming infinite space still pass.
# Description of Changes
The merge queue was (partly) getting borked because we were putting all
non-PR CI events into the same concurrency group, which meant they all
non-PR CI jobs would run sequentially instead of running in parallel.
This sometimes caused _painfully_ long delays in the merge queue.
This was due to my misunderstanding in
https://github.com/clockworklabs/SpacetimeDB/pull/3501#discussion_r2466570395,
where I didn't realize that `cancel-in-progress: false` would cause
everything to queue up.
Now, for non-PR events, we append the commit SHA to the concurrency
group. For merge queue events, this should be the SHA of the ephemeral
merge commit that GH creates, so it will never conflict. For push events
or manual workflow dispatch events, the SHA should be a sane way to
recognize/cancel redundant events.
# API and ABI breaking changes
None. CI-only change.
# Expected complexity level and risk
1
# Testing
- [x] PR CI passes on this PR
- [x] PR CI is still canceled on this PR if a new commit is pushed
Unfortunately it's hard to test the behavior for non-PR events without
merging and seeing if it works.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Based on #3461, which fixes a bug encountered in the quickstart.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
n/a
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Yes
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
These tests fail on external PRs, but not for any real reasons - just
because GH secrets are missing. "Skipped" is more informative than
"failed".
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None, but I just copied the logic from the unity testsuite.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Add `cancel-in-progress` to our GitHub workflows.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Pushing new commits to this PR causes cancels of previous CI runs
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
The quickstart smoketest was not correctly fudging our NuGet
dependencies to use the local versions of the packages, so it was
pulling them from NuGet. This ended up causing issues when we tried to
modify local packages and then use them in a way that affected the
quickstart (e.g.
https://github.com/clockworklabs/SpacetimeDB/pull/3386).
We had a few issues: we weren't ensuring that the local packages were
built, we weren't using the right directory for those packages, and we
weren't adding a "package source mapping" that forced those packages to
be used from the local directory.
# API and ABI breaking changes
None. CI-only.
# Expected complexity level and risk
2
# Testing
- [x] CI passes when
https://github.com/clockworklabs/SpacetimeDB/pull/3386 is merged
together with this PR (it wasn't before).
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Currently based on #3361
Implements most of the TS module API (not yet a function for type
aliases).
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Extremely basic module stuff works
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
See the inline comments for the motivation. This was originally
introduced to our Windows CI in #3351. This PR moves it from CI to
general Windows target builds, since it seems like Windows builds are
now generally having this issue.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Windows CI still passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
v8: use fast static strings for known strings
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fix bindgen tests (due to crate `timezone_provider`)
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Fixes CI tests.
# Description of Changes
We had places that weren't passing `-y`. I assume these were only
working because they were in environments where the things were already
installed, so there wasn't any confirmation dialog. For some reason, the
arm runner now newly needs to install packages, so CI invocations began
failing.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [ ] the "Test spacetimedb-update" flow passes on arm, which is not
currently true elsewhere.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Move the global.json file to the repo root, which should make `dotnet`
resolve the correct SDK regardless of the directory it is run in.
Where a copy of the global.json file is needed, symlink it to the one at
the root.
Should fix "C#/Unity - Test Suite".
# Description of Changes
As the title says.
Add `--pg-port NUM` to the `start` command
# API and ABI breaking changes
Before this, it was set to `5432` unconditionally.
Docs updated at https://github.com/clockworklabs/SpacetimeDB/pull/3302.
# Expected complexity level and risk
1
# Testing
- [x] Run smoke tests
# Description of Changes
This PR removes the `@clockworklabs/typescript-sdk` from the repository
and retains only `spacetimedb` in the `crates/bindings-typescript`
directory. Some files are migrated to `spacetimedb`. I have also updated
the appropriate READMEs.
In addition I have symlinked the old `sdks/typescript` directory to
point to `crates/bindings-typescript`.
# API and ABI breaking changes
This is not technically a breaking change of any kind, although it does
orphan and deprecate the
[@clockworklabs/spacetimedb-sdk](https://www.npmjs.com/package/@clockworklabs/spacetimedb-sdk)
npm package. This package will no longer work with SpacetimeDB.
Users should now install and use the `spacetimedb` package.
# Expected complexity level and risk
2, it's a straightforward change but affects many files.
# Testing
- [ ] I ran `pnpm test` in the `spacetimedb` package
- [ ] I ran the quickstart app
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes#3219
This adds the Unreal SDK, the new Unreal test cases, updates the test
runner to handle Unreal, codegen updates for Unreal, and a QuickStart
Chat.
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
2 - This impacts the subcommand generate.rs to include unrealcpp and
crates/testing to expand for Unreal
# Testing
- [x] Run the new Unreal tests
- [x] Run any previous automation testing - with all the changes to
generate/testing I'm uncertain if there is an impact
- [x] Review the new CLI generate documentation changes
---------
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Closes
[#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686).
Add support for listening using the [PG wire
protocol](https://www.postgresql.org/docs/current/protocol.html) so `pg`
clients could be used against the database.
# API and ABI breaking changes
The output of `duration` is changed to `rfc3339`, instead of the way is
made with `sats` because is what is done in `pg`, see note below.
# Expected complexity level and risk
2
~~There is open questions that are in the [ticket
#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686). Also
the crate used here require `RustTls`, so it could be good idea to
decide if~~:
* ~~Rewrite a big chunk of code to use `OpenSSL`~~
* ~~Move to `RustTls`
https://github.com/clockworklabs/SpacetimeDB/pull/1700~~
* ~~Pay for the extra compilation cost~~.
I open another port(`5433`) to listen for `pg` connections using `ssl`.
Need to be decided if this is the way or instead try to multi-plex the
current port for both protocols.
# Testing
Only manual testing so far. Solving the above questions allow me to
implement some unit tests. Also, not yet integrated into cloud for the
same reasons.
- [x] Adding some test for the binary encoding of special and primitive
types
- [x] Smoke test using `psql` that connect to the db instance and run
some queries
- [x] Manually inspect using a UI database explorer how infer the types,
some of this tools generate special widgets when displaying `json,
duration, etc`
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
This PR:
- standardizes the prettier config across all TypeScript projects
- adds a root level package.json
- standardizes all `pnpm` commands to be the same
- updates documentation accordingly
- adds some additional typescript testing for serialization and
deserialization
**IMPORTANT!** Once this PR merges we will need to change the
`compile-and-test` required check to `build-and-test`
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
2 - It in principle doesn't change any code, but could affect deploy
processes.
# Testing
- [x] Just the automated testing that we had previously
- [x] I added additional automated tests
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
A few of our new CI checks weren't set up to run in the merge queue,
which prevented PRs from merging when those checks were marked required.
# API and ABI breaking changes
None. CI only change.
# Expected complexity level and risk
1
# Testing
This will have to be tested by making these checks required again, and
then seeing if this PR can merge.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
bradleyshep