Smoketests require pre-built binaries and have their own dedicated
command (cargo ci smoketests) that builds binaries first. Exclude
them from cargo test --all to prevent failures in CI.
- Rename existing smoketests job to use cargo smoketest (xtask) which
handles pre-building binaries, using nextest, and optimal parallelism
- Add cargo-nextest installation step for better parallel execution
- Add new "Smoketests (Python Legacy)" job that runs the old Python
smoketests alongside the Rust ones for comparison during transition
- Add warn-python-smoketests job that posts a PR comment when Python
smoketests are modified, encouraging use of Rust smoketests instead
- Update smoketests/README.md to note the transition to Rust smoketests
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
This PR started as an experiment to see if restricting the amount of
test job threads would reduce the flakiness in the `Test Suite` CI job.
I have tested several values here (`1`, `2` and `10`) and I believe 2 is
very stable. I'm also very confused with this change because it seems
like somehow the `Test Suite` CI job is now faster than it was before:
Before:
<img width="502" height="57" alt="image"
src="https://github.com/user-attachments/assets/d944041c-8ccc-4382-a8a7-06e7ff5cba12"
/>
After:
<img width="481" height="46" alt="image"
src="https://github.com/user-attachments/assets/49b89143-b201-4a1a-820c-13e5b3feda76"
/>
It's possible that having less test threads means less cpu contention on
the github runners, but I'm not completely sure. It could also somehow
be less lock contention in SpacetimeDB. I have looked at the output of
the `Test Suite` job and it does appear that all tests are running.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
0 - this only updates our workflow and has no impact on the code.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] CI is passing and `Test Suite` is less flakey
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
* Added a new `cargo ci dlls` subcommand to build/pack the in-repo C#
NuGet packages and the C# SDK.
* `cargo ci dlls` restores `sdks/csharp/SpacetimeDB.ClientSDK.csproj`
using the freshly built local package outputs as to populate
`sdks/csharp/packages/**`.
* Added a Unity `.meta` skeleton under
`sdks/csharp/unity-meta-skeleton~/**` and overlays those `.meta` files
onto the latest restored versioned package directory to keep Unity GUIDs
stable and import settings consistent.
* Unity-specific import fixes are captured in the skeleton overlay
(notably: preventing Unity from importing incompatible TFMs like
`net8.0`, and marking analyzer DLLs with the `RoslynAnalyzer` label so
Unity can recognize them).
# How to use (local)
```bash
# Build/pack + restore local packages into sdks/csharp/packages/**
cargo ci dlls
```
# API and ABI breaking changes
N/A
# Expected complexity level and risk
2 - Local developer tooling + file overlay into restore output; no
runtime/SDK API behavior changes.
# Testing
- [x] `cargo check -p ci`
- [x] Ran `cargo ci dlls` and verified the output under
`sdks/csharp/packages/**` and the various NuGet package locations.
- [x] Tested a Unity project importing the SpacetimeDB SDK after
generating output and confirmed no errors.
---------
Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Make `cargo ci` work properly on Windows, in preparation for
https://github.com/clockworklabs/SpacetimeDB/pull/3702.
# API and ABI breaking changes
No. CI-only.
# Expected complexity level and risk
2. Not trivial, but not complicated.
# Testing
- [x] CI output seems to be genuinely running the tests, and it's
passing on Windows
- [x] Make a change to `crates/bindings-csharp` and see that `cargo ci
test` fails
- [x] I can manually run a minimal `cargo ci smoketests` invocation on a
Windows machine
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Kasama <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Add back the instructions for regenerating CLI docs, which were removed
in https://github.com/clockworklabs/SpacetimeDB/pull/3343. I also made a
script for it.
This also fixes the CI checking this file, which was silently broken in
the same PR.
I have **not** verified that this works in Git Bash in Windows.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [x] CI fails if I change the CLI reference
- [x] CLI reference looks visually reasonable on a local `pnpm dev`
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This changes the ci runs to execute `cargo ci` instead of running
commands directly from the github workflow.
The goal here is to unify the commands under `cargo ci` so that it's
easier and more intuitive to run locally
# API and ABI breaking changes
There are no API/ABI changes.
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
Complexity: 1
It is not a complex change as it is mostly localized to the ci runs and
is easily reversible if something goes wrong. The biggest risk here is
to have future CI runs break, which can be remediated by reverting these
changes.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] run `cargo ci` and its subcommands locally
- [x] run the github workflow against this branch to check if the CI
jobs are working properly.
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Signed-off-by: Roberto Pommella Alegro <robertoaall@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Tyler Cloutier