PublicArchive/SpacetimeDB
2
0
Fork 0
mirror of https://github.com/clockworklabs/SpacetimeDB.git synced 2026-05-06 15:49:35 -04:00
Code Issues Packages Projects Releases Wiki Activity
Files
master
SpacetimeDB/.github/workflows/ci.yml
T
Zeke Foppa 0e8973c306 CI - version upgrade check happens in ci.yml (#4950) 
# Description of Changes

Merged the `upgrade-version-check.yml` into `ci.yml`, and moved the
business logic under `cargo ci`.

I would also be very open to just removing this test until we choose to
define a better suite of tests for `cargo bump-version`.

# API and ABI breaking changes

None. CI only.

# Expected complexity level and risk

1

# Testing

- [x] Ran it locally. It made a diff

---------

Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2026-05-05 20:05:33 +00:00

1256 lines
46 KiB
YAML
Raw Permalink Blame History

on:
pull_request:
push:
branches:
- master
merge_group:
workflow_dispatch:
inputs:
pr_number:
description: "Pull Request Number"
required: false
default: ""
name: CI
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.inputs.pr_number || format('sha-{0}', github.sha) }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: read
jobs:
smoketests:
needs: [lints]
name: Smoketests (${{ matrix.name }})
strategy:
matrix:
include:
- name: Linux
runner: spacetimedb-new-runner-2
- name: Windows
runner: spacetimedb-windows-runner
runs-on: ${{ matrix.runner }}
timeout-minutes: 120
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
SPACETIMEDB_CPP_DIR: ${{ github.workspace }}/crates/bindings-cpp
steps:
- name: Find Git ref
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
PR_NUMBER="${{ github.event.inputs.pr_number || null }}"
if test -n "${PR_NUMBER}"; then
GIT_REF="$( gh pr view --repo clockworklabs/SpacetimeDB $PR_NUMBER --json headRefName --jq .headRefName )"
else
GIT_REF="${{ github.ref }}"
fi
echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
cache-on-failure: false
cache-all-crates: true
cache-workspace-crates: true
prefix-key: v1
- uses: actions/setup-dotnet@v4
with:
global-json-file: global.json
# nodejs and pnpm are required for the typescript quickstart smoketest
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 18
- uses: pnpm/action-setup@v4
with:
run_install: true
# Install emscripten for C++ module compilation tests.
- name: Install emscripten (Linux)
if: runner.os == 'Linux'
shell: bash
run: |
git clone https://github.com/emscripten-core/emsdk.git ~/emsdk
cd ~/emsdk
./emsdk install 4.0.21
./emsdk activate 4.0.21
- name: Install emscripten (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
git clone https://github.com/emscripten-core/emsdk.git $env:USERPROFILE\emsdk
cd $env:USERPROFILE\emsdk
.\emsdk install 4.0.21
.\emsdk activate 4.0.21
- name: Install psql (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
# Fail properly if any individual command fails
$ErrorActionPreference = 'Stop'
$PSNativeCommandUseErrorActionPreference = $true
choco install psql -y --no-progress
# Check for existence, since `choco` doesn't seem to fail the step if it fails to install..
# See https://github.com/clockworklabs/SpacetimeDB/pull/4399 for more background.
Get-Command psql
- name: Update dotnet workloads
if: runner.os == 'Windows'
run: |
# Fail properly if any individual command fails
$ErrorActionPreference = 'Stop'
$PSNativeCommandUseErrorActionPreference = $true
cd modules
# the sdk-manifests on windows-latest are messed up, so we need to update them
dotnet workload config --update-mode manifests
dotnet workload update
- name: Override NuGet packages
shell: bash
run: |
dotnet pack -c Release crates/bindings-csharp/BSATN.Runtime
dotnet pack -c Release crates/bindings-csharp/Runtime
cd sdks/csharp
./tools~/write-nuget-config.sh ../..
# This step shouldn't be needed, but somehow we end up with caches that are missing librusty_v8.a.
# ChatGPT suspects that this could be due to different build invocations using the same target dir,
# and this makes sense to me because we only see it in this job where we mix `cargo build -p` with
# `cargo build --manifest-path` (which apparently build different dependency trees).
# However, we've been unable to fix it so... /shrug
- name: Check v8 outputs
shell: bash
run: |
find "${CARGO_TARGET_DIR}"/ -type f | grep '[/_]v8' || true
if ! [ -f "${CARGO_TARGET_DIR}"/debug/gn_out/obj/librusty_v8.a ]; then
echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
cargo clean -p v8 || true
cargo build -p v8
fi
- name: Install cargo-nextest
uses: taiki-e/install-action@nextest
# --test-threads=1 eliminates contention in the C# tests where they fight over bindings
# build artifacts.
# It also seemed to improve performance a fair amount (11m -> 6m)
- name: Run smoketests (Linux)
if: runner.os == 'Linux'
shell: bash
run: |
if [ -f ~/emsdk/emsdk_env.sh ]; then
source ~/emsdk/emsdk_env.sh
fi
cargo ci smoketests -- --test-threads=1
# Due to Emscripten PATH issues this was separated to make sure OpenSSL still builds correctly
- name: Run smoketests (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
if (Test-Path "$env:USERPROFILE\emsdk\emsdk_env.ps1") {
& "$env:USERPROFILE\emsdk\emsdk_env.ps1" | Out-Null
}
cargo ci smoketests -- --test-threads=1
test:
needs: [lints]
name: Test Suite
runs-on: spacetimedb-new-runner-2
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- name: Find Git ref
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
PR_NUMBER="${{ github.event.inputs.pr_number || null }}"
if test -n "${PR_NUMBER}"; then
GIT_REF="$( gh pr view --repo clockworklabs/SpacetimeDB $PR_NUMBER --json headRefName --jq .headRefName )"
else
GIT_REF="${{ github.ref }}"
fi
echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the smoketests job save the cache since it builds the most things
save-if: false
prefix-key: v1
- uses: actions/setup-dotnet@v3
with:
global-json-file: global.json
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- uses: pnpm/action-setup@v4
with:
run_install: true
# Install cmake and emscripten for C++ module compilation tests.
- name: Install cmake and emscripten
run: |
sudo apt-get update
sudo apt-get install -y cmake
git clone https://github.com/emscripten-core/emsdk.git ~/emsdk
cd ~/emsdk
./emsdk install 4.0.21
./emsdk activate 4.0.21
- name: Install wasm-bindgen CLI
run: |
REQUIRED_WASM_BINDGEN_VERSION="$(
awk '
$1 == "name" && $3 == "\"wasm-bindgen\"" { in_pkg = 1; next }
in_pkg && $1 == "version" {
gsub(/"/, "", $3);
print $3;
exit;
}
' Cargo.lock
)"
if [ -z "${REQUIRED_WASM_BINDGEN_VERSION}" ]; then
echo "Failed to determine wasm-bindgen version from Cargo.lock"
exit 1
fi
INSTALLED_WASM_BINDGEN_VERSION="$(wasm-bindgen --version 2>/dev/null | awk '{print $2}' || true)"
if [ "${INSTALLED_WASM_BINDGEN_VERSION}" != "${REQUIRED_WASM_BINDGEN_VERSION}" ]; then
cargo install --locked --force wasm-bindgen-cli --version "${REQUIRED_WASM_BINDGEN_VERSION}"
fi
wasm-bindgen --version
# Source emsdk environment to make emcc (Emscripten compiler) available in PATH.
- name: Run tests
run: |
source ~/emsdk/emsdk_env.sh
cargo ci test
lints:
name: Lints
runs-on: spacetimedb-new-runner-2
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- name: Checkout sources
uses: actions/checkout@v3
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- run: echo ::add-matcher::.github/workflows/rust_matcher.json
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the smoketests job save the cache since it builds the most things
save-if: false
prefix-key: v1
- uses: actions/setup-dotnet@v3
with:
global-json-file: global.json
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- uses: pnpm/action-setup@v4
with:
run_install: true
- name: Run ci lint
run: cargo ci lint
wasm_bindings:
name: Build and test wasm bindings
runs-on: spacetimedb-new-runner-2
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- uses: actions/checkout@v3
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- run: echo ::add-matcher::.github/workflows/rust_matcher.json
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the smoketests job save the cache since it builds the most things
save-if: false
prefix-key: v1
- name: Run bindgen tests
run: cargo ci wasm-bindings
publish_checks:
name: Check that packages are publishable
runs-on: spacetimedb-new-runner-2
permissions: read-all
steps:
- uses: actions/checkout@v3
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Run checks
run: cargo ci publish-checks
update:
name: Test spacetimedb-update flow (${{ matrix.target }})
permissions: read-all
strategy:
matrix:
include:
- { target: x86_64-unknown-linux-gnu, runner: spacetimedb-new-runner-2 }
- { target: aarch64-unknown-linux-gnu, runner: arm-runner }
- { target: aarch64-apple-darwin, runner: macos-latest }
- { target: x86_64-pc-windows-msvc, runner: windows-latest }
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install Rust
uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Install rust target
run: rustup target add ${{ matrix.target }}
- name: Install packages
if: ${{ matrix.runner == 'arm-runner' }}
shell: bash
run: sudo apt install -y libssl-dev
- name: Test spacetimedb-update
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: cargo ci update-flow --target=${{ matrix.target }} --github-token-auth
unreal_engine_tests:
name: Unreal Engine Tests
# This can't go on e.g. ubuntu-latest because that runner runs out of disk space. ChatGPT suggested that the general solution tends to be to use
# a custom runner.
runs-on: spacetimedb-new-runner-2
# Disable the tests because they are very flaky at the moment.
# TODO: Remove this line and re-enable the `if` line just below here.
if: false
# Skip if this is an external contribution. GitHub secrets will be empty, so the step would fail anyway.
# if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork }}
container:
image: ghcr.io/epicgames/unreal-engine:dev-5.6
credentials:
# Note(bfops): I don't think that `github.actor` needs to match the user that the token is for, because I'm using a token for my account and
# it seems to be totally happy.
# However, the token needs to be for a user that has access to the EpicGames org (see
# https://dev.epicgames.com/documentation/en-us/unreal-engine/downloading-source-code-in-unreal-engine?application_version=5.6)
username: ${{ github.actor }}
password: ${{ secrets.GHCR_TOKEN }}
# Run as root because otherwise we get permission denied for various directories inside the container. I tried doing dances to allow it to run
# without this (reassigning env vars and stuff), but was unable to get it to work and it felt like an uphill battle.
options: --user 0:0
steps:
# Uncomment this before merging so that it will run properly if run manually through the GH actions flow. It was playing weird with rolled back
# commits though.
# - name: Find Git ref
# env:
# GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# shell: bash
# run: |
# PR_NUMBER="${{ github.event.inputs.pr_number || null }}"
# if test -n "${PR_NUMBER}"; then
# GIT_REF="$( gh pr view --repo clockworklabs/SpacetimeDB $PR_NUMBER --json headRefName --jq .headRefName )"
# else
# GIT_REF="${{ github.ref }}"
# fi
# echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Run Unreal Engine tests
working-directory: sdks/unreal
env:
UE_ROOT_PATH: /home/ue4/UnrealEngine
run: |
apt-get update
apt-get install -y acl curl ca-certificates
REPO="$GITHUB_WORKSPACE"
# Let ue4 read/write the workspace & tool caches without changing ownership
for p in "$REPO" "${RUNNER_TEMP:-/__t}" "${RUNNER_TOOL_CACHE:-/__t}"; do
[ -d "$p" ] && setfacl -R -m u:ue4:rwX -m d:u:ue4:rwX "$p" || true
done
# Rust tool caches live under the runner tool cache so they persist
export CARGO_HOME="${RUNNER_TOOL_CACHE:-/__t}/cargo"
export RUSTUP_HOME="${RUNNER_TOOL_CACHE:-/__t}/rustup"
mkdir -p "$CARGO_HOME" "$RUSTUP_HOME"
chown -R ue4:ue4 "$CARGO_HOME" "$RUSTUP_HOME"
# Make sure the UE build script is executable (and parents traversable)
UE_DIR="${UE_ROOT_PATH:-/home/ue4/UnrealEngine}"
chmod a+rx "$UE_DIR" "$UE_DIR/Engine" "$UE_DIR/Engine/Build" "$UE_DIR/Engine/Build/BatchFiles/Linux" || true
chmod a+rx "$UE_DIR/Engine/Build/BatchFiles/Linux/Build.sh" || true
# Run the build & tests as ue4 (who owns the UE tree)
sudo -E -H -u ue4 env \
HOME=/home/ue4 \
XDG_CONFIG_HOME=/home/ue4/.config \
CARGO_HOME="$CARGO_HOME" \
RUSTUP_HOME="$RUSTUP_HOME" \
PATH="$CARGO_HOME/bin:$PATH" \
bash -lc '
set -euxo pipefail
# Install rustup for ue4 if needed (uses the shared caches)
if ! command -v cargo >/dev/null 2>&1; then
curl -sSf https://sh.rustup.rs | sh -s -- -y
fi
rustup show >/dev/null
git config --global --add safe.directory "$GITHUB_WORKSPACE" || true
cd "$GITHUB_WORKSPACE/sdks/unreal"
cargo --version
cargo test -- --test-threads=1
'
ci_command_docs:
name: Check CI command docs
runs-on: spacetimedb-new-runner-2
steps:
- name: Find Git ref
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
PR_NUMBER="${{ github.event.inputs.pr_number || null }}"
if test -n "${PR_NUMBER}"; then
GIT_REF="$( gh pr view --repo clockworklabs/SpacetimeDB $PR_NUMBER --json headRefName --jq .headRefName )"
else
GIT_REF="${{ github.ref }}"
fi
echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Check for docs change
run: cargo ci self-docs --check
cli_docs:
name: Check CLI docs
permissions: read-all
runs-on: spacetimedb-new-runner-2
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- name: Find Git ref
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
PR_NUMBER="${{ github.event.inputs.pr_number || null }}"
if test -n "${PR_NUMBER}"; then
GIT_REF="$( gh pr view --repo clockworklabs/SpacetimeDB $PR_NUMBER --json headRefName --jq .headRefName )"
else
GIT_REF="${{ github.ref }}"
fi
echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- uses: pnpm/action-setup@v4
with:
run_install: true
- name: Get pnpm store directory
shell: bash
run: |
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
- uses: actions/cache@v4
name: Setup pnpm cache
with:
path: ${{ env.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the smoketests job save the cache since it builds the most things
save-if: false
prefix-key: v1
- name: Check for docs change
run: |
cargo ci cli-docs
llm_ci_check:
name: Verify LLM benchmark is up to date
permissions:
contents: read
runs-on: ubuntu-latest
# Disable the tests because they are causing us headaches with merge conflicts and re-runs etc.
if: false
steps:
# Build the tool from master to ensure consistent hash computation
# with the llm-benchmark-update workflow (which also uses master's tool).
- name: Checkout master (build tool from trusted code)
uses: actions/checkout@v4
with:
ref: master
fetch-depth: 1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Install llm-benchmark tool from master
run: |
cargo install --path tools/xtask-llm-benchmark --locked
command -v llm_benchmark
# Now checkout the PR branch to verify its benchmark files
- name: Checkout PR branch
uses: actions/checkout@v4
with:
clean: false
- name: Run hash check (both langs)
run: llm_benchmark ci-check
unity-testsuite:
needs: [lints]
# Skip if this is an external contribution.
# The license secrets will be empty, so the step would fail anyway.
if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork }}
permissions:
contents: read
checks: write
runs-on: spacetimedb-unity-runner
timeout-minutes: 30
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- name: Checkout repository
id: checkout-stdb
uses: actions/checkout@v4
# Run cheap .NET tests first. If those fail, no need to run expensive Unity tests.
- name: Setup dotnet
uses: actions/setup-dotnet@v3
with:
global-json-file: global.json
- name: Override NuGet packages
run: |
dotnet pack crates/bindings-csharp/BSATN.Runtime
dotnet pack crates/bindings-csharp/Runtime
# Write out the nuget config file to `nuget.config`. This causes the spacetimedb-csharp-sdk repository
# to be aware of the local versions of the `bindings-csharp` packages in SpacetimeDB, and use them if
# available. Otherwise, `spacetimedb-csharp-sdk` will use the NuGet versions of the packages.
# This means that (if version numbers match) we will test the local versions of the C# packages, even
# if they're not pushed to NuGet.
# See https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file for more info on the config file.
cd sdks/csharp
./tools~/write-nuget-config.sh ../..
- name: Restore .NET solution
working-directory: sdks/csharp
run: dotnet restore --configfile NuGet.Config SpacetimeDB.ClientSDK.sln
# Now, setup the Unity tests.
- name: Patch spacetimedb dependency in Cargo.toml
working-directory: demo/Blackholio/server-rust
run: |
sed -i "s|spacetimedb *=.*|spacetimedb = \{ path = \"../../../crates/bindings\" \}|" Cargo.toml
cat Cargo.toml
- name: Install Rust toolchain
uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the main CI job save the cache since it builds the most things
save-if: false
prefix-key: v1
# This step shouldn't be needed, but somehow we end up with caches that are missing librusty_v8.a.
# ChatGPT suspects that this could be due to different build invocations using the same target dir,
# and this makes sense to me because we only see it in this job where we mix `cargo build -p` with
# `cargo build --manifest-path` (which apparently build different dependency trees).
# However, we've been unable to fix it so... /shrug
- name: Check v8 outputs
run: |
find "${CARGO_TARGET_DIR}"/ -type f | grep '[/_]v8' || true
if ! [ -f "${CARGO_TARGET_DIR}"/release/gn_out/obj/librusty_v8.a ]; then
echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
cargo clean --release -p v8 || true
cargo build --release -p v8
fi
- name: Install SpacetimeDB CLI from the local checkout
run: |
export CARGO_HOME="$HOME/.cargo"
echo "$CARGO_HOME/bin" >> "$GITHUB_PATH"
cargo install --force --path crates/cli --locked --message-format=short
cargo install --force --path crates/standalone --locked --message-format=short
# Add a handy alias using the old binary name, so that we don't have to rewrite all scripts (incl. in submodules).
ln -sf $CARGO_HOME/bin/spacetimedb-cli $CARGO_HOME/bin/spacetime
- name: Generate client bindings
working-directory: demo/Blackholio/server-rust
run: bash ./generate.sh -y
- name: Check for changes
run: |
tools/check-diff.sh demo/Blackholio/client-unity/Assets/Scripts/autogen || {
echo 'Error: Bindings are dirty. Please run `demo/Blackholio/server-rust/generate.sh`.'
exit 1
}
- name: Hydrate Unity SDK DLLs
run: cargo ci dlls
- name: Check Unity meta files
uses: DeNA/unity-meta-check@v3
with:
enable_pr_comment: ${{ github.event_name == 'pull_request' }}
target_path: sdks/csharp
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: Start SpacetimeDB
run: |
spacetime start &
disown
- name: Publish unity-tests module to SpacetimeDB
working-directory: demo/Blackholio/server-rust
run: |
spacetime logout && spacetime login --server-issued-login local
bash ./publish.sh
- name: Patch com.clockworklabs.spacetimedbsdk dependency in manifest.json
working-directory: demo/Blackholio/client-unity/Packages
run: |
yq e -i '.dependencies["com.clockworklabs.spacetimedbsdk"] = "file:../../../../sdks/csharp"' manifest.json
cat manifest.json
- uses: actions/cache@v3
with:
path: demo/Blackholio/client-unity/Library
key: Unity-${{ github.head_ref }}
restore-keys: Unity-
- name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
- name: Run Unity tests
uses: game-ci/unity-test-runner@v4
with:
unityVersion: 2022.3.32f1 # Adjust Unity version to a valid tag
projectPath: demo/Blackholio/client-unity # Path to the Unity project subdirectory
githubToken: ${{ secrets.GITHUB_TOKEN }}
testMode: playmode
useHostNetwork: true
artifactsPath: ""
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
csharp-testsuite:
needs: [lints]
runs-on: spacetimedb-new-runner-2
timeout-minutes: 30
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- name: Checkout repository
id: checkout-stdb
uses: actions/checkout@v4
# Run cheap .NET tests first. If those fail, no need to run expensive Unity tests.
- name: Setup dotnet
uses: actions/setup-dotnet@v3
with:
global-json-file: global.json
- name: Override NuGet packages
run: |
dotnet pack crates/bindings-csharp/BSATN.Runtime
dotnet pack crates/bindings-csharp/Runtime
# Write out the nuget config file to `nuget.config`. This causes the spacetimedb-csharp-sdk repository
# to be aware of the local versions of the `bindings-csharp` packages in SpacetimeDB, and use them if
# available. Otherwise, `spacetimedb-csharp-sdk` will use the NuGet versions of the packages.
# This means that (if version numbers match) we will test the local versions of the C# packages, even
# if they're not pushed to NuGet.
# See https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file for more info on the config file.
cd sdks/csharp
./tools~/write-nuget-config.sh ../..
- name: Restore .NET solution
working-directory: sdks/csharp
run: dotnet restore --configfile NuGet.Config SpacetimeDB.ClientSDK.sln
- name: Run .NET tests
working-directory: sdks/csharp
run: dotnet test -warnaserror --no-restore
- name: Verify C# formatting
working-directory: sdks/csharp
run: dotnet format --no-restore --verify-no-changes SpacetimeDB.ClientSDK.sln
- name: Install Rust toolchain
uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the main CI job save the cache since it builds the most things
save-if: false
prefix-key: v1
# This step shouldn't be needed, but somehow we end up with caches that are missing librusty_v8.a.
# ChatGPT suspects that this could be due to different build invocations using the same target dir,
# and this makes sense to me because we only see it in this job where we mix `cargo build -p` with
# `cargo build --manifest-path` (which apparently build different dependency trees).
# However, we've been unable to fix it so... /shrug
- name: Check v8 outputs
run: |
find "${CARGO_TARGET_DIR}"/ -type f | grep '[/_]v8' || true
if ! [ -f "${CARGO_TARGET_DIR}"/debug/gn_out/obj/librusty_v8.a ]; then
echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
cargo clean -p v8 || true
cargo build -p v8
fi
find "${CARGO_TARGET_DIR}"/ -type f | grep '[/_]v8' || true
if ! [ -f "${CARGO_TARGET_DIR}"/release/gn_out/obj/librusty_v8.a ]; then
echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
cargo clean --release -p v8 || true
cargo build --release -p v8
fi
- name: Install SpacetimeDB CLI from the local checkout
run: |
export CARGO_HOME="$HOME/.cargo"
echo "$CARGO_HOME/bin" >> "$GITHUB_PATH"
cargo install --force --path crates/cli --locked --message-format=short
cargo install --force --path crates/standalone --features allow_loopback_http_for_tests --locked --message-format=short
# Add a handy alias using the old binary name, so that we don't have to rewrite all scripts (incl. in submodules).
ln -sf $CARGO_HOME/bin/spacetimedb-cli $CARGO_HOME/bin/spacetime
- name: Check quickstart-chat bindings are up to date
working-directory: sdks/csharp
run: |
bash tools~/gen-quickstart.sh
"${GITHUB_WORKSPACE}"/tools/check-diff.sh examples~/quickstart-chat || {
echo 'Error: quickstart-chat bindings have changed. Please run `sdks/csharp/tools~/gen-quickstart.sh`.'
exit 1
}
# TODO: Re-enable this once csharp is using the v2 ws api.
# - name: Check client-api bindings are up to date
# working-directory: sdks/csharp
# run: |
# bash tools~/gen-client-api.sh
# "${GITHUB_WORKSPACE}"/tools/check-diff.sh src/SpacetimeDB/ClientApi || {
# echo 'Error: Client API bindings are dirty. Please run `sdks/csharp/tools~/gen-client-api.sh`.'
# exit 1
# }
- name: Start SpacetimeDB
run: |
spacetime start &
disown
- name: Run regression tests
run: |
bash sdks/csharp/tools~/run-regression-tests.sh
tools/check-diff.sh sdks/csharp/examples~/regression-tests || {
echo 'Error: Bindings are dirty. Please run `sdks/csharp/tools~/gen-regression-tests.sh`.'
exit 1
}
internal-tests:
name: Internal Tests
needs: [lints]
# Skip if not a PR or a push to master
# Skip if this is an external contribution. GitHub secrets will be empty, so the step would fail anyway.
if: ${{ (github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref == 'refs/heads/master'))
&& (github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork) }}
permissions:
contents: read
runs-on: ubuntu-latest
env:
TARGET_OWNER: clockworklabs
TARGET_REPO: SpacetimeDBPrivate
steps:
- id: dispatch
name: Trigger tests
uses: actions/github-script@v7
with:
github-token: ${{ secrets.SPACETIMEDB_PRIVATE_TOKEN }}
script: |
const workflowId = 'ci.yml';
const targetRef = 'master';
const targetOwner = process.env.TARGET_OWNER;
const targetRepo = process.env.TARGET_REPO;
// Use the ref for pull requests because the head sha is brittle (github does some extra dance where it merges in master).
const publicRef = (context.eventName === 'pull_request') ? context.payload.pull_request.head.ref : context.sha;
const publicPrNumber = context.payload.pull_request?.number ?? context.payload.inputs?.pr_number;
const preDispatch = new Date().toISOString();
const inputs = { public_ref: publicRef };
if (publicPrNumber) {
inputs.public_pr_number = String(publicPrNumber);
}
// Dispatch the workflow in the target repository
await github.rest.actions.createWorkflowDispatch({
owner: targetOwner,
repo: targetRepo,
workflow_id: workflowId,
ref: targetRef,
inputs,
});
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
// Find the dispatched run by name
let runId = null;
for (let attempt = 0; attempt < 20 && !runId; attempt++) { // up to ~10 minutes to locate the run
await sleep(5000);
const runsResp = await github.rest.actions.listWorkflowRuns({
owner: targetOwner,
repo: targetRepo,
workflow_id: workflowId,
event: 'workflow_dispatch',
branch: targetRef,
per_page: 50,
});
const expectedName = `CI [public_ref=${publicRef}]`;
const candidates = runsResp.data.workflow_runs
.filter(r => r.name === expectedName && new Date(r.created_at) >= new Date(preDispatch))
.sort((a, b) => new Date(b.created_at) - new Date(a.created_at));
if (candidates.length > 0) {
runId = candidates[0].id;
break;
}
}
if (!runId) {
core.setFailed('Failed to locate dispatched run in the private repository.');
return;
}
const runUrl = `https://github.com/${targetOwner}/${targetRepo}/actions/runs/${runId}`;
core.info(`View run: ${runUrl}`);
core.setOutput('run_id', String(runId));
core.setOutput('run_url', runUrl);
- name: Wait for Internal Tests to complete
uses: actions/github-script@v7
with:
github-token: ${{ secrets.SPACETIMEDB_PRIVATE_TOKEN }}
script: |
const targetOwner = process.env.TARGET_OWNER;
const targetRepo = process.env.TARGET_REPO;
const runId = Number(`${{ steps.dispatch.outputs.run_id }}`);
const runUrl = `${{ steps.dispatch.outputs.run_url }}`;
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
core.info(`Waiting for workflow result... ${runUrl}`);
let conclusion = null;
for (let attempt = 0; attempt < 240; attempt++) { // up to ~2 hours
const runResp = await github.rest.actions.getWorkflowRun({
owner: targetOwner,
repo: targetRepo,
run_id: runId
});
const { status, conclusion: c } = runResp.data;
if (status === 'completed') {
conclusion = c || 'success';
break;
}
await sleep(30000);
}
if (!conclusion) {
core.setFailed('Timed out waiting for private workflow to complete.');
return;
}
if (conclusion !== 'success') {
core.setFailed(`Private workflow failed with conclusion: ${conclusion}`);
}
- name: Cancel invoked run if workflow cancelled
if: ${{ cancelled() }}
uses: actions/github-script@v7
with:
github-token: ${{ secrets.SPACETIMEDB_PRIVATE_TOKEN }}
script: |
const targetOwner = process.env.TARGET_OWNER;
const targetRepo = process.env.TARGET_REPO;
const runId = Number(`${{ steps.dispatch.outputs.run_id }}`);
if (!runId) return;
await github.rest.actions.cancelWorkflowRun({
owner: targetOwner,
repo: targetRepo,
run_id: runId,
});
global_json_policy:
name: Verify global.json files are symlinks
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Find Git ref
env:
PR_NUMBER: ${{ github.event.inputs.pr_number }}
run: |
if [ -n "$PR_NUMBER" ]; then
GIT_REF="refs/pull/$PR_NUMBER/merge"
else
GIT_REF="${{ github.ref }}"
fi
echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
save-if: false
prefix-key: v1
- name: Check global.json policy
run: cargo ci global-json-policy
smoketests_mod_rs_complete:
name: Check smoketests/mod.rs is complete
runs-on: ubuntu-latest
permissions:
contents: read
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target
steps:
- name: Find Git ref
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
PR_NUMBER="${{ github.event.inputs.pr_number || null }}"
if test -n "${PR_NUMBER}"; then
GIT_REF="$( gh pr view --repo clockworklabs/SpacetimeDB $PR_NUMBER --json headRefName --jq .headRefName )"
else
GIT_REF="${{ github.ref }}"
fi
echo "GIT_REF=${GIT_REF}" >>"$GITHUB_ENV"
- name: Checkout sources
uses: actions/checkout@v4
with:
ref: ${{ env.GIT_REF }}
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
cache-on-failure: false
cache-all-crates: true
cache-workspace-crates: true
prefix-key: v1
# This step shouldn't be needed, but somehow we end up with caches that are missing librusty_v8.a.
# ChatGPT suspects that this could be due to different build invocations using the same target dir,
# and this makes sense to me because we only see it in this job where we mix `cargo build -p` with
# `cargo build --manifest-path` (which apparently build different dependency trees).
# However, we've been unable to fix it so... /shrug
- name: Check v8 outputs
shell: bash
run: |
find "${CARGO_TARGET_DIR}"/ -type f | grep '[/_]v8' || true
if ! [ -f "${CARGO_TARGET_DIR}"/debug/gn_out/obj/librusty_v8.a ]; then
echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
cargo clean -p v8 || true
cargo build -p v8
fi
- name: Verify crates/smoketests/tests/smoketests/mod.rs lists all entries
run: |
cargo ci smoketests check-mod-list
docs-build:
name: Docs build
runs-on: spacetimedb-new-runner-2
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '22'
- uses: pnpm/action-setup@v4
with:
run_install: true
- name: Get pnpm store directory
working-directory: sdks/typescript
shell: bash
run: |
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
- uses: actions/cache@v4
name: Setup pnpm cache
with:
path: ${{ env.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Docusaurus build
run: cargo ci docs
typescript-test:
name: TypeScript - Tests
runs-on: spacetimedb-new-runner-2
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- uses: pnpm/action-setup@v4
with:
run_install: true
- name: Get pnpm store directory
shell: bash
working-directory: crates/bindings-typescript
run: |
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
- uses: actions/cache@v4
name: Setup pnpm cache
with:
path: ${{ env.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
# - name: Extract SpacetimeDB branch name from file
# id: extract-branch
# run: |
# # Define the path to the branch file
# BRANCH_FILE=".github/spacetimedb-branch.txt"
# # Default to master if file doesn't exist
# if [ ! -f "$BRANCH_FILE" ]; then
# echo "::notice::No SpacetimeDB branch file found, using 'master'"
# echo "branch=master" >> $GITHUB_OUTPUT
# exit 0
# fi
# # Read and trim whitespace from the file
# branch=$(cat "$BRANCH_FILE" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
# # Fallback to master if empty
# if [ -z "$branch" ]; then
# echo "::warning::SpacetimeDB branch file is empty, using 'master'"
# branch="master"
# fi
# echo "branch=$branch" >> $GITHUB_OUTPUT
# echo "Using SpacetimeDB branch from file: $branch"
- name: Install Rust toolchain
uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ github.workspace }}
shared-key: spacetimedb
# Let the main CI job save the cache since it builds the most things
save-if: false
prefix-key: v1
# # This step shouldn't be needed, but somehow we end up with caches that are missing librusty_v8.a.
# # ChatGPT suspects that this could be due to different build invocations using the same target dir,
# # and this makes sense to me because we only see it in this job where we mix `cargo build -p` with
# # `cargo build --manifest-path` (which apparently build different dependency trees).
# # However, we've been unable to fix it so... /shrug
# - name: Check v8 outputs
# run: |
# find "${CARGO_TARGET_DIR}"/ -type f | grep '[/_]v8' || true
# if ! [ -f "${CARGO_TARGET_DIR}"/debug/gn_out/obj/librusty_v8.a ]; then
# echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
# cargo clean -p v8 || true
# cargo build -p v8
# fi
# if ! [ -f "${CARGO_TARGET_DIR}"/release/gn_out/obj/librusty_v8.a ]; then
# echo "Could not find v8 output file librusty_v8.a; rebuilding manually."
# cargo clean --release -p v8 || true
# cargo build --release -p v8
# fi
# - name: Install SpacetimeDB CLI from the local checkout
# run: |
# export CARGO_HOME="$HOME/.cargo"
# echo "$CARGO_HOME/bin" >> "$GITHUB_PATH"
# cargo install --force --path crates/cli --locked --message-format=short
# cargo install --force --path crates/standalone --locked --message-format=short
# # Add a handy alias using the old binary name, so that we don't have to rewrite all scripts (incl. in submodules).
# ln -sf $CARGO_HOME/bin/spacetimedb-cli $CARGO_HOME/bin/spacetime
# # Clear any existing information
# spacetime server clear -y
- name: Run TypeScript tests
run: cargo ci typescript-test
# - name: Run quickstart-chat tests
# working-directory: examples/quickstart-chat
# run: pnpm test
#
# # Run this step always, even if the previous steps fail
# - name: Print rows in the user table
# if: always()
# run: spacetime sql quickstart-chat "SELECT * FROM user"
version_upgrade_check:
runs-on: spacetimedb-new-runner-2
steps:
- name: Checkout
uses: actions/checkout@v3
- uses: dsherret/rust-toolchain-file@v1
- name: Set default rust toolchain
run: rustup default $(rustup show active-toolchain | cut -d' ' -f1)
# pnpm is required for regenerating the typescript bindings
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 20
- uses: pnpm/action-setup@v4
with:
run_install: true
- name: Verify that upgrade-version still works
run: cargo ci version-upgrade-check
- name: Show diff
run: git diff HEAD
Reference in New Issue View Git Blame Copy Permalink