Files
SpacetimeDB/docs
Ben Snyder 292bda8eed docs: self-hosted prod/test/dev Azure VM guide with key rotation, Azure Key Vault workflows, and rsync data migration pattern (#4545)
## Summary

This PR creates a comprehensive self-hosted JWT key rotation how-to for
SpacetimeDB with reproducible defaults, identity-preserving rotation
guidance, and host-scoped data migration runbooks. It is written for
both operators and automation workflows, with explicit guardrails for
production-like VM environments.

## Key Changes

- Adds a top-level assumptions/risk section for a multi-host topology
(`prod`, `test`, `dev`, optional `local`) including backup prerequisites
before rotation or sync.
- Defines an opinionated, end-to-end path contract for keys:
  - host source: `./.generated/spacetimedb-keys`
  - runtime mount: `/etc/spacetimedb`
- startup args pinned to `/etc/spacetimedb/id_ecdsa` and
`/etc/spacetimedb/id_ecdsa.pub`
- Includes reproducible quickstart scaffolding:
  - OpenSSL commands for compatible ES256/P-256 key generation
  - non-container and Docker startup examples
  - rotation/verify command sequence and marker checks
- Documents one unified tooling surface around `spacetimedb-tooling.ts`
for:
  - rotate (`--dry-run`, `--yes`)
  - verify (`--verify-only`)
  - token continuity (`--resign-token-only`)
- explicit token/key path overrides (`--publisher-cli-toml-path`,
`--private-key-path`)
- Clarifies 401 vs 403 behavior and why identity drift causes publish
failures after rotation.
- Covers all rotation strategies in one place:
  - clean-slate rotation (stateless/dev)
  - identity-preserving rotation (stateful)
  - OIDC-backed identity model for production
- Adds host-scoped migration runbooks for staged sync (`prod -> test ->
dev`) with destination-side token re-signing semantics (run on
destination host context, including SSH examples).
- Explicitly separates conceptual topology guidance from currently
implemented sync primitives in reference tooling to avoid over-claiming.
- Adds operational guardrails for sync promotions:
  - `rsync --delete` is destructive
  - stop/start ordering around sync
- required acceptance gates: key parity, destination re-sign,
restart/redeploy, publish marker validation
- Expands verification and automation content:
  - PEM/parity/fingerprint/newline checks
- AI/automation contract with inputs, outputs, required command order,
and success/failure markers
- troubleshooting flow for `InvalidSignature` and ownership mismatch
errors

## Why This Matters

The most common failure mode in self-hosted fleets is identity drift:
signatures validate, but `spacetime publish` still fails with `403
Forbidden` because destination ownership no longer matches the
publishing identity after key rotation or data movement.

The updated how-to makes these operator requirements explicit:

1. preserve identity continuity across rotations and host promotions
2. re-sign destination tokens after `rsync` in destination host context
3. treat restart/redeploy and publish markers as promotion gates

Following this runbook prevents regressions where key material is
correct but publish still fails due to stale signatures or owner
mismatch state.

## Why JWT Key Rotation Is Essential for Self-Hosters

In self-hosted deployments, operators are the signing authority.
Rotation is both a security control and an operational correctness
requirement.

### Security value

- reduces exposure window for compromised private keys
- invalidates old/stolen tokens after rotation
- mitigates risk from snapshots, backups, clones, and long-lived VM
access

### Operational value

- keeps key material consistent across multi-VM environments
- reduces publish outages after promotion/sync operations
- prevents avoidable `401`/`403` failures in routine release workflows

## Scope

- docs-only change
- public-safe examples only (no internal org names, private vault IDs,
or private secret prefixes)
- includes non-container and Docker workflows, self-publish/versioning
guidance, and marker-based validation (`PUBLISH_SUCCESS`,
`PUBLISH_FAILED`)
- includes data migration workflow guidance (`rsync` + destination
re-sign + restart/redeploy + publish checks)

---------

Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
2026-03-05 02:55:57 +00:00
..
2025-10-24 14:36:38 +00:00
2025-10-24 14:36:38 +00:00
2026-03-01 02:22:59 +00:00
2025-10-24 14:36:38 +00:00
2026-02-20 22:41:39 +00:00
2025-10-24 14:36:38 +00:00
2026-02-20 22:41:39 +00:00
2026-02-20 22:41:39 +00:00

SpacetimeDB Documentation

This repository contains the markdown files which are used to display documentation on our website. This documentation is built using Docusaurus.

Making Edits

To make changes to our docs, you can open a pull request in this repository. You can typically edit the files directly using the GitHub web interface, but you can also clone our repository and make your edits locally.

Instructions

  1. Fork our repository
  2. Clone your fork:
git clone ssh://git@github.com/<username>/SpacetimeDB
cd SpacetimeDB/docs
  1. Make your edits to the docs that you want to make + test them locally (See Testing Locally)
  2. Commit your changes:
git add .
git commit -m "A specific description of the changes I made and why"
  1. Push your changes to your fork as a branch
git checkout -b a-branch-name-that-describes-my-change
git push -u origin a-branch-name-that-describes-my-change
  1. Go to our GitHub and open a PR that references your branch in your fork on your GitHub

CLI Reference Section

To regenerate the CLI reference section, run pnpm generate-cli-docs.

Docusaurus Documentation

For more information on how to use Docusaurus, see the Docusaurus documentation.

Testing Locally

Installation

  1. Make sure you have Node.js installed (version 22 or higher is recommended).
  2. Clone the repository and navigate to the docs directory.
  3. Install the dependencies: pnpm install
  4. Run the development server: pnpm dev, which will start a local server and open a browser window. All changes you make to the markdown files will be reflected live in the browser.

Cutting Docs Versions

Use Docusaurus versioning to snapshot the current docs into versioned_docs.

  1. From docs/, cut a version:
pnpm docusaurus docs:version <version-name>

Example:

pnpm docusaurus docs:version 1.12.0

This updates:

  • docs/versions.json
  • docs/versioned_docs/version-<version-name>/
  • docs/versioned_sidebars/version-<version-name>-sidebars.json

After cutting, update docs/docusaurus.config.ts as needed:

  • lastVersion for the default version at /docs
  • versions.current label/path for prerelease docs
  • versions['<version-name>'] label/banner for the stable snapshot

Re-cutting a Version From an Older Commit

If you need a version snapshot from an old commit (instead of current docs/docs), use:

./docs/scripts/get-old-docs.sh <commit> <version-name>

Example:

./docs/scripts/get-old-docs.sh e45cf891c20d87b11976e1d54c04c0e4639dbe81 1.12.0

The script creates a temporary worktree, snapshots docs from that commit, and copies the generated versioned_docs artifacts back into your current branch.

Absolute links like /quickstarts/react can resolve to the default docs version. To keep links inside the current version, rewrite internal links to relative paths.

Dry run:

pnpm --dir docs rewrite-links

Apply changes:

pnpm --dir docs rewrite-links:write

This script rewrites internal absolute links in:

  • docs/docs (current/prerelease docs)
  • docs/versioned_docs/version-* (all version snapshots)

Adding new pages

All of our directory and file names are prefixed with a five-digit number which determines how they're sorted. We started with the hundreds place as the smallest significant digit, to allow using the tens and ones places to add new pages between. When adding a new page in between two existing pages, choose a number which:

  • Doesn't use any more significant figures than it needs to.
  • Is approximately halfway between the previous and next page.

For example, if you want to add a new page between 00300-foo and 00400-bar, name it 00350-baz. To add a new page between 00350-baz and 00400-bar, prefer 00370-quux or 00380-quux, rather than 00375-quux, to avoid populating the ones place.

To add a new page after all previous pages, use the smallest multiple of 100 larger than all other pages. For example, if the highest-numbered existing page is 01350-abc, create 01400-def.

Best practices

  • Use relative links for linking between documentation pages, as this will ensure that links work correctly with versioning and localization.

License

This documentation repository is licensed under Apache 2.0. See LICENSE.txt for more details