* core: Fix host controller to not replace module if lifecyle hooks failed
Previously, `spawn_module_host` would unconditionally insert the new
module into the controller state, and not remove it if the lifecycle
hooks (`init_database` / `update_database`) returned an error.
This would mean that the module code was replaced with the new one, even
if it should be rejected because the schema was not updated or the init
/ update reducer failed.
Fix this by starting the module, and later "committing" it to the
controller state in two phases.
* Add commentary about database mutations / transactionality
The `TableDef` type is nominally the same for the database schema and
the schema description from a module, but only the former contains
information about indexes and sequences. This would cause all indexes to
be dropped when updating a database, because none of the existing
indexes would ever be present in the module's schema definition.
It is unclear if the functionality could be recovered, but it does not
seem to be robust without a dedicated schema manipulation language
either way.
We still want to allow module updates which do not change the schema at
all, or only introduce new tables, as otherwise users may be forced to
re-run expensive initialisation steps even though they only introduced
debugging code or somesuch.
To make this work, a somewhat dubious equivalence comparison is made,
comparing the constraint definitions in a way deemed correct in post-hoc
analysis.
* Store the current module hash in a system table
* Add more user logs
Fill the user-retrievable database log with more info about what is
going on while a database is being initialized or updated.
* Silence unused warning
* sats: Add `field_as_u64` to `ProductValue`
* db: Add `epoch` to st_module
Add a u64 field `epoch` to the st_module table, used to store a fencing
token.
* core: Add a way to obtain a keyed lock to `ControlDB`
* host: Thread through the fencing token
* standalone: Implement module lifecycle using the new locking facilities
* test: Fix update-module
Need another line of logs, as there is now more output.
* Refine API
Make getting and setting the program hash trait methods. Also widen the
epoch / fencing token to fit stdb sequences.
* db: Hand out opaque index / sequence ids instead of numbers
This allows to actually use relational db methods taking those types as
arguments outside the core crate.
Also make `next_sequence` and `create_sequence` not take `&mut self`
unnecessarily.
* Fix type error
* Fix test
* Working on this but found another bug
* Fixing describe
* Working on this but found another bug
* Tested working, have to update tests
* Updated tests, hopefully I caught everything
* Clippy fix
* Addressing more clippy issues
* Cargo fmt
* Implementing Phoebe's feedback
---------
Co-authored-by: John Detter <no-reply@boppygames.gg>
* Client API changes to accomodate new cloud architecture
The exact trait interfaces for `client-api` is TBD
* Ensure we're not blocking when accessing the filesystem
* Derive Clone for SendGridController
* Add YOLO error variant to InsertDomainResult
* Rebase
* fixup! Rebase
* Fix SpacetimeType for Address
* Temporarily disable message / frame size limits for SDK WS
* Remove get_database_instance_state from API trait
It's an internal (worker db) thing, which does not need to be satisfied
by impls.
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Fix -S instead of -s for update-module smoke test
-s now means "server", -S "skip clippy", changed in:
a1e9984 (Multiple server configurations for CLI (#214), 2023-09-01)
* Invalidate schema cache when committing a tx
* Use long options in update-module.sh, fix unused warning
* Add test asserting schema_for_table reflects index updates
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Invalidate schema cache when committing a tx
* Use long options in update-module.sh, fix unused warning
* Add test asserting schema_for_table reflects index updates
Alters the CLI's configuration format to support storing multiple server configurations,
and having each server configuration store an (optional) default identity.
Many CLI commands take an additional -s argument, the server on which to operate,
which defaults to the configured default_server.
Using -s consistently requires renaming the publish skip-clippy short flag to -S.
Per discussion with @jdetter , this PR does not directly associate identities with servers.
Instead, each server configuration stores the server's "fingerprint" (i.e. PEM-formatted ECDSA public key),
and the CLI uses that public key to decode tokens to determine if they apply to a given server.
This architecture allows the CLI to behave reasonably
when multiple server configurations use the same set of tokens,
e.g. if multiple distinct URLs resolve to the same SpacetimeDB instance.
For example, one could imagine a configuration with server configurations for both http://127.0.0.1:3000
and http://localhost:3000, which should use the same set of identities.
* Working on improving commands that use identities
* Fix lints
* Reverted file that shouldn't have changed
* Found and fixed all other todos
* Addressed more CLI TODOs
* Fixes for formatting issues
* Set names of identities
* Set name of identities + clippy
* Small fix
* Added the start of a doc comment, switching over to another PR
* Fixed tests that needed to be updated
* Addressed more feedback and fixed several clippy issues
* Small fix
* Apply suggestions from code review
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
* Added more doc comments
* Addressing more feedback
* Fixed really old bug in SpacetimeDB
* Tests to verify new functionality
* Fix clippy lints
* Email during identity creation is optional
* Some work
* Getting smoketests working on mac
* All tests are passing except known failing tests
* Working on parallel smoketests
* Fixed some bugs in saving configs that was preventing this from working
* Fixes required for parallel tests
* Tests are working in parallel
* Pruned changes
* retab
* re-retab
* retab the lib file
* Cargo profile for building more quickly
* I have to rebase on another PR
* smoketest fixes
* create_project and reset_project are now the same thing, removed
create_project
* More fixes
* Removed print statement
* Small fix
* Another fix
* Tons of improvements to the smoketests
* Have to rebase on master
* Small fixes
* More progress
* Finally working correctly!
* Apparently we're missing this
* Enable command output
* Listing installed targets
* Clean before building
* What is going on
* Something super wonky going on
* Another test
* Skip building containers for now
* Small fix
* Test using cargo instead
* Changed workflow a bit
* CI is stuck
* Small fix
* Another fix
* Try cargo run instead of building spacetime CLI
* Removed workflow step
* Fixing all of the tests
* Identity test
* Tests should finally be working
* Enable debug
* Remove spacetime from path
* Another try
* Logic fix
* Another fix
* Another fix
* Working now?
* Another fix
* Finally working again
* Adding github containers back in
* CI fix
* Use SpacetimeDB Large Runner
* Updated test to get more output
* Changed 0ms to 10ms to improve parallel test stability
* Removed unused logs
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
* Removed unnecessary reset_project
* Removed reset_config where its not needed
* Reset template
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Boppy <no-reply@boppygames.gg>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
`IndexDef` contains a table id, which is not yet known (i.e. zero) when
constructing `TableDef` from the module describers. But it /is/ known
for the schema obtained from the database catalog, so schemas compare
inequal even if they're structurally equivalent.
Fixed by just updating the proposed schema for known tables before
comparing. Also update the smoke test to contain an indexed column.
Kim Altintop