Make it so `HostController` manages both the module host (wasm
machinery) and the database (`RelationalDB` / `DatabaseInstanceContext`)
of spacetime databases deployed to a server.
The `DatabaseInstanceContextController` (DBIC) is removed in the
process.
This allows to make database accesses panic-safe, in that uncaught
panics will cause all resouces to be released and the database to be
restarted on subsequent access. This is a prerequisite for #985.
It also allows to move towards storage of the module binary directly in
the database / commitlog. This patch, however, makes some contortions in
order to **not** introduce a breaking change just yet.
Closes#1024.
Before this change,
we would serialize messages **before** inserting into the send queue.
Because we commit the tx only after inserting into the send queue,
this meant we were holding onto the database lock unnecessarily.
After this change,
we serialize messages **after** inserting into the send queue.
This means we serialize only after committing the tx.
* Rewrite smoketests as python unittests
* Get all tests working and do some work on parallel unittest
* Give up on parallel unittests
* Fix CI + address comments
* Fix skip-clippy arg confusion (just use the env var)
* fix ci
* Add comments
- Move it and friends from sats to vm.
- MemTable now stores a Vec<PV>.
- Other related improvements.
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
* made MutTxId -> MutTx
* adopt using ass. type for tx
* fmt
* read tx in datastore
lint
test
lint
made MutTxId -> MutTx
adopt using ass. type for tx
fmt
added more methods to TxId
fix test
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
comments on Test
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
fmt
typo
* fix imports
* clippy
* read tx in datastore
lint
test
lint
added read tx in subs
fmt
fix iters
* comments
* comments
* fix schema_for_table
* fmt
* merge queue helped
* moved all_read_query anaylyzing to vm
* lint
* added StateView trait
* lint
* fix tests for read-tx-in-subs (#695)
Intention of this PR is to keep diff of main PR smaller - feat: subscription to use read type tx #685 by separating trivial unit test related changes from main logic.
naming changes begin_tx() -> begin_mut_tx() and begin_read_tx() -> begin_tx()
* rollback_tx -> release_tx
* Compiler should work with Write Tx
* execute sql with mut tx
* fix result len
* compiler to work with write tx (#716)
* Compiler should work with Write Tx
* merge queue helped
* Move connection events to reducers
* More rebase fixups
* Avoid double-reference
* Filter out special reducers in generate cli
This updates filtering of `__init__` to exclude all special reducers, as well as moves the filtering to centralised place before calling language-specific generate command.
* Incrememted ABI version number
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
* Saving because I'm testing writing files
* New upgrade program working quite well
* Update license file as well
* Tool seems good to go
* Cargo check is passing, new upgrade-version is ready, old version
removed
* Updating lock file is required for CI to pass
* main.rs clippy lints
* More sensible default
* Version upgrade to 0.7.0 via new version-upgrade util
---------
Co-authored-by: Boppy <no-reply@boppygames.gg>
* Add `address: Address` to `ReducerContext`
Initial support for identifying connections via an `Address`,
passed as the caller address in `ReducerContext`.
Notable design choices:
- The same type `Address` is used for clients and dbs,
as opposed to having distinct `DbAddress` and `ClientAddress` types.
- For automatic reducers (init, scheduled, &c), the passed `Address`
is the database's address, not the address of the client which called `publish`.
- Clients may specify an `Address` as a query parameter
to the `subscribe` and `call` endpoints.
If they do not, an `Address` will be randomly generated for them.
Still to do:
- Send the client's `Address` alongside their `Identity` and token
upon WebSocket init in `IdentityToken`,
so the client can save its `Address` for re-connection.
- SDK support.
- C# module support.
- Documentation.
- Refuse new connections if an existing connection
with the same `(Identity, Address)` pair exists.
- Ensure we can store `Address` in tables,
so modules can track connected clients,
and ser/de `Address`, so those tables can be synced to clients.
* Use `None` in `ReducerContext` for HTTP calls without client address
* Fix typo in trait argument name
* `Address` representation amenable to SDK usage
Similar to `Identity`, make `Address` a product type with a double-underscored field name.
This will allow SDKs to distinguish `Address` from `Vec<u8>`,
but does necessitate some ugly `AddressForUrl` hackery
to get ser/de working correctly in different contexts.
This commit does not yet include SDK support for `Address`,
only the server-side machinery necessary for it.
* Add client_address parameter to /publish; pass to init and update reducers
Per discussion with Kim, it's useful for SpacetimeDB-cloud
for the client_address passed to the init reducer
to be that of the caller of /database/publish,
rather than the database's address.
This commit implements that behavior. Now:
- `init` and `update` take the client_address passed to publish, if any,
or `None` if no client_address was supplied.
- Scheduled reducers never receive a client_address.
- Reducers invoked by HTTP /call take the client_address passed to /call, if any,
or `None` if no client_address was supplied.
- Reducers invoked by WebSocket always receive the address of the WebSocket connection.
* `Address` support in Rust client SDK
* Add test that addresses are stable within a client process
* Run rustfmt against merge changes
* Not sure why my local `cargo fmt` didn't get this...
* Add caller_address argument to reducer arguments in Rust SDK
* rustfmt again...
* Add caller address to `EventJson` message
* Python codegen for client addresses
* C# module support for client addresses
* Fix scoping error
* Add `Address`-related stuff to C# codegen
- Emit `SpacetimeDB.Address` for address types
- Add `callerAddress` field to `ReducerEvent`
* TypeScript codegen changes
* Fix merge conflict with new test
* Run rustfmt
---------
Co-authored-by: Ingvar Stepanyan <me@rreverser.com>
* SpacetimeDB is hanging, might be because of my changes
* Fixed issue - put connect/disconnect in right spot
* Tested, working
* Reverted changes that are irrelevant
* Added test for detecting connect/disconnect when calling from the CLI
* Smoketest fix
* Fix compilation issue with test
* Addressing some feedback
* Updated test for a better test case
* Addressing review
* connect is a substring of disconnect which was causing a failure
* Fixing another test bug, should be correct now 👍
---------
Co-authored-by: John Detter <no-reply@boppygames.gg>
In the client API's log_and_500,
return an ErrorResponse with a body derived from the error,
rather than a bare 500 code.
Note that this exposes previously-internal error messages to users.
* Client API changes to accomodate new cloud architecture
The exact trait interfaces for `client-api` is TBD
* Ensure we're not blocking when accessing the filesystem
* Derive Clone for SendGridController
* Add YOLO error variant to InsertDomainResult
* Rebase
* fixup! Rebase
* Fix SpacetimeType for Address
* Temporarily disable message / frame size limits for SDK WS
* Remove get_database_instance_state from API trait
It's an internal (worker db) thing, which does not need to be satisfied
by impls.
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Fix -S instead of -s for update-module smoke test
-s now means "server", -S "skip clippy", changed in:
a1e9984 (Multiple server configurations for CLI (#214), 2023-09-01)
* Invalidate schema cache when committing a tx
* Use long options in update-module.sh, fix unused warning
* Add test asserting schema_for_table reflects index updates
Fixes#259
(1) Updates MemTable to use RelValue instead of ProductValue
(2) Adds a DataKey member to DataRef and RelValue
(3) Subscriptions compute DataKey only when not present on row
Noa