* Move connection events to reducers
* More rebase fixups
* Avoid double-reference
* Filter out special reducers in generate cli
This updates filtering of `__init__` to exclude all special reducers, as well as moves the filtering to centralised place before calling language-specific generate command.
* Incrememted ABI version number
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
* Saving because I'm testing writing files
* New upgrade program working quite well
* Update license file as well
* Tool seems good to go
* Cargo check is passing, new upgrade-version is ready, old version
removed
* Updating lock file is required for CI to pass
* main.rs clippy lints
* More sensible default
* Version upgrade to 0.7.0 via new version-upgrade util
---------
Co-authored-by: Boppy <no-reply@boppygames.gg>
* core: Provide read access to commit/message log and odb
This is a first approximation to provide data access for Kafka-style
replication. It punts on notifications of segment rotation (which
could also be solved via filesystem events).
ObjectDB access is fairly crude, and will likely require it's own
replication subsystem.
* fixup! Merge remote-tracking branch 'origin/master' into kim/log-access
* Add some tests
Also make max segment size configurable, so tests don't have to write
>= 1GiB worth of data.
* Abide to the Rust naming conventions
* Add test for commit iter
* Remove `MessageLogIter`, use `commit_log::Iter` instead (#272)
* Saving because I'm testing writing files
* New upgrade program working quite well
* Update license file as well
* Tool seems good to go
* Cargo check is passing, new upgrade-version is ready, old version
removed
* Updating lock file is required for CI to pass
* main.rs clippy lints
* More sensible default
---------
Co-authored-by: Boppy <no-reply@boppygames.gg>
* Print the elapsed time & # of returned rows in the interactive repl
* Fixed clippy warning
* Change formatting of timings to be like psql & remove it from direct sql calls
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
* Commit test clients' module_bindings
Having discussed with the team, I've come around to it being correct
to commit the `module_bindings` for the SDK tests' two test clients.
This way, people not working on codegen can run `cargo test --workspace` &c,
and open the test clients with rust-analyzer without any additional rigamarole.
The downside is that any time codegen changes,
we'll see very large commits touching all of these files,
but consensus is we don't really care.
An additional upside is that now we can run clippy against the bindings in CI,
which will incentivize ensuring that the bindings generate lint-free code.
* Add linguist-generated=true for generated module bindings
* Top-level .gitattributes sets `module_bindings` as generated
We've had continual issues with test isolation when developing breaking changes.
This commit doesn't fully address those, but is a step in the right direction:
the SDK tests now create a tempdir as their `STDB_PATH`,
rather than using `~/.spacetime`.
* Move ABI check to instantiation
This is a follow-up to #197.
This fixes an extra case I was worried about even with that fix but didn't hit it in my initial examples so hoped it's not worth the complexity.
Now I've hit it in a different module: because ABI can have zeroes - e.g. 3.0 is encoded as `\00\00\03\00` in the data segment - linker might place it in the beginning of a data segment, in which case `wasm-opt` will trim those prefix \00 bytes and there won't be any single data segment containing the entire version, and the ABI check will still fail.
It's possible to make the `determine_spacetime_abi` account for this case and merge chunks from different data segments when they overlap with version, but I feel it's growing to the point where it doesn't justify the complexity.
Instead, I'm moving the check into Wasmer's instantiation, since we construct a Wasm instance right away anyway, and it takes care of zero-filling and loading all the data segments into memory.
This makes the check more reliable and simpler, as well as avoids double-parsing of Wasm, but it does mean that we'll need to copy it to other hosts in the unlikely event we want to support 2 Wasm hosts simultaneously in the future.
IMO in this case the tradeoff is worth it.
* Add comment explanation
* Fixup conflict resolution
Signed-off-by: Ingvar Stepanyan <me@rreverser.com>
---------
Signed-off-by: Ingvar Stepanyan <me@rreverser.com>
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
* Add `address: Address` to `ReducerContext`
Initial support for identifying connections via an `Address`,
passed as the caller address in `ReducerContext`.
Notable design choices:
- The same type `Address` is used for clients and dbs,
as opposed to having distinct `DbAddress` and `ClientAddress` types.
- For automatic reducers (init, scheduled, &c), the passed `Address`
is the database's address, not the address of the client which called `publish`.
- Clients may specify an `Address` as a query parameter
to the `subscribe` and `call` endpoints.
If they do not, an `Address` will be randomly generated for them.
Still to do:
- Send the client's `Address` alongside their `Identity` and token
upon WebSocket init in `IdentityToken`,
so the client can save its `Address` for re-connection.
- SDK support.
- C# module support.
- Documentation.
- Refuse new connections if an existing connection
with the same `(Identity, Address)` pair exists.
- Ensure we can store `Address` in tables,
so modules can track connected clients,
and ser/de `Address`, so those tables can be synced to clients.
* Use `None` in `ReducerContext` for HTTP calls without client address
* Fix typo in trait argument name
* `Address` representation amenable to SDK usage
Similar to `Identity`, make `Address` a product type with a double-underscored field name.
This will allow SDKs to distinguish `Address` from `Vec<u8>`,
but does necessitate some ugly `AddressForUrl` hackery
to get ser/de working correctly in different contexts.
This commit does not yet include SDK support for `Address`,
only the server-side machinery necessary for it.
* Add client_address parameter to /publish; pass to init and update reducers
Per discussion with Kim, it's useful for SpacetimeDB-cloud
for the client_address passed to the init reducer
to be that of the caller of /database/publish,
rather than the database's address.
This commit implements that behavior. Now:
- `init` and `update` take the client_address passed to publish, if any,
or `None` if no client_address was supplied.
- Scheduled reducers never receive a client_address.
- Reducers invoked by HTTP /call take the client_address passed to /call, if any,
or `None` if no client_address was supplied.
- Reducers invoked by WebSocket always receive the address of the WebSocket connection.
* `Address` support in Rust client SDK
* Add test that addresses are stable within a client process
* Run rustfmt against merge changes
* Not sure why my local `cargo fmt` didn't get this...
* Add caller_address argument to reducer arguments in Rust SDK
* rustfmt again...
* Add caller address to `EventJson` message
* Python codegen for client addresses
* C# module support for client addresses
* Fix scoping error
* Add `Address`-related stuff to C# codegen
- Emit `SpacetimeDB.Address` for address types
- Add `callerAddress` field to `ReducerEvent`
* TypeScript codegen changes
* Fix merge conflict with new test
* Run rustfmt
---------
Co-authored-by: Ingvar Stepanyan <me@rreverser.com>
Based on my limited understanding of Criterion,
I think we can get add transactions per second reporting to benchmarks
using Criterion's Throughput::Elements tracking,
by treating transactions as elements.
This is not ideal, as the output says e.g.
`thrpt: [38.763 Kelem/s 38.781 Kelem/s 38.799 Kelem/s]`,
where we'd like `Ktx/s`,
but it's not clear whether Criterion allows us that much customization.
Each of the benchmarks currently does one tx per iteration,
so pass `Throughput::Elements(1)` for each of them.
Do this as close to the test as possible, despite redundancy,
in case we ever add benchmarks to the same group
which do multiple transactions per iteration.
* SpacetimeDB is hanging, might be because of my changes
* Fixed issue - put connect/disconnect in right spot
* Tested, working
* Reverted changes that are irrelevant
* Added test for detecting connect/disconnect when calling from the CLI
* Smoketest fix
* Fix compilation issue with test
* Addressing some feedback
* Updated test for a better test case
* Addressing review
* connect is a substring of disconnect which was causing a failure
* Fixing another test bug, should be correct now 👍
---------
Co-authored-by: John Detter <no-reply@boppygames.gg>
Prior to this commit, the Rust SDK passed autogenerated dispatch functions around as
function pointers (i.e. values of `fn` types).
This was brittle and inextensible; adding a new dispatch function
or changing the type of a dispatch function
required threading a new argument through `BackgroundDbConnection::connect`,
and a new field to whatever struct was supposed to invoke the dispatch function.
This commit adds a new trait `SpacetimeModule`,
which has a method for each autogenerated function.
`BackgroundDbConnection::connect` accepts an `Arc<dyn SpacetimeModule>`,
which it passes to various components which need access to the autogenerated methods.
The CLI's `generate` with `--lang rust` generates a unit struct `Module`,
which implements the trait.
The generated `connect` function passes `Arc::new(Module)` to the internal connect method.
This greatly reduces the maintenance burden of adding new dispatch functions.
Now, we only need to alter:
- The definition of `trait SpacetimeModule`.
- The generation of the instance.
- Whatever internals call the method.
In the client API's log_and_500,
return an ErrorResponse with a body derived from the error,
rather than a bare 500 code.
Note that this exposes previously-internal error messages to users.
Fixes#332.
If an index semijoin is to be incrementally evaluated,
it is first converted to an inner join which supports MemTables.
As part of this change, I had to abstract the projection part of the
inner join spec to be able to express semijoins.
* Store the current module hash in a system table
* Add more user logs
Fill the user-retrievable database log with more info about what is
going on while a database is being initialized or updated.
* Silence unused warning
* sats: Add `field_as_u64` to `ProductValue`
* db: Add `epoch` to st_module
Add a u64 field `epoch` to the st_module table, used to store a fencing
token.
* core: Add a way to obtain a keyed lock to `ControlDB`
* host: Thread through the fencing token
* standalone: Implement module lifecycle using the new locking facilities
* test: Fix update-module
Need another line of logs, as there is now more output.
* Refine API
Make getting and setting the program hash trait methods. Also widen the
epoch / fencing token to fit stdb sequences.
* db: Hand out opaque index / sequence ids instead of numbers
This allows to actually use relational db methods taking those types as
arguments outside the core crate.
Also make `next_sequence` and `create_sequence` not take `&mut self`
unnecessarily.
* Fix type error
* Fix test
* Working on this but found another bug
* Fixing describe
* Working on this but found another bug
* Tested working, have to update tests
* Updated tests, hopefully I caught everything
* Clippy fix
* Addressing more clippy issues
* Cargo fmt
* Implementing Phoebe's feedback
---------
Co-authored-by: John Detter <no-reply@boppygames.gg>
Piotr Sarnacki