Closes#747.
Before this change,
we would evaluate each and every query,
for each and every subscription,
on each and every row update.
If N subscriptions had a query Q in common,
it would be evaluated N different times.
With this change,
distinct queries are evaluated once,
and the results copied for each client.
So in the example above, Q would be evaluated once,
with the results transmitted to N different clients.
* chore: Remove unused metrics
* chore: Move table size metrics back into core
Table size metrics were previously moved out of core.
This was due to the query planner needing access to them.
However that dependency was ultimately managed differently via #677.
Query optimization will soon need this metric.
Therefore it must be moved out of core and into lib.
Eventually we might want to move all metrics into lib.
* Refactoring bootstrapping and adding full support for constraints & multi-column indexes
* Removed deleted code and add some comments
* Add comment for Constraints
* Update crates/core/src/db/datastore/locking_tx_datastore/mod.rs
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
* Update crates/core/src/db/datastore/locking_tx_datastore/mod.rs
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
* Addressing some PR comments & remove cloning
* Adding details for when fail to decode rows in replay
* Apply Kim's fix for replay bug
* Addressing some PR comments
* Merge
* Fix conflicts
* Fix conflicts
* Fix test
* Fix generation of code for csharp & typescript, now it generate FilterBy for all columns
* Fix for index generation
* Fixing C# index generation
* Merge
* Fix compilation on csharp bindings
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: John Detter <no-reply@boppygames.gg>
* Format binary data as hex in sql/satn
* Add json test for identity
* Addressing some PR comments
* Fix test
---------
Signed-off-by: Mario Montoya <mamcx@elmalabarista.com>
* Moving object defs to SATS
* move ColumnIndexAttribute to primitives, nix path attr
* revert unnecessary reformating
* ProductValue: don't take ColId
ProductValues can be more than rows and can be nested
* simplify sats::db::def wrt. Constraints + impl ser/de
* cargo fmt
---------
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
There is no guarantee that a named field is in the supplied table
header, nor that it is in the row. Panicking at this location can be
disastrous, while all call sites are inside Result already. So, return
Result.
* Fix authorization check for CrudCode
Make all mutating SQL operations require the caller to be the database
owner.
* Add auth check tests + ensure all tables in a join are considered
* Move connection events to reducers
* More rebase fixups
* Avoid double-reference
* Filter out special reducers in generate cli
This updates filtering of `__init__` to exclude all special reducers, as well as moves the filtering to centralised place before calling language-specific generate command.
* Incrememted ABI version number
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
* Add `address: Address` to `ReducerContext`
Initial support for identifying connections via an `Address`,
passed as the caller address in `ReducerContext`.
Notable design choices:
- The same type `Address` is used for clients and dbs,
as opposed to having distinct `DbAddress` and `ClientAddress` types.
- For automatic reducers (init, scheduled, &c), the passed `Address`
is the database's address, not the address of the client which called `publish`.
- Clients may specify an `Address` as a query parameter
to the `subscribe` and `call` endpoints.
If they do not, an `Address` will be randomly generated for them.
Still to do:
- Send the client's `Address` alongside their `Identity` and token
upon WebSocket init in `IdentityToken`,
so the client can save its `Address` for re-connection.
- SDK support.
- C# module support.
- Documentation.
- Refuse new connections if an existing connection
with the same `(Identity, Address)` pair exists.
- Ensure we can store `Address` in tables,
so modules can track connected clients,
and ser/de `Address`, so those tables can be synced to clients.
* Use `None` in `ReducerContext` for HTTP calls without client address
* Fix typo in trait argument name
* `Address` representation amenable to SDK usage
Similar to `Identity`, make `Address` a product type with a double-underscored field name.
This will allow SDKs to distinguish `Address` from `Vec<u8>`,
but does necessitate some ugly `AddressForUrl` hackery
to get ser/de working correctly in different contexts.
This commit does not yet include SDK support for `Address`,
only the server-side machinery necessary for it.
* Add client_address parameter to /publish; pass to init and update reducers
Per discussion with Kim, it's useful for SpacetimeDB-cloud
for the client_address passed to the init reducer
to be that of the caller of /database/publish,
rather than the database's address.
This commit implements that behavior. Now:
- `init` and `update` take the client_address passed to publish, if any,
or `None` if no client_address was supplied.
- Scheduled reducers never receive a client_address.
- Reducers invoked by HTTP /call take the client_address passed to /call, if any,
or `None` if no client_address was supplied.
- Reducers invoked by WebSocket always receive the address of the WebSocket connection.
* `Address` support in Rust client SDK
* Add test that addresses are stable within a client process
* Run rustfmt against merge changes
* Not sure why my local `cargo fmt` didn't get this...
* Add caller_address argument to reducer arguments in Rust SDK
* rustfmt again...
* Add caller address to `EventJson` message
* Python codegen for client addresses
* C# module support for client addresses
* Fix scoping error
* Add `Address`-related stuff to C# codegen
- Emit `SpacetimeDB.Address` for address types
- Add `callerAddress` field to `ReducerEvent`
* TypeScript codegen changes
* Fix merge conflict with new test
* Run rustfmt
---------
Co-authored-by: Ingvar Stepanyan <me@rreverser.com>
* Initial draft for multi-column indexes
* Fix index seek for multi-columns
* Restore use of ColId
* Fix compilation
* Merge
* Addressing some PR comments
* Clippy
* Clarify usage of AlgebraicValue for indexes
* .
We've had recurring issues with `println` calls sneaking in
where `log` crate macros would be more appropriate.
This commit adds a Clippy warning for uses of the global I/O macros,
i.e. `print`, `println`, `eprint`, `eprintln` and `dbg`.
The lint is disabled by a more-specific `clippy.toml` in the `cli` and `sqltest` crates,
as well as using `allow` attributes in `standalone`'s `subscommands` module.
Additionally, this commit converts a handful of prints in `core/utils` to `log::info`.
* Client API changes to accomodate new cloud architecture
The exact trait interfaces for `client-api` is TBD
* Ensure we're not blocking when accessing the filesystem
* Derive Clone for SendGridController
* Add YOLO error variant to InsertDomainResult
* Rebase
* fixup! Rebase
* Fix SpacetimeType for Address
* Temporarily disable message / frame size limits for SDK WS
* Remove get_database_instance_state from API trait
It's an internal (worker db) thing, which does not need to be satisfied
by impls.
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Update indexes when updating a database
It turns out that the order of the index definitions in the proposed
schema may differ from those returned from the catalog, causing valid
(i.e. no-op) updates to be rejected. While at it, allow updating table
indexes so as long as the (column) schema remains unchanged.
* Fix -S instead of -s for update-module smoke test
-s now means "server", -S "skip clippy", changed in:
a1e9984 (Multiple server configurations for CLI (#214), 2023-09-01)
* Invalidate schema cache when committing a tx
* Use long options in update-module.sh, fix unused warning
* Add test asserting schema_for_table reflects index updates
Fixes#259
(1) Updates MemTable to use RelValue instead of ProductValue
(2) Adds a DataKey member to DataRef and RelValue
(3) Subscriptions compute DataKey only when not present on row
Shubham Mishra