Commit Graph

3186 Commits

Author SHA1 Message Date
Noa 06affca012 Fix deadlock (#854)
* Fix deadlock

* Drop lock before send_message
2024-02-16 19:15:04 +00:00
Phoebe Goldman 3a31c33e81 Add delete_by_{nonunique_column} method to #[spacetimedb(table)] (#859)
- Refactor `bindings::query::delete_by_field` to require `FilterableValue`
  and return `u32` count of deleted rows.
- Add `bindings::query::delete_by_unique_field`
  with the previous behavior of `delete_by_field`,
  i.e. requiring `UniqueValue` and returning `bool` with `true = deleted`.
  This is implemented in terms of `delete_by_field`.
- Macro-generated unique filter funcs call `delete_by_unique_field`
  rather than `delete_by_field`.
- The table macro now generates non-unique delete functions
  alongside the existing non-unique filter functions
  for non-unique fields with primitive types.
- Add uses of `::delete_by_{field}` for both unique and non-unique columns
  to the `rust-wasm-test` module.

Note that this PR does not include any host-side changes;
the diff is localized to the `bindings` and `bindings-macro` crates.
2024-02-16 18:42:05 +00:00
Nathaniel Richards 41d679ba3c Merge pull request #21 from jkbz64/fix/protobuf-dev
Remove `protobufjs` from devDependencies
2024-02-16 13:41:19 -05:00
John Detter 4c63b8280b Fix issues caused by disabling metrics (#856)
Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-16 15:28:26 +00:00
Piotr Sarnacki f9a36228e5 Use a version label for tag builds (#751)
When building a release for a tag we have to build a multi arch image
ourselves. So far we tagged it with a tag `<short-sha>-full`. This is
nice to keep the history of all the tagged commits, but it's not very
usable as usually we just want the latest build for a given tag. This
commit introduces tagging the build with a git tag
2024-02-16 08:35:26 +00:00
Kim Altintop 09a495f792 core: More reliable detection of incompatible schema updates (#734)
Compare `TableSchema` instead of `TableDef`, which should reject all
modifications to a table. Note that the values require conversion for
both the module and database schema to be comparable.

With this change, updating a module which only adds tables or modifies
reducers will be possible, but any change (including indexes,
constraints, etc) to an existing table will fail the update.
2024-02-16 08:03:41 +00:00
John Detter ec3423986f Version upgrade v0.8.2-beta (#858)
Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-16 00:32:12 +00:00
John Detter 04d4771a21 Fix identity subcommand ordering (#857)
Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-16 00:21:31 +00:00
John Detter 79fd5ad75e Fix publish issues 0.8.2 (#853)
* Fix some issues related to publishing 0.8.2

* Added phoebe's fix

* Reverting file

---------

Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-15 23:29:37 +00:00
joshua-spacetime a16bbf86fa fix: Record durations for aborted and read transactions (#850) 2024-02-15 18:15:05 +00:00
joshua-spacetime 29f3c89766 fix: Wrong algebraic type in query benchmarks (#849) 2024-02-15 16:44:17 +00:00
Noa 92e5a62ae1 Reduce allocations for prometheus with_label_values (#810) 2024-02-14 20:17:05 +00:00
Phoebe Goldman 5f02afcb54 Core: depend on "serde" from lib (#845)
I seem to be unable to build `core` directly on master,
though building the workspace root, i.e. `cli`, works.
This is caused by `core` depending on `spacetimedb-lib.workspace`,
which has `default-features = false`, excluding serde support.

This commit correctly specifies the `"serde"` feature for core's dependency on lib,
so it is now possible to `cd crates/core && cargo check`.
2024-02-14 20:14:55 +00:00
joshua-spacetime 6c1c1c4b84 test(835): Benchmark full subscription eval for table scans and join (#839)
* Adding game loading benchmarks

* Change the dat load to use 1_000_000 for both tables

* Move subscription benchmarks alongside incremental ones

* Update README

---------

Co-authored-by: Mario Alejandro Montoya Cortés <mamcx@elmalabarista.com>
2024-02-14 17:34:47 +00:00
Piotr Sarnacki 531ca340f4 Add workflow to tag latest docker image on release (#756) 2024-02-14 17:02:40 +00:00
joshua-spacetime d3dc6ead16 test: Deduplicate utilities for creating tables (#842) 2024-02-14 16:44:25 +00:00
Zeke Foppa 98fa0867f3 Fix: Publish fails if rustup not present (#814)
* empty

* remove

* fixed?

* clarify

* [bfops/fix-799-rustup-issue]: fix log

* [bfops/fix-799-rustup-issue]: restore

* [bfops/fix-799-rustup-issue]: cargo fmt

* [bfops/fix-799-rustup-issue]: nit
2024-02-13 20:00:43 +00:00
Zeke Foppa 86e816d1dc [bfops/remove-unused-file]: remove unused file (#841) 2024-02-13 19:52:05 +00:00
Shubham Mishra 4540bad1a1 To make eval subscription concurrent (#807)
* Added Rwlock over subscription list

* removed lock from subscription manager and .await from add_subscription
2024-02-13 18:39:05 +00:00
John Detter 8adad4c4da Verify that the upgrade-version tool still works for every PR (#825)
* Update upgrade-version tool

* Verify that version upgrade still works for every PR

* Add git diff to the end of CI

---------

Co-authored-by: Noa <coolreader18@gmail.com>
Co-authored-by: John Detter <no-reply@boppygames.gg>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
2024-02-13 17:29:27 +00:00
joshua-spacetime 0260b09710 test(829): Benchmark incremental update for table subscription (#830)
Closes #829.
2024-02-13 16:23:40 +00:00
joshua-spacetime def2cae94b chore(836): Remove tracing instrumentation from database iterators (#837)
Closes #836.

The tracing library does not fully remove all instrumentation at compile time.
And since tracing is not zero cost,
it must be removed from the hot path of query execution.
2024-02-13 16:21:47 +00:00
Noa a32ef34556 Update upgrade-version tool (#826)
* Update upgrade-version tool

* Small cludge

---------

Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-13 15:16:09 +00:00
Phoebe Goldman d4bfb9af4b ScanIterByColRange: Avoid unnecessary ProductValue allocations (#804)
Prior to this commit, `ScanIterByColRange::next` called `RowRef::to_product_value`
and then `ProductValue::project_not_empty` on every row in the table
to get a key, which it compared to the sought range.
This always allocated at least a `ProductValue`,
even when seeking a range of primitive type like `u64`.
It also unnecessarily deserialized (and potentially allocated)
columns not relevant to the search for rows which did not match the range.

With this commit, we call `RowRef::project_not_empty` directly.
When the sought range is a single column, this avoids allocating a `ProductValue`.
Even when seeking a multi-column range, this avoids deserializing unrelated columns
of non-matching rows.

Non-indexed `IterByColEq` is always going to be slow, so this probably doesn't matter,
but it's a trivial change.
2024-02-13 07:56:33 +00:00
Tyler Cloutier faf8766b72 This fixes replaying of the transaction log to no longer check constraints (#806)
* This fixes replaying of the transaction log to no longer check constraints

* Fixed based on Phoebe and Kim's comments

* Cargo fmt

* Cargo fmt

* This fixes index updating for replaying deletes

---------

Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
2024-02-12 21:42:29 +00:00
Noa 295ae3dd82 Fix benchmarks (#819) 2024-02-12 20:45:55 +00:00
joshua-spacetime 45ae7f7cd7 chore(datastore): Remove debug instrumentation (#812) 2024-02-12 19:57:56 +00:00
Noa 7491835e99 ResultInspectExt is obsolete (#818) 2024-02-12 18:28:42 +00:00
joshua-spacetime 2de5bf0d18 perf(subscriptions): Add metric counter for subscribe calls (#817) 2024-02-12 18:13:36 +00:00
joshua-spacetime 4b41998f96 test: Add performance benchmark for incremental join query (#795) 2024-02-10 01:49:15 +00:00
Noa b4f75d272f Bump Rust to 1.76.0 (#811)
* Bump Rust to 1.76.0

* Fix new clippy lints

* Use rust-toolchain in smoketests
2024-02-09 19:40:03 +00:00
Noa 6c292152aa Move legacy control_db from core to standalone (#732)
* Move legacy control_db to standalone

* Remove unused ArcEnv
2024-02-08 21:56:08 +00:00
joshua-spacetime 8d5561640b perf(query): Moves Code::Block instead of cloning during eval (#805)
Closes #809.

For a table scan of 1M rows, this clone was over 30% of the total execution time.
2024-02-08 17:00:19 +00:00
Noa 4c5c8259ba Add a RwLock around module subscriptions to prevent a race condition (#792)
* Add a RwLock around module subscriptions to prevent a race condition

* lint

---------

Co-authored-by: Shubham Mishra <shubham@clockworklabs.io>
2024-02-08 08:47:12 +00:00
Noa a7a707c0f2 Fix build errors in spacetimedb_table benches (#787) 2024-02-07 21:20:10 +00:00
Noa 0653e17201 Simplify broadcast_commit_event (#791) 2024-02-07 21:16:50 +00:00
Noa 37658aae7e Add rust-version to Cargo.toml (#802)
* Add rust-version to Cargo.toml

* Use workspace inheritance to make bumping the spacetime version easier
2024-02-07 20:02:05 +00:00
james gilles fa93200397 1_000_000 insert + update benchmarks (#764)
* Add 1_000_000 insert benches, disabled by default.

* memory architecture fix
2024-02-07 18:24:08 +00:00
james gilles b7c8a089d4 Remove separate callgrind please script (#800) 2024-02-07 16:58:42 +00:00
Phoebe Goldman c6ae09676e BTreeIndex: use ReadColumn instead of ProductValue fields (#796)
* BTreeIndex: Specialize on primitive key types for great performance

Rewrite of Tyler's #771, because I thought this would be easier than rebasing.

This commit hoists branching on key types outside of comparisons and searches
in `BTreeIndex`,
so that in many cases we can use e.g. `u64::cmp`
instead of the much slower `AlgebraicValue::cmp`.

This design is kind of ugly, and will likely change in the future,
but for now it's good enough, and is a meaningful performance improvement.

* BTreeIndex: use `ReadColumn` instead of `ProductValue` fields

Prior to this commit, `BTreeIndex::insert` and `BTreeIndex::delete`
took a `&ProductValue`, the row to be inserted or deleted, as an argument,
and extracted the indexed column value(s) from it.

With this commit, we instead take a `RowRef`
referring to the row to be inserted or deleted,
and access the indexed column value(s) using `ReadColumn`.
For specialized indexes, we extract them directly as native types,
rather than as `AlgebraicValue`.

Making this change involved a nasty split borrow problem,
as we now need to have a `RowRef` to the inserted/deleted row
coexisting with a mutable borrow of the index to be modified.
This required introducing a `TableInner` struct,
which is the referent of `RowRef`.

* Lints: superfluous borrows
2024-02-07 15:55:45 +00:00
Tyler Cloutier b622c28802 Make datastore public for use in the cutlery crate 2024-02-06 18:27:54 -08:00
Zeke Foppa bda431cd12 fix (#798) 2024-02-07 00:16:15 +00:00
John Detter 061f1c3fbf Added new callback - onBeforeSubscriptionApplied (#71)
* onBeforeSubscriptionApplied

* Small bug fix

---------

Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-06 14:09:54 -06:00
John Detter b1705390f3 Fixed an issue where only one network packet was being processed per (#70)
frame

Co-authored-by: John Detter <no-reply@boppygames.gg>
2024-02-06 12:50:05 -06:00
james gilles a13c1363a9 Unify benchmark runs (to some extent) (#742)
* Unify benchmark runs (to some extent)

* Another yml fix
2024-02-06 18:27:23 +00:00
Phoebe Goldman b8cecb3caf BTreeIndex: Specialize on primitive key types for great performance (#793)
Rewrite of Tyler's #771, because I thought this would be easier than rebasing.

This commit hoists branching on key types outside of comparisons and searches
in `BTreeIndex`,
so that in many cases we can use e.g. `u64::cmp`
instead of the much slower `AlgebraicValue::cmp`.

This design is kind of ugly, and will likely change in the future,
but for now it's good enough, and is a meaningful performance improvement.
2024-02-06 17:57:45 +00:00
joshua-spacetime 9a7e2e2e3f feat(query): Add a utility for checking table refs (#790)
Prerequisite for #747.

Adds a boolean utility method that probes a QueryExpr for table references.
2024-02-06 16:04:20 +00:00
Phoebe Goldman cf6aa6dde0 trait ReadColumn, to read a single column from a RowRef (#789)
This commit defines `trait ReadColumn`,
which is implemented for types that can be stored in a table column
and can be read out of said table column.

Its interesting method is
`read_column(row: RowRef<'_>, idx: usize) -> Result<Self, TypeError>`,
which attempts to read the `idx`th column of `row` as a value of type `Self`,
returning a `TypeError` if the row in question does not have the appropriate types.

`ReadColumn` is implemented for Rust equivalents of all non-compound `AlgebraicType`s,
i.e. integers, floats, `bool` and `String`.
It is also implemented for all SATS value types,
including those which represent values of compound types,
i.e. `AlgebraicValue`, `ProductValue`, `SumValue`, `ArrayValue` and `MapValue`.
2024-02-05 20:45:51 +00:00
Zeke Foppa 510c9aba5a pt1 (#785) 2024-02-05 15:45:14 +00:00
joshua-spacetime befe8b7b3f feat(query): Make current plan type hashable (#782)
A prerequisite for #747.
2024-02-02 17:46:18 +00:00