Commit Graph

94 Commits

Author SHA1 Message Date
Mario Montoya f9f38543c8 Add readmes to all implementation crates specifying that they do no offer stable interfaces (#2320) 2025-03-06 19:50:17 +00:00
Phoebe Goldman aedc601145 Rename Address to ConnectionId (#2220)
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: James Gilles <jameshgilles@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
2025-02-10 00:40:16 +00:00
Mazdak Farrokhzad b50b6841f1 Add direct indices to datastore (#2221) 2025-02-07 01:06:38 +00:00
Mazdak Farrokhzad 25d74091c3 Make MutTxId::update really idempotent wrt. TxData (#2212) 2025-02-05 20:40:02 +00:00
Phoebe Goldman d839657008 Data size SKU for billing (#2098) 2025-02-03 14:15:30 +00:00
Mazdak Farrokhzad 6f20597600 Make check_and_insert & find_old_row and friends do less checks (#2192) 2025-01-31 15:00:19 +00:00
Mazdak Farrokhzad 6630a90ee1 Switch DeleteTable impl to one based on FixedBitSet (#2183) 2025-01-29 17:15:17 +00:00
Noa 293aebaef9 Bump to Rust 1.84 (#2001) 2025-01-28 23:11:29 +00:00
Mazdak Farrokhzad 5b5a0a6515 Cache whether a Table is a scheduler table, avoiding fetching the schema (#2141) 2025-01-28 19:57:15 +00:00
Mazdak Farrokhzad afdc0d67dd Add the update ABI (#2137)
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2025-01-27 19:38:18 +00:00
Phoebe Goldman d171b44a89 Don't create indexes during bootstrapping; wait until after replay (#2161) 2025-01-23 19:41:39 +00:00
Mazdak Farrokhzad a1573af5a1 Fix unique index + MutTxId::insert not un-deleteting (#2156) 2025-01-22 09:51:34 +00:00
Mazdak Farrokhzad 6f428f3afb Make the key of Table.indexes be IndexId (#2124) 2025-01-17 18:42:54 +00:00
joshua-spacetime 3798f468a6 query engine integration (#2074) 2025-01-11 00:01:24 +00:00
Mazdak Farrokhzad f667f65b15 perf(2019): skip PointerMap given a unique index (#2092) 2025-01-09 18:10:20 +00:00
Mazdak Farrokhzad d4f03b74b9 datastore: insert via BSATN instead of via PV (#2069) 2025-01-07 21:36:02 +00:00
Mazdak Farrokhzad e103971e29 Avoid a multimap when the index is unique (#2044) 2024-12-09 22:53:04 +00:00
Mazdak Farrokhzad 96a3871646 move unique constraint checking until after optimistic insertion (#2037) 2024-12-06 12:14:14 +00:00
Mazdak Farrokhzad 03cf2f4b4b Add Static(Bsatn)Layout::deserialize_row_into (#2032) 2024-12-03 16:06:31 +00:00
Mazdak Farrokhzad 586929cf7c Add static_bsatn_validator & validate_bsatn (#2029) 2024-12-03 15:26:08 +00:00
Mazdak Farrokhzad bc71e25ad8 Add Layout.fixed: bool for a fast path in eq_row_in_page & row_type_visitor (#2025) 2024-12-03 15:07:26 +00:00
Phoebe Goldman 8075567da4 Remove problematic debug assertions (#2010) 2024-11-26 03:42:02 +00:00
james gilles c657b4f088 Rip useless names out of RawDef (#1918) 2024-11-20 21:02:00 +00:00
joshua-spacetime c840eda037 Physical query plan + executors (#1881) 2024-10-24 17:30:59 +00:00
Shubham Mishra 0a86b46152 Fix Build: Use default() to init custom hash based datastructures (#1871)
Signed-off-by: Shubham Mishra <shubham@clockworklabs.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2024-10-17 14:41:38 +00:00
Mazdak Farrokhzad 06cc688885 No Maps for you! (#1770) 2024-10-15 22:34:13 +00:00
Noa c4e637e98b Energy metering for persistent memory usage (#766) 2024-10-15 17:58:34 +00:00
Mazdak Farrokhzad 1ac741aa6b Generic and optimized WebSocket API (#1761)
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2024-09-30 21:57:00 +00:00
james gilles c32f297c47 Update TableSchema & system tables to resemble ABI V9 (#1697) 2024-09-25 20:03:22 +00:00
Mazdak Farrokhzad 4f297880ca WASM ABI: add datastore_btree_scan_bsatn & index_id_from_name (#1699)
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
2024-09-16 19:03:17 +00:00
james gilles f3e3d924f7 Add schema::type_for_generate, update validation to use it (#1661)
Signed-off-by: james gilles <jameshgilles@gmail.com>
Co-authored-by: Ingvar Stepanyan <me@rreverser.com>
2024-09-11 15:38:56 -04:00
Mazdak Farrokhzad da71d0f9b1 WASM ABI: insert -> datastore_insert_bsatn & impl new semantics (#1639) 2024-09-05 19:32:26 +00:00
james gilles 2be42156b2 Allow converting new ModuleDef to old TableSchema (#1630) 2024-08-29 18:17:29 +00:00
Mazdak Farrokhzad 3be5c83d99 [WASM ABI 1.0] __call_reducer__ receives Identity & Address by value (#1607)
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Ingvar Stepanyan <me@rreverser.com>
Co-authored-by: Noa <coolreader18@gmail.com>
2024-08-19 22:20:57 +00:00
Mazdak Farrokhzad 1ca9b1a933 [WASM ABI 1.0] Change ColId from u32 to u16 (#1597) 2024-08-19 17:56:28 +00:00
Mazdak Farrokhzad 6a08674ccb Allow empty ColList (#1588) 2024-08-15 16:08:51 +00:00
Mazdak Farrokhzad 1e8e18d74b Add support for I256 and U256 (#1477) 2024-08-08 18:40:35 +00:00
Mazdak Farrokhzad 3340ceea8a SATS: Flatten AlgebraicType, getting rid of BuiltinType (#1559)
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
2024-08-06 18:15:19 +00:00
Kim Altintop 85082077e2 table: Make with_mut_schema clone-on-write (#1530) 2024-07-29 13:45:21 +00:00
james gilles 45b2ceee9a Move schemas to schema crate, rename Def to RawDefV8 (#1498) 2024-07-24 17:38:30 +00:00
james gilles f81f2a7492 Move db module from spacetimedb_sats to spacetimedb_lib (#1479) 2024-07-17 20:59:44 +00:00
Phoebe Goldman 04a7508120 Table::is_row_present: don't panic (#1526) 2024-07-17 18:41:04 +00:00
Phoebe Goldman 6c45e76a98 Integrate snapshotting into core (#1344) 2024-06-11 12:40:02 +00:00
Phoebe Goldman 8c5f40db8d Add the snapshot crate, which implements snapshotting at a low level (#1340)
* Add the `snapshot` crate, which implements snapshotting at a low level

- Requires making `BlobHash` be `Serialize` and `Deserialize`.
  For arcane macro-ology reasons, this requires writing `BlobHash::SIZE`
  instead of `Self::SIZE` (it gets embedded in a visitor struct or something).
- Requires adding two new operators to `BlobStore`.
- Adds a return value to `Page::save_content_hash`, for convenience.
- Impls `DerefMut` for `Pages`.
- **Scary change:** adds `Table::pages_mut`.
  I think possibly this operator should be `unsafe`,
  since write access to the `Pages` allows an undisciplined caller
  to violate the `Table`'s assumptions by corrupting a `Page`.
  It seems like an anti-pattern to mark a method `unsafe` on the grounds that
  misusing its return value can cause UB,
  but I don't see a plausible alternative
  without making most methods on `Page` unsafe.
  Open to feedback on this one!

* Nix `Table::pages_mut`

* Address Mazdak's feedback

* Use `thiserror` rather than `anyhow` for better error hygiene
2024-06-05 21:58:12 +00:00
Phoebe Goldman a214f78f0b Impl Serialize, Deserialize for Page (#1335)
* Impl `Serialize`, `Deserialize` for `Page`

Snapshotting needs to write `Page`s to files and read them back again.
To that effect, this commit implements `Serialize` and `Deserialize` for `Page`.

* Address Mazdak's review

- Fix soundness in `FixedBitSet` by moving an assert.
- Add commentary to test.
- Add commentary to `spacetimedb-lib` dependency.
2024-06-04 15:49:27 +00:00
Shubham Mishra cf4b9aa282 metric for table size (#1319)
* table size metric

* feld blob_store_bytes in table

* address comments

* NumBlobBytes type

* table size metrics: adjust comments, visibility + harden test

---------

Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2024-05-31 17:44:12 +00:00
Mazdak Farrokhzad 25513b37b9 Simplify btree_index module with more idiomatic Rust (#1285)
* simplify btree_index module, more idiomatic Rust

* test
2024-05-23 13:39:48 +00:00
joshua-spacetime 88a8adad70 feat(1231): Basic query cardinality estimation (#1273)
* feat(1231): Basic query cardinality estimation

This patch implements basic cardinality estimation for QueryExpr.
It utilizes table cardinalities and number of distinct values for index related operators.

* estimation tests: dedup + define constants for readability

* row_est: simplify with slice patterns

* fn ndv -> fn num_distinict_values

* simplify TypedIndex::num_keys

* is_range -> is_point (invert) + fuse arms in row_est

* estimation: fix logic for IndexJoin

---------

Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2024-05-23 08:49:07 +00:00
Mazdak Farrokhzad ebc921849e privatize Table::row_layout + related BTreeIndex refactoring (#1262) 2024-05-20 18:44:04 +00:00
Mazdak Farrokhzad 91f7e8c917 add PageHeader::unmodified_hash, a BLAKE3 hash for snapshotting (#1249) 2024-05-20 17:47:42 +00:00