The definition of `FixedHeader` within `Page` previously contained
a `#[cfg(debug_assertions)]` member which was used for some extra assertions.
This was fine originally, but now that we write pages to disk in snapshots,
having the serialized format of the page vary depending on compilation profile
is problematic.
This commit removes said `#[cfg(debug_assertions)]` conditional member from `FixedHeader`,
so that the structure, and thus the on-disk format, of `Page` is consistent
between debug and release builds.
This commit also removes an unrelated but unused `#[cfg(debug_assertions)]` conditional
from the pointer map code. We never called this assertion, even in debug builds.
If we want to re-enable it in the future, we can restore it from git history;
for now, I would prefer not to have unused code committed.
* Add the `snapshot` crate, which implements snapshotting at a low level
- Requires making `BlobHash` be `Serialize` and `Deserialize`.
For arcane macro-ology reasons, this requires writing `BlobHash::SIZE`
instead of `Self::SIZE` (it gets embedded in a visitor struct or something).
- Requires adding two new operators to `BlobStore`.
- Adds a return value to `Page::save_content_hash`, for convenience.
- Impls `DerefMut` for `Pages`.
- **Scary change:** adds `Table::pages_mut`.
I think possibly this operator should be `unsafe`,
since write access to the `Pages` allows an undisciplined caller
to violate the `Table`'s assumptions by corrupting a `Page`.
It seems like an anti-pattern to mark a method `unsafe` on the grounds that
misusing its return value can cause UB,
but I don't see a plausible alternative
without making most methods on `Page` unsafe.
Open to feedback on this one!
* Nix `Table::pages_mut`
* Address Mazdak's feedback
* Use `thiserror` rather than `anyhow` for better error hygiene
* Impl `Serialize`, `Deserialize` for `Page`
Snapshotting needs to write `Page`s to files and read them back again.
To that effect, this commit implements `Serialize` and `Deserialize` for `Page`.
* Address Mazdak's review
- Fix soundness in `FixedBitSet` by moving an assert.
- Add commentary to test.
- Add commentary to `spacetimedb-lib` dependency.
* Make `Page` always fully init
Per discussion on the snapshotting proposal,
this PR changes the type of `Page.row_data` to `[u8; _]`,
where previously it was `[MaybeUninit<u8>; _]`.
This turns out to be shockingly easy,
as our serialization codepaths never write padding bytes into a page.
The only place pages ever became `poison` was the initial allocation;
changing this to `alloc_zeroed` causes the `row_data` to always be valid at `[u8; _]`.
The majority of this diff is replacing `MaybeUninit`-specific operators
with their initialized equivalents,
and updating comments and documentation to reflect the new requirements.
This change also revealed a bug in the benchmarks
introduced when we swapped the order of sum tags and payloads
( https://github.com/clockworklabs/SpacetimeDB/pull/1063 ),
where benchmarks used a hardcoded offset for the tag which had not been updated.
* Update blake3
Blake3 only supports running under Miri as of 1.15.1, the latest version.
Prior versions hard-depended on SIMD intrinsics which Miri doesn't support.
* Address Mazdak's review.
Still pending his agreeing with me that `poison` is a better name than `uninit`.
* "Poison" -> "uninit"
Against my best wishes, for consistency with the broader Rust community's poor choices.
* Remove unnecessary `unsafe` blocks
* More unnecessary `unsafe`; remove forgotten SAFETY comments
2. Make `RowRef::row_hash` use the above.
3. Make `Table::insert` return a `RowRef`.
4. Use less unsafe because of 1-3.
5. Use `second-stack` to reuse temporary allocations in hashing and serialization.
- Arcing `TableSchema`, and this has benefits elsewhere too.
- Arc<[_]>ing the visitor program instructions.
The data behind the Arcs very rarely change,
which is the perfect case for an Arc.
* Swap the location of tags to go before variant data in the BFLATN encoding
* Fix a comment
* Apply suggestions from code review (@gefjon @Centril)
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Signed-off-by: james gilles <jameshgilles@gmail.com>
* Implement memcpy consolidation for sums
* Vanquish clippy
---------
Signed-off-by: james gilles <jameshgilles@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Phoebe Goldman