Commit Graph

12 Commits

Author SHA1 Message Date
Noa 483a9488e2 Update rand (#2568) 2025-04-11 17:39:41 +00:00
Mario Montoya 3fd78203c4 Compress the snapshot (#2034) 2025-04-11 15:18:17 +00:00
Kim Altintop 85c347cba3 snapshot: Remote synchronization (#2559) 2025-04-08 07:57:52 +00:00
Noa a5212a5f75 Commitlog compression (#2504) 2025-03-31 22:00:52 +00:00
Mario Montoya f9f38543c8 Add readmes to all implementation crates specifying that they do no offer stable interfaces (#2320) 2025-03-06 19:50:17 +00:00
Noa f136670420 Directory structure impl (#1879)
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
2024-11-12 04:24:43 +00:00
Noa 66112bbdf0 Impl subscribe subcommand & subscription smoketests (#1343)
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
2024-06-14 09:16:05 +00:00
Zeke Foppa 8f3f6bd9d7 Fix Config.save failing if /tmp is on a different filesystem (#1346)
* [bfops/fix-config-saving]:  do thing

* [bfops/fix-config-saving]: review

* [bfops/fix-config-saving]: fix smoketests

* [bfops/fix-config-saving]: use create_new to avoid race condition

---------

Co-authored-by: Zeke Foppa <github.com/bfops>
2024-06-06 16:28:17 +00:00
Zeke Foppa b06b2e59f1 Fix bug with Lockfile sticking around (#1341)
* [bfops/fix-config-lock]: do thing

* [bfops/fix-config-lock]: review

* [bfops/fix-config-lock]: review

* [bfops/fix-config-lock]: fix

* [bfops/fix-config-lock]: TODOs

* [bfops/fix-config-lock]: review

* [bfops/fix-config-lock]: review

* [bfops/fix-config-lock]: review

* [bfops/fix-config-lock]: review

---------

Co-authored-by: Zeke Foppa <github.com/bfops>
2024-06-05 22:40:24 +00:00
Phoebe Goldman 8c5f40db8d Add the snapshot crate, which implements snapshotting at a low level (#1340)
* Add the `snapshot` crate, which implements snapshotting at a low level

- Requires making `BlobHash` be `Serialize` and `Deserialize`.
  For arcane macro-ology reasons, this requires writing `BlobHash::SIZE`
  instead of `Self::SIZE` (it gets embedded in a visitor struct or something).
- Requires adding two new operators to `BlobStore`.
- Adds a return value to `Page::save_content_hash`, for convenience.
- Impls `DerefMut` for `Pages`.
- **Scary change:** adds `Table::pages_mut`.
  I think possibly this operator should be `unsafe`,
  since write access to the `Pages` allows an undisciplined caller
  to violate the `Table`'s assumptions by corrupting a `Page`.
  It seems like an anti-pattern to mark a method `unsafe` on the grounds that
  misusing its return value can cause UB,
  but I don't see a plausible alternative
  without making most methods on `Page` unsafe.
  Open to feedback on this one!

* Nix `Table::pages_mut`

* Address Mazdak's feedback

* Use `thiserror` rather than `anyhow` for better error hygiene
2024-06-05 21:58:12 +00:00
Phoebe Goldman f9cc84e3b4 Define DirTrie, a git-like on-disk object store (#1336)
* Define `DirTrie`, a git-like on-disk object store

* Remove unused iteration code; add simple tests

* Address Mazdak's review
2024-06-05 17:16:59 +00:00
Phoebe Goldman db34ff6a8e Create new crate fs-utils; move Lockfile and create_parent_dir (#1334)
* Create new crate `fs-utils`; move `Lockfile` and `create_parent_dir`

The snapshot crate will need to create lockfiles.
Rather than duplicating code to do so, we choose to move our definition of `Lockfile`
into a crate that can be depended on by both `cli` and `snapshot`.
No existing crate seems like an obvious choice for this
-- a `Lockfile` is not really a data structure, so `data-structures` seems wrong --
so we add a new crate, `fs-utils`.
Currently this contains only `Lockfile` and `create_parent_dir`,
but a follow-up PR will add `DirTrie`, a Git-like on-disk object store.

* Deduplicate `map_err` closure

* Zeke's nit: simplify control flow

Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>

---------

Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
2024-06-04 16:57:28 +00:00