mirror of
https://github.com/clockworklabs/SpacetimeDB.git
synced 2026-05-13 03:08:40 -04:00
Kim Altintop
ffaf79100c
Report more metrics about snapshot compression, namely: - time to compress a single snapshot (histogram) - for each compression pass: - number of snapshots found to be already compressed (gauge) - number of snapshots compressed (gauge) - cumulative number of objects compressed (gauge) - cumulative number of objects hardlinked (gauge) Those metrics are collected from the `spacetimedb-snapshot` crate without imposing a prometheus dependency on it, i.e. they can be observed by the caller as ordinary Rust types. This is exploited to avoid scanning the entire snapshot repository on each pass -- only the range `(last_compressed + 1)..newest_snapshot` is visited (note that the `compress_snapshots` method now short-circuits on errors). Lastly, the snapshot worker can now be configured to disable compression. This greatly simplifies implementation of alternative post-processing strategies, e.g. involving archival, for which a more coarse-grained compression strategy may be more appropriate. Subcribers are notified of a new snapshot _after_ compression, such that any filesystem locks should be released. # Expected complexity level and risk 2 # Testing May need some, I'm pondering.
⚠️ Internal Crate ⚠️
This crate is intended for internal use only. It is not stable and may change without notice.