# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/2824.
Defines a global pool `BsatnRowListBuilderPool` which reclaims the
buffers of a `ServerMessage<BsatnFormat>` and which is then used when
building new `ServerMessage<BsatnFormat>`s.
Notes:
1. The new pool `BsatnRowListBuilderPool` reports the same kind of
metrics to prometheus as `PagePool` does.
2. `BsatnRowListBuilder` now works in terms of `BytesMut`.
3. The trait method `fn to_bsatn_extend` is redefined to be capable of
dealing with `BytesMut` as well as `Vec<u8>`.
4. A trait `ConsumeEachBuffer` is defined from
`ServerMessage<BsatnFormat>` and down to extract buffers.
`<ServerMessage<_> as ConsumeEachBuffer>::consume_each_buffer(...)` is
then called in `messages::serialize(...)` just after bsatn-encoding the
entire message and before any compression is done. This is the place
where the pool reclaims buffers.
# Benchmarks
Benchmark numbers vs. master using `cargo bench --bench subscription --
--baseline subs` on i7-7700K, 64GB RAM:
```
footprint-scan time: [21.607 ms 21.873 ms 22.187 ms]
change: [-62.090% -61.438% -60.787%] (p = 0.00 < 0.05)
Performance has improved.
full-scan time: [22.185 ms 22.245 ms 22.324 ms]
change: [-36.884% -36.497% -36.166%] (p = 0.00 < 0.05)
Performance has improved.
```
The improvements in `footprint-scan` are mostly thanks to
https://github.com/clockworklabs/SpacetimeDB/pull/2918, but 7 ms of the
improvements here are thanks to the pool. The improvements to
`full-scan` should be only thanks to the pool.
# API and ABI breaking changes
None
# Expected complexity level and risk
2?
# Testing
- Tests for `Pool<T>` also apply to `BsatnRowListBuilderPool`.
# Description of Changes
Provides new WASM ABIs:
- `datastore_index_scan_point_bsatn`
- `datastore_delete_by_index_scan_point_bsatn`
These are then used where applicable to speed up `.find(_)` and friends.
Point scans are also used more internally where applicable.
What remains after this is use in C# module bindings and to expose this
in TS as well.
The PR makes TPS go from roughly 36k to 38k TPS on my machine and also
makes a difference in flamegraphs where the time spent in some index
scans are substantially decreased.
# API and ABI breaking changes
None
# Expected complexity level and risk
3? This touches the datastore an how we expose it to modules.
# Testing
Some existing tests now exercise the new ABIs by changing what
`.find(_)` and friends do.
---------
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
View tables have private metadata columns that need to be dropped before
sending results to clients. Before this patch we dropped these columns
for sql queries and initial subscriptions, but we didn't drop them after
incremental update which is what this patch does.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Smoketest
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
The query engine can now execute queries using mutable transactions
because this patch implements the `Datastore` trait for `MutTxId`.
Note this is both a refactor and a feature patch. It is a refactor
because the `Datastore` trait was updated to allow for mutable
transactions.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1.5
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Current tests pass after refactor
- [ ] Use `MutTxId` for query execution
# Description of Changes
Necessary for pulling in rolldown.
# API and ABI breaking changes
None
# Expected complexity level and risk
1, with the caveat that this updates the Rust version and therefore
touches all the code.
# Testing
- [ ] Just the automated testing
# Description of Changes
Implements [subscribing to durable
commits](https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1594).
The setting works on a per-connection level, and essentially just delays
sending transaction updates until the transaction is reported as durable
by the database.
For connectionless SQL operations, the setting works per-request. No SQL
syntax is provided by this patch to toggle the configuration.
After some deliberation, I opted to obtain the offset when a transaction
commits (as opposed to when it starts). This creates some mild
inconvenience, because we prevent the transaction from committing until
the corresponding subscription updates are enqueued.
The strategy is, however, more correct should we ever support weaker
isolation levels, and it is easier to document.
Follow-ups include:
- Provide SQL syntax (`SET synchronous_commit = ON` or something)
- C# and TypeScript SDKs
- Reference docs?
# API and ABI breaking changes
Not breaking, but adds a parameter to the subscribe and sql endpoints.
# Expected complexity level and risk
4
To the author's understanding, ordering of outbound messages is not
changed by this patch, even if there are messages that don't have a
transaction offset (such as error messages). I.e. while waiting for the
transaction offset of a message to become durable, no message enqueued
after that message will be delivered. This may not be desirable in some
cases.
The patch may contain concurrency bugs, e.g. awaiting futures that may
never resolve.
# Testing
- [x] Implemented a new test in the `module_subscription_actor` module
- [x] Added unit tests for the core logic in `ClientConnectionReceiver`
It would be desirable to also have integration-level tests, but I'm
currently unsure how to write those without being able to control if and
when the database reports an offset as durable.
---------
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Mazdak Farrokhzad