# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/2824.
Defines a global pool `BsatnRowListBuilderPool` which reclaims the
buffers of a `ServerMessage<BsatnFormat>` and which is then used when
building new `ServerMessage<BsatnFormat>`s.
Notes:
1. The new pool `BsatnRowListBuilderPool` reports the same kind of
metrics to prometheus as `PagePool` does.
2. `BsatnRowListBuilder` now works in terms of `BytesMut`.
3. The trait method `fn to_bsatn_extend` is redefined to be capable of
dealing with `BytesMut` as well as `Vec<u8>`.
4. A trait `ConsumeEachBuffer` is defined from
`ServerMessage<BsatnFormat>` and down to extract buffers.
`<ServerMessage<_> as ConsumeEachBuffer>::consume_each_buffer(...)` is
then called in `messages::serialize(...)` just after bsatn-encoding the
entire message and before any compression is done. This is the place
where the pool reclaims buffers.
# Benchmarks
Benchmark numbers vs. master using `cargo bench --bench subscription --
--baseline subs` on i7-7700K, 64GB RAM:
```
footprint-scan time: [21.607 ms 21.873 ms 22.187 ms]
change: [-62.090% -61.438% -60.787%] (p = 0.00 < 0.05)
Performance has improved.
full-scan time: [22.185 ms 22.245 ms 22.324 ms]
change: [-36.884% -36.497% -36.166%] (p = 0.00 < 0.05)
Performance has improved.
```
The improvements in `footprint-scan` are mostly thanks to
https://github.com/clockworklabs/SpacetimeDB/pull/2918, but 7 ms of the
improvements here are thanks to the pool. The improvements to
`full-scan` should be only thanks to the pool.
# API and ABI breaking changes
None
# Expected complexity level and risk
2?
# Testing
- Tests for `Pool<T>` also apply to `BsatnRowListBuilderPool`.
# Description of Changes
View tables have private metadata columns that need to be dropped before
sending results to clients. Before this patch we dropped these columns
for sql queries and initial subscriptions, but we didn't drop them after
incremental update which is what this patch does.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Smoketest
Permissions for evaluating SQL/DML are not generally "actions", but more
a set of permissions that are checked during evaluation.
To make this work with the teams feature, this patch extends `AuthCtx`
to allow checking a set of permissions as mandated by the spec. This set
is a bit more fine-grained than "is owner", so as to avoid baking in the
concept of teams/collaborators, or assumptions about what a role might
entail. Both are likely to evolve in the future, so evaluation of
permissions / capabilities should be confined to the impl of the
`Authorization` trait.
Unlike "actions", the `AuthCtx` must be able to evaluate permission
checks quickly and without side-effects, nor can it enter an `async`
context. In that sense, it is precomputed (if you will), and stored as a
closure in the `AuthCtx` for external authorization.
A challenge posed is how to thread through the constructed `AuthCtx` for
subscriptions.
A tempting approach would have been to equip the `HostController` with
the ability to summon an `AuthCtx`. That, however, would have created a
gnarly circular dependency, because the `HostController` also controls
the controldb, which itself demands an `AuthCtx`.
Instead, the `AuthCtx` is obtained in the endpoint handler and passed to
each method call that requires one. That's less pretty, but more
effective.
---------
Signed-off-by: Kim Altintop <kim@eagain.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes
We recently merged several repos together. This PR clarifies the license
terms for several subdirectories, as well as the relationship between
the licenses.
The licenses in our subdirectories have become symbolic links to
licenses in our toplevel `licenses` directory. For any particular
subdirectory's license file in the diff, you can click `... -> View
file` and then click on the text that says "Symbolic Link" on that page.
This will take you to the license file that it links to.
I have also updated the `tools/upgrade-version` script to update the
change date in the new `licenses/BSL.txt` file.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None. Only changes to license files.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
* 1. Split ColumnOp into ColumnOp & FieldOp, former storing ColId
2. Shrink SqlAst to 80 bytes, so it can be passed in registers
3. Store end-result Header in IndexSemiJoin
4. Remove operational use of Header in ColumnOp & build_query
5. Simplify RowRef::{get, project, project_owned}
* Make parts of build_query actually infallible.
1. Make IndexSemiJoin::filter infallible.
2. Make ColumnOp::compare and friends infallible.
3. Make RowRef::{get, project, project_owned} infallible.
* Make RelOps::next itself infallible
* 1. with_select{_cmp}: ensure type safety o query exec cannot panic
2. Document RelValue::{get, read_or_take_column, project_owned}
3. Refactor optimize_select
4. Ensure in optimize_select that conditions are merged with preceding selects
* remove RelOps::{head, row_count}; head is redundant & row_count is useless
* remove Relation trait; it does not carry its weight
* make build_query infallible
* simplify IndexSemiJoin, make it slightly less branchy
* simplify try_index_join
* split IndexSemiJoin into Left & Right parts
* move get_field_pos to test code
* move test version of build_query to test code
* make user tables private by default and define privacy via attribute
* switch to spacetimedb(table(public)) syntax
* accept codegen snap changes
* sdk: use public in define_tables!
* bindings-macro: adjust some doc comments
* sdk-test-connect-disconnect: make Connected/Disconnected public tables
* Make Public Private again
* remove dead code wrt. Table
* simplify DbProgram: Insert/Update/Delete only use DbTables
* consistency: move CrudExpr::Update logic to own function
* use TxMode::unwrap_mut more
* Detect unsatisfiable range queries; warn and short-circuit.
This commit fixes a panic caused by unsatisfiable range bounds on an index query,
e.g. `WHERE x < 5 AND x > 5`.
These unsatisfiable bounds made Rust's `BTreeMap` angry
(See https://doc.rust-lang.org/src/alloc/collections/btree/search.rs.html#106-124),
and panicked.
They also represent probable bugs,
as it's silly to write a query which statically will return no rows.
With this commit, we detect statically unsatisfiable bounds in two cases:
- When compiling queries, we log a message at `WARN` containing the offending query.
- When evaluating queries, we silently construct an `EmptyRelOps`
rather than a real query iterator.
This commit also adds a test that the offending queries can be compiled and executed
without panicking, and select no rows.
* Per Joshua's review, add comments that this is a suboptimal solution
* Fix typo
---------
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Mazdak Farrokhzad