* 1. Split ColumnOp into ColumnOp & FieldOp, former storing ColId
2. Shrink SqlAst to 80 bytes, so it can be passed in registers
3. Store end-result Header in IndexSemiJoin
4. Remove operational use of Header in ColumnOp & build_query
5. Simplify RowRef::{get, project, project_owned}
* Make parts of build_query actually infallible.
1. Make IndexSemiJoin::filter infallible.
2. Make ColumnOp::compare and friends infallible.
3. Make RowRef::{get, project, project_owned} infallible.
* Make RelOps::next itself infallible
* 1. with_select{_cmp}: ensure type safety o query exec cannot panic
2. Document RelValue::{get, read_or_take_column, project_owned}
3. Refactor optimize_select
4. Ensure in optimize_select that conditions are merged with preceding selects
* remove RelOps::{head, row_count}; head is redundant & row_count is useless
* remove Relation trait; it does not carry its weight
* make build_query infallible
* simplify IndexSemiJoin, make it slightly less branchy
* simplify try_index_join
* split IndexSemiJoin into Left & Right parts
* move get_field_pos to test code
* move test version of build_query to test code
* Always send TransactionUpdate to the sender
This commit changes transaction update broadcast to always send an
update to the sender, even if the sender is not subscribed to any
data that was committed. In such case the transaction update is sent
with an empty database update.
* Remove debug printlns
* Update code after rebase
* Fix test
* Update crates/core/src/subscription/module_subscription_manager.rs
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: Piotr Sarnacki <drogus@gmail.com>
* Extract sending message with error handing in module_subscription_manager
* Bring back a new line
* SDK test that the client is notified of a reducer it called
* cleanup subscription code a bit
* Fix merge problem
* Lint + fmt
* Fix test
---------
Signed-off-by: Piotr Sarnacki <drogus@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
* Add SDK test for `SELECT * FROM *`
Which doesn't pass, because we broke it.
* Fix select * from *
---------
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
- Arcing `TableSchema`, and this has benefits elsewhere too.
- Arc<[_]>ing the visitor program instructions.
The data behind the Arcs very rarely change,
which is the perfect case for an Arc.
`TxData` is the representation of a transaction after it was committed,
and is passed around for evaluation of subscription queries and sending
the result to clients.
With the new commitlog, it can be represented more compactly, such that
copies for writing to the log can be avoided.
Note that this patch stops short of refactoring `DatabaseUpdate`, which
is another representation of the same information as sent to clients.
This means that `ProductValue`s need to be cloned from `TxData`, just as
before.
This patch attempts to integrate the new commitlog with the minimum
changes.
Most of the diff comes from deletions of the legacy log and the need to
adjust tests due to the requirement for a tokio runtime when a durable
database is used in tests.
The "meat" of the patch are the `RelationalDB` constructors,
`RelationalDB::commit_tx`, and the replay logic in
`locking_tx_datastore`.
While `DataKey` is gone, there is still some redundant data being passed
around, which will be addressed in the follow-up patch.
* Commit iter metrics
* test: Make full-join benchmark async
* use Vec instead of Hash
* Revert "test: Make full-join benchmark async"
This reverts commit eed5aa8a55.
* rebase on master
* lint
* refector api
* remove lifetime from ctx (#1017)
* move ctx to higher level
* fix boilterplate
* batch more
---------
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Closes#1024.
Before this change,
we would serialize messages **before** inserting into the send queue.
Because we commit the tx only after inserting into the send queue,
this meant we were holding onto the database lock unnecessarily.
After this change,
we serialize messages **after** inserting into the send queue.
This means we serialize only after committing the tx.
* Implement (but do not use) a fast path for BFLATN -> BSATN conversion
* fmt and clippy
* `u16` offset rather than `usize`
* Address Joshua's review
* Define methods on `RowRef` and `RelValue` which use the new serializer
* Comment in `align_to` about div-by-zero
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
* Add benchmark comparing BFLATN -> BSATN with and without the fast path
* Add benchmark on `u64_u64_u32`, which has less interior padding than `u32_u64_u64`
* Remove `to_len` from `to_bsatn_extend`
It turns out to be slower than just eating the `realloc`s.
* Remove unused `to_bsatn_slice`
I thought I would need it, but it ended up not being useful.
* Expand comment with example; `Box<[...]>` to reduce memory footprint
* Comments from Mazdak's review
---------
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
* eval_updates: use map entry apis
* dedup logic in remove_subscription + use entry api to hash only once
* stop cloning PVs in eval_updates
* address Phoebe's comments
* add tracing for perf testing
---------
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
Mazdak Farrokhzad