# Description of Changes
Reviving a previous patch I wrote during our (internal) TPCC
experimentation. This has become important because, in addition to its
performance implications, it makes row insertion locations deterministic
regardless of datastore restarts, which previously they were not.
Previously, restarting the datastore would re-order the `non_full_pages`
list (i.e. sort it by increasing `PageIndex`, where normally it was not
sorted), meaning that which page a new row would be inserted into
depended on when the datastore was last restarted.
With this patch, that is not the case: the `non_full_pages` are always
kept in a deterministic order, so which page a new row goes into is also
deterministic.
Original commit message follows:
And sort them by number of available var-len granules. This prevents an
accidentally quadratic behavior where, for a table where the average row
contains many var-len granules, after inserting a large number of rows,
there would be a large number of pages in `non_full_pages` each of which
had enough space for at least one fixed-len row part, but insufficient
space for an actual row in practice due to insufficient var-len
granules. Each insertion would then do a linear scan over
`non_full_pages` before either inserting into the last page or
allocating a new page which went to the end.
Now, non-full pages are stored in a `BTreeSet` sorted by the number of
free var-len granules, and the search for a useable page is done with a
`BTreeSet::range` iterator for only the pages with enough granules. I
think there may still be an off-by-one-ish bug here, where a page may
have enough bytes in the gap that it could either store the fixed-len
part or the var-len granules, but not both, but this fix hopefully will
suffice for now.
# API and ABI breaking changes
N/a
# Expected complexity level and risk
2? Table code is a bit fiddly, and this path is performance-sensitive
when inserting rows.
# Testing
- [x] Passes table crate tests.
- [x] Was included in our internal TPCC experimentation, where it
significantly improved performance (due to that benchmark exercising the
accidentally-quadradic behavior this patch is designed to protect).
- [x] Joshua ran the keynote-2 benchmarks with this patch and did not
observe a decrease in throughput.
# Description of Changes
There were mentions of `hashbrown` in the repo that did not go through
`spacetimedb_data_structures::map`.
This caused compile errors on master when running certain tests locally.
These have been replaced with the proper imports.
The PR also bump hashbrown to 0.16.1 and foldhash to 0.2.0.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
Covered by existing tests.
# Description of Changes
Aternative to and closes
https://github.com/clockworklabs/SpacetimeDB/pull/3210.
This version relies on `pending_schema_changes`.
The first commit adds `clear_table` to the datastore that's efficient
and can be exposed to the module ABI in a follow up.
The second commit fixes `drop_table`.
# API and ABI breaking changes
None
# Expected complexity level and risk
3?
# Testing
`test_drop_table_is_transactional` is amended to check `TxData`.
---------
Signed-off-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Shubham Mishra <shubham@clockworklabs.io>
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
We recently merged several repos together. This PR clarifies the license
terms for several subdirectories, as well as the relationship between
the licenses.
The licenses in our subdirectories have become symbolic links to
licenses in our toplevel `licenses` directory. For any particular
subdirectory's license file in the diff, you can click `... -> View
file` and then click on the text that says "Symbolic Link" on that page.
This will take you to the license file that it links to.
I have also updated the `tools/upgrade-version` script to update the
change date in the new `licenses/BSL.txt` file.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None. Only changes to license files.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Phoebe Goldman