mirror of
https://github.com/clockworklabs/SpacetimeDB.git
synced 2026-05-09 00:59:46 -04:00
Ryan
6fea15f745
This is the implementation of a fix for #4093 # Description of Changes * Updated `RawTableIterBase.Enumerator` to rent its scratch buffer from `ArrayPool<byte>.Shared` (with dynamic re-rent on `BUFFER_TOO_SMALL`) and return it on dispose, so iterating rows no longer allocates a fresh `byte[]` per step. * The enumerator still exposes the row bytes via `ArraySegment<byte> Current`, but now the underlying storage is reused across iterations rather than recreated each time. Testing results from `master`: ``` allocated_bytes=14000 elapsed_ticks=1521829 sum=1249975000 row_count=50000 Find() TinyRecord (8 bytes payload) -> 132208 bytes allocated Find() MediumRecord (~50 bytes payload) -> 132440 bytes allocated Find() LargeRecord (1 KB payload) -> 134528 bytes allocated Find() LargeRecord (10 KB payload) -> 152408 bytes allocated Find() LargeRecord (100 KB payload) -> 336728 bytes allocated Iter() 10 rows -> 131896 bytes allocated Filter() iterate 20 rows -> 133288 bytes allocated Filter() iterate 100 rows -> 135976 bytes allocated 10x consecutive Find() (TinyRecord) -> 1319120 bytes allocated ``` Testing results with this fix: ``` allocated_bytes=14000 elapsed_ticks=1504949 sum=1249975000 row_count=50000 Find() TinyRecord (8 bytes payload) -> 1096 bytes allocated Find() MediumRecord (~50 bytes payload) -> 1280 bytes allocated Find() LargeRecord (1 KB payload) -> 4464 bytes allocated Find() LargeRecord (10 KB payload) -> 27464 bytes allocated Find() LargeRecord (100 KB payload) -> 234312 bytes allocated Iter() 10 rows -> 680 bytes allocated Filter() iterate 20 rows -> 1872 bytes allocated Filter() iterate 100 rows -> 3280 bytes allocated 10x consecutive Find() (TinyRecord) -> 8000 bytes allocated ``` # API and ABI breaking changes No API or ABI changes # Expected complexity level and risk 2 - low/moderate: Touches the hot-path iterator that every `Iter`/`Find`/`Filter` call uses # Testing - [X] Compiled CLI and ran regression tests locally - [X] Verified iterator-based harness runs (client + module reducers) on both `master` and this branch, confirming allocations drop from ~131 KB per read to payload-scaled values. - [X] Ensured no regressions in standard harness sanity checks (`row_count=50000`, `sum=1249975000`).