# Description of Changes
To resolve#3875, we added exact-match unique index point lookup support
to the C# bindings by introducing and using
`datastore_index_scan_point_bsatn`.
Previously, generated unique index `Find()` was (in at least one
codepath) implemented as:
* A range scan (`datastore_index_scan_range_bsatn`) over a BTree bound,
then
* `SingleOrDefault()` to collapse the results into a single row.
When the scan is empty, `SingleOrDefault()` returns `default(T)`. For
value-type rows this can manifest as a default-initialized row instead
of “missing”, which is what surfaced as “default-ish row” behavior in
views.
Using `datastore_index_scan_point_bsatn` makes the C# implementation
match Rust semantics more closely by performing an exact point lookup
and returning:
* `null` when no rows are found
* the row when exactly one row is found
* (defensively) an error if >1 row is returned (unique index invariant
violation)
Similarly, `datastore_delete_by_index_scan_point_bsatn` was added and
used so deletes-by-unique-key are also exact-match point operations
rather than range deletes.
Runtime updates were made to utilize point scan in `FindSingle(key)` and
in both mutable/read-only unique-index paths.
To keep this non-breaking for existing modules, codegen now detects
whether the table row is a struct or a class and chooses the appropriate
base type:
* Struct rows: `Find()` returns `Row?` (`Nullable<Row>`).
* Class rows: `Find()` returns `Row?` (nullable reference, `null` on
miss).
# API and ABI breaking changes
This change is non-breaking with respect to row type kinds, because
class/record table rows continue to work via
RefUniqueIndex/ReadOnlyRefUniqueIndex while struct rows use
UniqueIndex/ReadOnlyUniqueIndex.
API surface changes:
* Generated `Find()` return type is now nullable (`Row?`) to correctly
represent “missing”.
ABI/runtime:
* Requires the point-scan hostcall import
(`datastore_index_scan_point_bsatn`) to be available; the runtime uses
point-scan for unique lookup (and point delete for unique delete).
# Expected complexity level and risk
Low 2
# Testing
- [X] Local testing: repro module + client validate view and direct
Find() behavior
---------
Signed-off-by: rekhoff <r.ekhoff@clockworklabs.io>
# Description of Changes
Implements the C# equivalent of #3638
This implement uses inheritance, where abstract base classes (like
`ProcedureContextBase` in `ProcedureContext.cs`) store the core of the
implementation, and then generated wrappers (like `ProcedureContext` in
the generated FFI.cs file) inherit from them.
For error handling, we work like Rust's implementation of `Result<T,E>`
but we require `where E : Exception` because of how exceptions work in
C#. Transaction-level failures come back as a `TxOutcome` and user
errors should follow the `Result<T,E>` pattern. In this implementation,
we have `UnwrapOrThrow()` throws exceptions directly because of C#'s
error handling pattern.
Unlike the Rust implementation's direct `Result` propagation, we are
using an `AbortGuard` pattern (in `ProcedureContext.cs`) for exception
handling, which uses `IDisposable` for automatic cleanup.
Most changes should have fairly similar Rust-equivalents beyond that.
For module authors, the changes here allow for the transation logic to
work like:
```csharp
ctx.TryWithTx<ResultType, Exception>(tx => {
// transaction logic
return Result<ResultType, Exception>.Ok(result);
});
```
This change includes a number of tests added to the
`sdks/csharp/examples~/regression-tests/`'s `server` and `client` to
validate the behavior of the changes. `server` changes provide further
usage examples for module authors.
# API and ABI breaking changes
Should not be a breaking change
# Expected complexity level and risk
2
# Testing
- [x] Created Regression Tests that show transitions in procedures
working in various ways, all of which pass.
# Description of Changes
Implements the C# module bindings for Procedures (#3510)
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
- [X] Locally tested against existing C# Procedures regression test
client (minus the Transaction and HTTP portions, as those are not in
this change).
# Description of Changes
1. Updates the Replication Tests in
`sdks/csharp/examples~/regression-tests` to include better coverage of
Views
2. Added missing linkage for __call_view__ and __call_view_anon__
3. Updated *ViewDispatcher Invoke to transform BSATN.ValueOption<> into
BSATN.List<>
4. Fixed issues with the indexing of views to match correctly during
__call_view__ and __call_view_anon__
# API and ABI breaking changes
No
# Expected complexity level and risk
2
# Testing
- [x] Running `run-regression-tests.sh` passes.
---------
Signed-off-by: rekhoff <r.ekhoff@clockworklabs.io>
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
Updates C# bindings to allow module authors to define `View`s and
`AnonymousView`s using the pattern:
```
[SpacetimeDB.View]
public static ExampleData? GetExampleDataById(ViewContext ctx, uint id)
{
return ctx.Db.ExampleData.Id.Find(id);
}
[SpacetimeDB.View]
public static ExampleData? GetAnonymousExampleDataById(AnonymousViewContext ctx, uint id)
{
return ctx.Db.ExampleData.Id.Find(id);
}
```
During publishing of a C# module, the views data will be converted into
`RawViewDefV9`
# API and ABI breaking changes
No known breaking changes
# Expected complexity level and risk
2
# Testing
- [X] Locally tested locally adding the above sample pattern to the
`sdks\csharp\examples~\regression-tests\server` test and performing a
`dotnet build` and a `spacetime publish test`, both of which succeeded.
---------
Signed-off-by: Jason Larabie <jason@clockworklabs.io>
Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes
`call_view` and `call_view_anon` maintains separate `Vec` list for view
inside Module but `ModuleDef` used to have single `Vec`. Hence, Views
index is not same across `ModuleDef` and module itself.
Added a `index` field inside `ViewDef` which stores it index inside
module.
# API and ABI breaking changes
NA
How complicated do you think these changes are? Grade on a scale from 1
to 5,
2,
there could be some existing code relying on poistion in `Vec` and may
not appear as compiler error
# Testing
Future smoketests should cover this.
# Description of Changes
This exposes JWT claims for csharp modules, similar to how they are
exposed to rust modules in
https://github.com/clockworklabs/SpacetimeDB/pull/3288.
This adds the new types `AuthCtx` and `JwtClaims`, and adds an `AuthCtx`
to the `ReducerContext`.
`AuthCtx` represents the credentials associated with the request, and
`JwtClaims` represents a jwt token.
One difference from the rust version is that I didn't create helpers to
build an `AuthCtx` from a jwt payload. The reason is that we would need
to be able to compute the identity from the payload claims, which
requires a blake3 hash implementation. The first two c# libraries I
found had issues at runtime
([Blake3](https://www.nuget.org/packages/Blake3) is wrapping a rust
implementation, and
[HashifyNet](https://github.com/Deskasoft/HashifyNET/tree/main/HashifyNet/Algorithms/Blake3)
seems to be broken by our trimming because it uses reflection heavily).
I can look into taking the implementation from `HashifyNet`, since it is
MIT licensed, but I don't think we need to block merging on that.
# API and ABI breaking changes
This adds the new types `AuthCtx` and `JwtClaims`, and adds an `AuthCtx`
to the `ReducerContext`.
This also adds a csharp wrapper for the get_jwt ABI function added in
https://github.com/clockworklabs/SpacetimeDB/pull/3288.
# Expected complexity level and risk
2.
# Testing
This has a very minimal unit test of JwtClaims.
I manually tested using this locally with the csharp quickstart, and I
was able to print jwt tokens inside the module.
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Adds type definitions for `RawViewDefV9` and `ViewDef`. Also validates
these module defs and populates the system tables:
- `st_table`
- `st_column`
- `st_view`
- `st_view_param`
- `st_view_column`
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
Integration tests to follow when views are added to
`__describe_module__`
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
This commit extends various schema and schema-adjacent structures to
describe procedures, a new kind of database function which are allowed
to perform side effects.
This includes extending `RawModuleDefV9` with a way to register
`RawProcedureDefV9`s in the `misc_exports`, preserving compatibility
with modules that predate procedures.
The module validation path is reorganized somewhat to validate various
properties related to procedures while preserving code clarity and
maintainability.
Additionally, the `ArgsTuple` machinery for ser/de-ing reducer arguments
using the argument type as a seed is extended to also support procedure
arguments.
All of this is currently unused.
# API and ABI breaking changes
Additive and backwards-compatible additions to `RawModuleDefV9` and
friends.
# Expected complexity level and risk
2 - some minor complexity in schema validation which may have gotten
borked in a merge at some point.
# Testing
Unsure what tests would be useful, open to suggestions from reviewers.
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
Host-side changes extracted from #3327
I added AUTO_INC_OVERFLOW even though we don't currently ever return it,
in order to future-proof so it's already there when we start emitting
it.
Prepublish was failing because it was expecting a wasm module
unconditionally, so now it takes ?host_type.
I tweaked JS deser to accept null/undefined when the unit type or an
option type is expected.
I switched to bsatn, because the native sats->js translator wasn't
matching what js was expecting.
I renamed the sys module: my thinking is that `spacetime:` as a scheme
will help disambiguate it, and maybe it could also be used for IMC in
the future or something? And I believe we had discussed wanting this to
be versioned, similar to wasm imports.
Trying to get a borrowed str from deserialize_js doesn't work, because
v8 strings don't store utf8.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] All this was done in the course of getting an actual typescript
module to successfully publish.
# Description of Changes
This is the implementation of issue #3191. This adds a Default attribute
to C# module fields.
**Note**: In C#, attribute arguments must be compile-time constants,
which means you can't directly use non-constant expressions like new
expressions, method calls, or dynamic values in attribute constructors.
(Ref:
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/attributes#2324-attribute-parameter-types)
For this reason, these default values are limited to primitive types,
enums, and strings.
This includes (shown as `C# Type` (`BSATN type`):
* `bool` (`Bool`)
* `sbyte` (`I8`)
* `byte` (`U8`)
* `short` (`I16`)
* `ushort` (`U16`)
* `int` (`I32`)
* `unit` (`U32`)
* `long` (`I64`)
* `ulong` (`U64`)
* `float` (`F32`)
* `double` (`F64`)
* `enum` (`Enum`)
* `string` (`String`)
* `null` (`RefOption`) <- Nullable type
Because of C# limitations, for nullable and complex data types, such as
a struct, can take use `[Default(null)]` to populate values with null
defaults. This allows things like structs to workaround the non-constant
expressions in attribute constructors limitation by allowing these
complex types to still be able to be added as new column tables.
The `int` type can also be in the form of Hex or Binary literals, such
as `[Default(0x2A)]` or `[Default(0b00101010)]`
Both Decimal (like `[Default(3.14m)]`) and Char (like `[Default('A')]`)
are unsupported types in BSATN and will still return `BSATN0001` errors.
# API and ABI breaking changes
Not API breaking.
This change only adds the `[Default(value)]` attribute logic.
Using the `[Default(value)]` attribute with older versions SpacetimeDB
C# modules will result in an error.
# Expected complexity level and risk
2
# Testing
Local testing of this requires use of CLI changes in #3278
- [x] Regression test of functionality added.
# Description of Changes
Each reducer gets its arguments through an `ArgSource`, a Unix-file-like
abstraction for streams of bytes. Prior to this commit, we had an ABI
designed as if it could support other args sources, but it actually
hardcoded the ID of the reducer args source, and errored elsewhere.
This commit extends the `BytesSource` infrastructure to support other
bytes sources. This will be useful for exposing JWT payloads and HTTP
responses. No other `BytesSource` uses are actually included in this
commit, only the infrastructure.
This commit also defines a new host call,
`bytes_source_remaining_length`. This is intended to allow callers to
pre-allocate a buffer correctly sized to read the entire `BytesSource`
all at once. The new host function is added to a new ABI minor version,
10.1, so that old SpacetimeDB hosts can detect and reject too-new
compiled modules. I have added uses of this new function to
`__call_reducer__` in both Rust and C#, even though it's not strictly
necessary,
and I haven't removed the loop which repeatedly calls
`bytes_source_read` and grows the buffer.
# API and ABI breaking changes
Adds a new ABI minor version, `spacetime_10.1`. This means that old
SpacetimeDB hosts will reject new compiled modules.
# Expected complexity level and risk
2-ish? WASM ABI code is always fiddly, but this is a pretty simple case.
# Testing
- [x] New behavior and new host function are both hit through existing
tests that instantiate modules and call reducers against them, so I
believe automated testing is sufficient.
---------
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes
Closes https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1987
# API and ABI breaking changes
This does not affect any APIs but it does affect the code generated
# Expected complexity level and risk
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Snap tests
# Description of Changes
This adds auto migration support to the `schema` crate.
Additional changes are needed to fully implement this feature: see
https://github.com/clockworklabs/SpacetimeDB/issues/2912
# API and ABI breaking changes
None
# Expected complexity level and risk
1, at this level the change is fairly straightforward. The integration
will be more difficult.
# Testing
Added `schema` unit tests for success/failure cases.
---------
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
Ryan