mirror of
https://github.com/clockworklabs/SpacetimeDB.git
synced 2026-07-26 12:03:37 -04:00
bot/sql-format-option
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Shubham Mishra
|
e2f8a60759 |
Case conversion (#4263)
# Description of Changes Update the Default casing policy to `snake_case` for `RawModuleDefV10`. Messy PR contains changes at different places, so that CI can pass: Here are the main changes as follows: - `bindings-macro` & `bindings` crate: `name` macro in Indexes for canonical name and supply it to `RawModuleDefV10` via `ExplicitNames`. - `bindings-typescript`: - Changes has been reviewed through this PR - https://github.com/clockworklabs/SpacetimeDB/pull/4308. - `binding-csharp`: a single line change to pass `sourceName` of index instead of null. - `codegen`: - Changes has been merged from branch - https://github.com/clockworklabs/SpacetimeDB/pull/4337. - Except a fix in rust codegen to use canonical name in Query buillder instead of accessor. - `lib/db/raw_def`: Extends `RawDefModuleV10` structure to support case conversion. - `schema` crate: - `validate/v9` - Nothing itself should change or changes in v9 validation logic but the file contains a `CoreValidator` which is shared with `validate/v10`. No test have t be updated to `validate/v9` which ensures we aren't regressing it. - `validate/v10`: This is the main meat, look at the new tests added in bottom to understand what it does. - Rest of the files are either test updates or module bindings. ## Testing: 1. Extensive unit tests have been added to verify generated `ModuleDef` is correct. 2. I have done some e2e testing to verify rust codegen with rust and typescript modules. 3. I would have like to do more testing for other codegens , I am continue doing . I have removed `sql.py` smoketest, as that seems to be already migated in new framework and was headache to update. ## Expected complexity level and risk 4, It could have side-effect which aren't easily visible. - - - --------- Signed-off-by: Shubham Mishra <shivam828787@gmail.com> Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com> Co-authored-by: joshua-spacetime <josh@clockworklabs.io> Co-authored-by: Noa <coolreader18@gmail.com> Co-authored-by: = <cloutiertyler@gmail.com> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@clockworklabs.io> Co-authored-by: Jason Larabie <jason@clockworklabs.io> Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io> |
||
|
Shubham Mishra
|
de2b65fb8d |
SQL alias (#4304)
# Description of Changes - [x] : Store "accessor"s names in ModuleDef (for tables, columns, and indexes for now) and pass them to TableSchema as aliases. - [x] Create accessor system tables for each of them: `st_table_accessor`,`st_column_accessor`, `st_index_accessor` - [x] Update the SQL parsing logic to also consider alias names. Alias name stored will be same as actual names until we merge https://github.com/clockworklabs/SpacetimeDB/pull/4263 # API and ABI breaking changes NA # Expected complexity level and risk 3 This patch updates SQL name resolution which isn't a very complex change since table and column name resolution are each handled by their own helper method. It also adds new system tables which need to be handle correctly during restore/replay. We do have regression tests for this that should catch any incorrect handling or missing steps. I would say the main risk is that we don't populate cached table and column schemas with the alias/accessor name. Regression tests have been added to catch this, but there are edge cases around migrations and replay that currently aren't covered. # Testing Added SQL smoketests that use canonical/accessor table/column names. They currently fail because we're currently using the accessor name for both. They should pass once this is updated. --------- Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com> Co-authored-by: joshua-spacetime <josh@clockworklabs.io> |
||
|
Shubham Mishra
|
e4098f98d9 |
Rust: macro change name -> accessor (#4264)
## Description of Changes
This PR primarily affects the `bindings-macro` and `schema` crates to
review:
### Core changes
1. Replaces the `name` macro with `accessor` for **Tables, Views,
Procedures, and Reducers** in Rust modules.
2. Extends `RawModuleDefV10` with a new section for:
* case conversion policies
* explicit names
New sections are not validated in this PR so not functional.
3. Updates index behavior:
* Index names are now always **system-generated** for clients. Which
will be fixed in follow-up PR when we start validating RawModuleDef with
explicit names.
* The `accessor` name for an index is used only inside the module.
## Breaking changes (API/ABI)
1. **Rust modules**
* The `name` macro must be replaced with `accessor`.
2. **Client bindings (all languages)**
* Index names are now system-generated instead of using explicitly
provided names.
**Complexity:** 3
A follow-up PR will reintroduce explicit names with support for case
conversion.
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
|
||
|
Tyler Cloutier
|
9686139dbc |
Translate smoketests from Python to Rust (#4102)
# Description of Changes
This PR translates all of our Python smoketests into Rust tests which
can be run from `cargo run`
## Motivation
The purpose of this fivefold:
1. All developers on the team are familiar with Rust
2. It simplifies our devops because we can drop Python as a dependency
to run the tests
3. You can now run all tests in the repo through the single `cargo test`
interface
4. Because we use the `SpacetimeDbGuard` and `cargo test`/`cargo
nextest` we can easily parallelize the smoke tests
5. The smoketests can now use machinery imported from SpacetimeDB crates
(e.g. `bsatn` etc.)
IMPORTANT NOTE!
There are several ways to implement the smoke tests in Rust (none are
great):
1. A separate xtask specifically for the smoke tests
- This doesn't solve the problem of the CLI tests which also use the
`guard` crate
- Idiosyncratic way to run the smoke tests as opposed to cargo test
- Does NOT resolve the cargo within cargo problem because we still have
to build the test modules with cargo
2. A `build.rs` script in `guard` which first builds the executables as
a compile step for compiling guard
- Deadlocks on a cargo lock file conflict (Outer cargo compiles guard →
runs build.rs, inner cargo tries to acquire the build directory lock,
outer cargo holds the directory lock, deadlock)
- If you fix the deadlock by using different target dirs, it still looks
stuck on building guard because it's actually compiling all of
spacetimedb-standalone and spacetimedb-cli.
- Still technically runs cargo inside of cargo.
3. Add `spacetimedb-cli` and `spacetimedb-standalone` as an artifact
dependency of the guard crate
- Has good and clear output but requires +nightly when running the
smoketests and CLI tests, otherwise won't do the right thing. See
https://github.com/rust-lang/cargo/issues/9096
4. Compile the executables at runtime during the tests themselves where
the first test takes a lock while the executables are building using
cargo within cargo
- Makes the tests look like they're taking a long time when they're just
waiting for the build to complete
- Requires relatively complex locking machinery across
binaries/tests/processes
5. A two step solution where the developer has to build the binaries
before calling the smoke tests
- Very error prone
None of these are good. `xtask` is not bad, but doesn't enable us to run
other integration tests in other crates (e.g. the CLI)
(3) is the correct solution and has the best user experience, but it
requires nightly and I don't want to introduce that for all of our
tests.
I have chosen to do a combination of (1) and (4). You will now run the
smoketests with `cargo smoketest`. If you run `cargo test --all` (or use
`guard`) without doing `cargo smoketest` it will fall back to (4) which
compiles the executables at runtime. Running `cargo build` is the **only
way** to ensure that the executables are not stale because of the
internal fingerprint checking. Everything else is fragile not robust.
NOTE! There is no way to avoid cargo within cargo and have the smoke
tests be run as cargo tests because the modules under test must be
compiled with cargo.
# API and ABI breaking changes
Note that this is a BREAKING CHANGE to `cargo test --all`. The
smoketests are now part of `cargo test --all` unless you specifically
exclude them.
# Expected complexity level and risk
3, this is partially AI translated. We need to carefully review to
ensure the semantics have not regressed.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
|