# Description of Changes
See the inline comments for the motivation. This was originally
introduced to our Windows CI in #3351. This PR moves it from CI to
general Windows target builds, since it seems like Windows builds are
now generally having this issue.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] Windows CI still passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Check that our generated C# files are up-to-date in our CI.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI all passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
* Small cleanups in `tools/check-diff.sh`
* Use `tools/check-diff.sh` wherever appropriate
* Simplify the `sdks/csharp/tools~/gen-*.sh` files after the repo merge
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] CI still passes
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Necessary for pulling in rolldown.
# API and ABI breaking changes
None
# Expected complexity level and risk
1, with the caveat that this updates the Rust version and therefore
touches all the code.
# Testing
- [ ] Just the automated testing
# Description of Changes
I changed some variables related to caching in the TypeScript test CI,
since it was failing on master due to suspected cache issues.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] CI passes on this PR when it didn't on master
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
v8: use fast static strings for known strings
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Covered by existing.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fix bindgen tests (due to crate `timezone_provider`)
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Fixes CI tests.
# Description of Changes
I chose to regenerate all of the bindings in
https://github.com/clockworklabs/SpacetimeDB/pull/3310, but that caused
the snapshots to change. This PR follows up to update them.
I also updated the outdated script `tools~/gen-quickstart-chat.sh`
(which was generating from a different module than the CI was), and
updated the CI to use it instead of manually running a similar command.
# API and ABI breaking changes
Test-only changes.
# Expected complexity level and risk
1
# Testing
- [x] unity-testsuite passes on this PR (which it does not on `master`).
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We had places that weren't passing `-y`. I assume these were only
working because they were in environments where the things were already
installed, so there wasn't any confirmation dialog. For some reason, the
arm runner now newly needs to install packages, so CI invocations began
failing.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI passes
- [ ] the "Test spacetimedb-update" flow passes on arm, which is not
currently true elsewhere.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Move the unity testsuite to `spacetimedb-runner` since it otherwise runs
out of disk space.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] Unity CI passes now.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Move the global.json file to the repo root, which should make `dotnet`
resolve the correct SDK regardless of the directory it is run in.
Where a copy of the global.json file is needed, symlink it to the one at
the root.
Should fix "C#/Unity - Test Suite".
# Description of Changes
As the title says.
Add `--pg-port NUM` to the `start` command
# API and ABI breaking changes
Before this, it was set to `5432` unconditionally.
Docs updated at https://github.com/clockworklabs/SpacetimeDB/pull/3302.
# Expected complexity level and risk
1
# Testing
- [x] Run smoke tests
# Description of Changes
This PR removes the `@clockworklabs/typescript-sdk` from the repository
and retains only `spacetimedb` in the `crates/bindings-typescript`
directory. Some files are migrated to `spacetimedb`. I have also updated
the appropriate READMEs.
In addition I have symlinked the old `sdks/typescript` directory to
point to `crates/bindings-typescript`.
# API and ABI breaking changes
This is not technically a breaking change of any kind, although it does
orphan and deprecate the
[@clockworklabs/spacetimedb-sdk](https://www.npmjs.com/package/@clockworklabs/spacetimedb-sdk)
npm package. This package will no longer work with SpacetimeDB.
Users should now install and use the `spacetimedb` package.
# Expected complexity level and risk
2, it's a straightforward change but affects many files.
# Testing
- [ ] I ran `pnpm test` in the `spacetimedb` package
- [ ] I ran the quickstart app
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Closes#3219
This adds the Unreal SDK, the new Unreal test cases, updates the test
runner to handle Unreal, codegen updates for Unreal, and a QuickStart
Chat.
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
2 - This impacts the subcommand generate.rs to include unrealcpp and
crates/testing to expand for Unreal
# Testing
- [x] Run the new Unreal tests
- [x] Run any previous automation testing - with all the changes to
generate/testing I'm uncertain if there is an impact
- [x] Review the new CLI generate documentation changes
---------
Co-authored-by: Phoebe Goldman <phoebe@goldman-tribe.org>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
This PR moves most of the contents of `@clockworklabs/spacetimedb-sdk`
into the `spacetimedb` module. The `spacetimedb` module now exports
`sdk` and `server` as separate subpaths where `sdk` contains the code
which was previously in `@clockworklabs/spacetimedb-sdk`.
In particular it makes the following moves:
- `/sdks/typescript/packages/sdk` -> `/sdks/typescript`
- most of the contents of `/sdks/typescript/packages/sdk` ->
`crates/bindings-typescript`
- `/sdks/typescript/packages/test-app` ->
`crates/bindings-typescript/test-app`
The following packags was NOT moved:
`/sdks/typescript/examples/quickstart-chat`
## Motivation
In accordance with
https://github.com/clockworklabs/SpacetimeDB/issues/3250, we would like
to consolidate `@clockworklabs/spacetimedb-sdk` into a single
`spacetimedb` package so that users can import the different things they
need from a single package.
### Pros:
- allow users to install a single package with subpaths `spacetimedb`,
`spacetimedb/react`, `spacetimedb/sdk`, `spacetimedb/server`, etc.
- Is much simpler for bundling, etc.
- Is backwards compatible with `@clockworklabs/spacetimedb-sdk` which
now becomes a thin wrapper
- eventually allow us to break up the `spacetimedb` package into other
packages if we want to split them up (e.g. `@spacetimedb/lib`,
`@spacetimedb/sdk`, etc.) and we can solve the build complexity that
introduces when we get to it
- eventually allow us to move `bindings-csharp` out of the crates
directory where it probably doesn't belong anyway
- organizes all TypeScript packages into the packages directory where
you'd normally expect them, with the possible exception of
`/sdks/typescript` if we wanted to leave that separate
### Cons:
- The `sdk` directory is now a bit of a ruse as to where the code
actually lives since it's just a thin wrapper. If it eventually becomes
its own independent package, we'll also have to break up spacetimedb
into `@spacetimedb/lib` and `@spacetimedb/server` so that
`@clockworklabs/spacetimedb-sdk` can depend on `@spacetimedb/lib` while
being a dependency of `spacetimedb`.
Ideally this change would have been made later, however, it became
necessary for the following **heinously disastrous chain of forcing
moves**:
1. Adding `react` support necessitated shipping react as an optional
peer dependency under `@clockworklabs/spacetimedb-sdk/react`
2. This required adding a new build target/export/bundle
3. Previously `@clockworklabs/spacetimedb-sdk` was configured to have
`noExternal` for `spacetimedb` meaning it would collect the library into
the sdk bundle. I attempted to continue this for react support but...
4. Creating a new `react` bundle which also pulled in `spacetimedb`
caused their to be nominal type conflicts between classes in the
duplicate `spacetimedb` bundles.
5. Changing `spacetimedb` to be included as `external` caused compile
errors because `@clockworklabs/spacetimedb-sdk` is configured in
`tsconfig.json` to fail on unused variables and it was now including the
source of `spacetimedb` which is not configured to error on unused
variables and has "unused" private variables which are actually used by
us secretly, but not exposed to the clients.
> SIDE NOTE: The unused variables settings cannot be turned off on a
line by line basis, so it has to be turned off entirely, but in order to
maintain the linting checks we had I used `eslint` to enforce the rule
so that we could disable it line by line. (This caused me to discover
quite a lot of things that were broken that were caught by `eslint`
being applied to the entire project. `eslint` was previously only
applied to the `quickstart-chat` and the `crates/bindings-typescript`
library)
6. Changing the build to be external, now requires `spacetimedb` to also
be published to npm as its own module which
`@clockworklabs/spacetimedb-sdk` now imports, which requires that we add
`tsup` config to `spacetimedb` to publish a built version of the
library.
7. The only way to avoid that is to move the `sdk` and `react` code from
`@clockworklabs/spacetimedb-sdk` into the existing `spacetimedb` package
to avoid the duplicate import problem on step 4 and change
`@clockworklabs/spacetimedb-sdk` back to again use `noExternal` for its
`spacetimedb` dependency.
And here we are. I chose not to move `/crates/bindings-typescript` even
though that's probably not a great place long term. It would be better
to have it in `/packages/spacetimedb` or `/npm-packages/spacetimedb` or
`/ts-packages/spacetimedb` or something, and move all our TypeScript
packages in there. But that is a different matter.
The net result however is that we have a new `spacetimedb` package which
exports the different parts of the API under:
- `spacetimedb`
- `spacetimedb/server`
- `spacetimedb/sdk`
- `spacetimedb/react`
while still not breaking the existing deploy process, nor any
users/developers who are currently using
`@clockworklabs/spacetimedb-sdk`.
I think long term should we ever decide to split `spacetimedb` up into
multiple packages or if we have additional unrelated packages, we should
publish them to the `@spacetimedb` org which I reserved for us here:
https://www.npmjs.com/org/spacetimedb
> NOTE: `spacetimedb` is a package and `@spacetimedb` is an org.
`spacetimedb/sdk` is not a separate package, it's a subpath export of
the `spacetimedb` package, whereas `@spacetimedb/sdk` would be (and
would need to be) it's own separate package. You can certainly have both
`spacetimedb/sdk` and `@spacetimedb/sdk`. We could for example host the
code for the sdk at `@spacetimedb/sdk` and just reexport it from
`spacetimedb` under the `spacetimedb/sdk` subpath.
# API and ABI breaking changes
This should not change or modify the API or ABI in any way. If it does
so accidentally it is a bug, although I carefully went through the
exports.
# Expected complexity level and risk
3 because it changes how the SDK is built a bit and rearranges a lot of
paths.
# Testing
- [x] All of the CI passes
- [x] I also ran quickstart-chat to confirm that it is not broken
- [x] I also ran test-app
# Description of Changes
Closes
[#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686).
Add support for listening using the [PG wire
protocol](https://www.postgresql.org/docs/current/protocol.html) so `pg`
clients could be used against the database.
# API and ABI breaking changes
The output of `duration` is changed to `rfc3339`, instead of the way is
made with `sats` because is what is done in `pg`, see note below.
# Expected complexity level and risk
2
~~There is open questions that are in the [ticket
#2686](https://github.com/clockworklabs/SpacetimeDB/issues/2686). Also
the crate used here require `RustTls`, so it could be good idea to
decide if~~:
* ~~Rewrite a big chunk of code to use `OpenSSL`~~
* ~~Move to `RustTls`
https://github.com/clockworklabs/SpacetimeDB/pull/1700~~
* ~~Pay for the extra compilation cost~~.
I open another port(`5433`) to listen for `pg` connections using `ssl`.
Need to be decided if this is the way or instead try to multi-plex the
current port for both protocols.
# Testing
Only manual testing so far. Solving the above questions allow me to
implement some unit tests. Also, not yet integrated into cloud for the
same reasons.
- [x] Adding some test for the binary encoding of special and primitive
types
- [x] Smoke test using `psql` that connect to the db instance and run
some queries
- [x] Manually inspect using a UI database explorer how infer the types,
some of this tools generate special widgets when displaying `json,
duration, etc`
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
This is a purely additive change to introduce, `TypeBuilder`s,
`ColumnBuilder`s, and a `t` factory export that has methods for creating
`TypeBuilder`s.
There are derived types from `TypeBuilder`s for each variant of
`AlgebraicType`. `TypeBuilder`s can be converted into a `ColumnBuilder`
which supports creating additional metadata specifying whether a column
is a primary key or should be unique. `ColumnBuilder`s are not allowed
within `TypeBuilder`s, but the opposite is allowed (for composite
types).
`ColumnBuilder`s can only be used at the top level and contain a
`TypeBuilder`.
`TypeBuilder` stores three pieces of information:
1. A phantom typescript type that the SATS type should be interpreted as
in TS
2. A phantom typescript type which is the type of the SATS type
3. A runtime AlgebraicType which stores the information to be reported
to host.
`ColumnBuilder` stores three pieces of information:
1. A `TypeBuilder`
2. A phantom typescript type with the column metadata
3. A runtime `ColumnMetadata` which stores the information to be
reported to the host.
Additionally, it is only possible to add metadata to `ColumnBuilder` or
`TypeBuilder` types which are compatible with that metadata. e.g. It is
a compile-time error to add `isAutoIncrementing` to a `String`
type/column.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I've created a set of type tests in `type_builders.test-d.ts`
- [x] I also created a set of runtime tests in `index.test.ts`
- [x] I have modified CI so that these tests run on every PR
# Description of Changes
This PR:
- standardizes the prettier config across all TypeScript projects
- adds a root level package.json
- standardizes all `pnpm` commands to be the same
- updates documentation accordingly
- adds some additional typescript testing for serialization and
deserialization
**IMPORTANT!** Once this PR merges we will need to change the
`compile-and-test` required check to `build-and-test`
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
2 - It in principle doesn't change any code, but could affect deploy
processes.
# Testing
- [x] Just the automated testing that we had previously
- [x] I added additional automated tests
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We had weird caching issues in the C#/Unity testsuite. Somehow, they got
triggered only as of
https://github.com/clockworklabs/SpacetimeDB/pull/3181 merging, and I
have no idea why/how.
I've restored the `id` field of the checkout step (which is used by the
cache step), and this _seems_ to have fixed it.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
- [x] It passes on this PR
- [x] It passes in a test PR that combines this change with
https://github.com/clockworklabs/SpacetimeDB/pull/3182
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We haven't used this in a while and we're pretty sure it's currently
broken.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
A few of our new CI checks weren't set up to run in the merge queue,
which prevented PRs from merging when those checks were marked required.
# API and ABI breaking changes
None. CI only change.
# Expected complexity level and risk
1
# Testing
This will have to be tested by making these checks required again, and
then seeing if this PR can merge.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Just what it says on the can. This test suite can sometimes mysteriously
hang for a long time, so a timeout will help kill this (and free up
resources) earlier.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None really. I guess I could test this by adding a step that just sleeps
forever, but this is basically just setting some data.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
New Unity test suite runs will cancel existing in-progress runs on the
same PR or github ref.
I tried to add a global limit of 2 test suites running at the same time,
but I did not find a successful way of making this work.
# API and ABI breaking changes
None. CI-only change.
# Expected complexity level and risk
1
# Testing
- [x] Test suite still runs and succeeds
- [x] If I push several commits in a row, previous runs get cancelled
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Some small CI tweaks related to the monorepo merge.
Once we've landed these, I think we can make most of the new checks
required.
# API and ABI breaking changes
CI only changes
# Expected complexity level and risk
1
# Testing
None, just tweaked names.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Noa