# Description of Changes
This PR removes the `@clockworklabs/typescript-sdk` from the repository
and retains only `spacetimedb` in the `crates/bindings-typescript`
directory. Some files are migrated to `spacetimedb`. I have also updated
the appropriate READMEs.
In addition I have symlinked the old `sdks/typescript` directory to
point to `crates/bindings-typescript`.
# API and ABI breaking changes
This is not technically a breaking change of any kind, although it does
orphan and deprecate the
[@clockworklabs/spacetimedb-sdk](https://www.npmjs.com/package/@clockworklabs/spacetimedb-sdk)
npm package. This package will no longer work with SpacetimeDB.
Users should now install and use the `spacetimedb` package.
# Expected complexity level and risk
2, it's a straightforward change but affects many files.
# Testing
- [ ] I ran `pnpm test` in the `spacetimedb` package
- [ ] I ran the quickstart app
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This adds `react` as an optional peer dependency. If the TypeScript SDK
is imported into a library that uses React (of the appropriate version)
then they will have access to two new React hooks:
```ts
useSpacetimeDB<DbConnection>();
useTable<DbConnection, MyTable>('my_table');
```
This PR also updates the TypeScript `quickstart-chat` tutorial in the
docs to use the new React hooks. I will split that tutorial into
separate `React` and `TypeScript` tutorials in a future PR.
# API and ABI breaking changes
This is a purely additive change to the SDK
# Expected complexity level and risk
2 because it changes how the SDK is built a bit. Namely it makes the
`spacetimedb` library an external dependency.
# Testing
TODO: I am not through testing yet.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Julien Lavocat <JulienLavocat@users.noreply.github.com>
# Description of Changes
This PR moves most of the contents of `@clockworklabs/spacetimedb-sdk`
into the `spacetimedb` module. The `spacetimedb` module now exports
`sdk` and `server` as separate subpaths where `sdk` contains the code
which was previously in `@clockworklabs/spacetimedb-sdk`.
In particular it makes the following moves:
- `/sdks/typescript/packages/sdk` -> `/sdks/typescript`
- most of the contents of `/sdks/typescript/packages/sdk` ->
`crates/bindings-typescript`
- `/sdks/typescript/packages/test-app` ->
`crates/bindings-typescript/test-app`
The following packags was NOT moved:
`/sdks/typescript/examples/quickstart-chat`
## Motivation
In accordance with
https://github.com/clockworklabs/SpacetimeDB/issues/3250, we would like
to consolidate `@clockworklabs/spacetimedb-sdk` into a single
`spacetimedb` package so that users can import the different things they
need from a single package.
### Pros:
- allow users to install a single package with subpaths `spacetimedb`,
`spacetimedb/react`, `spacetimedb/sdk`, `spacetimedb/server`, etc.
- Is much simpler for bundling, etc.
- Is backwards compatible with `@clockworklabs/spacetimedb-sdk` which
now becomes a thin wrapper
- eventually allow us to break up the `spacetimedb` package into other
packages if we want to split them up (e.g. `@spacetimedb/lib`,
`@spacetimedb/sdk`, etc.) and we can solve the build complexity that
introduces when we get to it
- eventually allow us to move `bindings-csharp` out of the crates
directory where it probably doesn't belong anyway
- organizes all TypeScript packages into the packages directory where
you'd normally expect them, with the possible exception of
`/sdks/typescript` if we wanted to leave that separate
### Cons:
- The `sdk` directory is now a bit of a ruse as to where the code
actually lives since it's just a thin wrapper. If it eventually becomes
its own independent package, we'll also have to break up spacetimedb
into `@spacetimedb/lib` and `@spacetimedb/server` so that
`@clockworklabs/spacetimedb-sdk` can depend on `@spacetimedb/lib` while
being a dependency of `spacetimedb`.
Ideally this change would have been made later, however, it became
necessary for the following **heinously disastrous chain of forcing
moves**:
1. Adding `react` support necessitated shipping react as an optional
peer dependency under `@clockworklabs/spacetimedb-sdk/react`
2. This required adding a new build target/export/bundle
3. Previously `@clockworklabs/spacetimedb-sdk` was configured to have
`noExternal` for `spacetimedb` meaning it would collect the library into
the sdk bundle. I attempted to continue this for react support but...
4. Creating a new `react` bundle which also pulled in `spacetimedb`
caused their to be nominal type conflicts between classes in the
duplicate `spacetimedb` bundles.
5. Changing `spacetimedb` to be included as `external` caused compile
errors because `@clockworklabs/spacetimedb-sdk` is configured in
`tsconfig.json` to fail on unused variables and it was now including the
source of `spacetimedb` which is not configured to error on unused
variables and has "unused" private variables which are actually used by
us secretly, but not exposed to the clients.
> SIDE NOTE: The unused variables settings cannot be turned off on a
line by line basis, so it has to be turned off entirely, but in order to
maintain the linting checks we had I used `eslint` to enforce the rule
so that we could disable it line by line. (This caused me to discover
quite a lot of things that were broken that were caught by `eslint`
being applied to the entire project. `eslint` was previously only
applied to the `quickstart-chat` and the `crates/bindings-typescript`
library)
6. Changing the build to be external, now requires `spacetimedb` to also
be published to npm as its own module which
`@clockworklabs/spacetimedb-sdk` now imports, which requires that we add
`tsup` config to `spacetimedb` to publish a built version of the
library.
7. The only way to avoid that is to move the `sdk` and `react` code from
`@clockworklabs/spacetimedb-sdk` into the existing `spacetimedb` package
to avoid the duplicate import problem on step 4 and change
`@clockworklabs/spacetimedb-sdk` back to again use `noExternal` for its
`spacetimedb` dependency.
And here we are. I chose not to move `/crates/bindings-typescript` even
though that's probably not a great place long term. It would be better
to have it in `/packages/spacetimedb` or `/npm-packages/spacetimedb` or
`/ts-packages/spacetimedb` or something, and move all our TypeScript
packages in there. But that is a different matter.
The net result however is that we have a new `spacetimedb` package which
exports the different parts of the API under:
- `spacetimedb`
- `spacetimedb/server`
- `spacetimedb/sdk`
- `spacetimedb/react`
while still not breaking the existing deploy process, nor any
users/developers who are currently using
`@clockworklabs/spacetimedb-sdk`.
I think long term should we ever decide to split `spacetimedb` up into
multiple packages or if we have additional unrelated packages, we should
publish them to the `@spacetimedb` org which I reserved for us here:
https://www.npmjs.com/org/spacetimedb
> NOTE: `spacetimedb` is a package and `@spacetimedb` is an org.
`spacetimedb/sdk` is not a separate package, it's a subpath export of
the `spacetimedb` package, whereas `@spacetimedb/sdk` would be (and
would need to be) it's own separate package. You can certainly have both
`spacetimedb/sdk` and `@spacetimedb/sdk`. We could for example host the
code for the sdk at `@spacetimedb/sdk` and just reexport it from
`spacetimedb` under the `spacetimedb/sdk` subpath.
# API and ABI breaking changes
This should not change or modify the API or ABI in any way. If it does
so accidentally it is a bug, although I carefully went through the
exports.
# Expected complexity level and risk
3 because it changes how the SDK is built a bit and rearranges a lot of
paths.
# Testing
- [x] All of the CI passes
- [x] I also ran quickstart-chat to confirm that it is not broken
- [x] I also ran test-app
# Description of Changes
This just constrains the types of rows from `any` to `Record<string,
any>`.
# API and ABI breaking changes
Not breaking, purely internal.
# Expected complexity level and risk
1
# Testing
- [x] Just the automated tests
# Description of Changes
This is a purely additive change to introduce, `TypeBuilder`s,
`ColumnBuilder`s, and a `t` factory export that has methods for creating
`TypeBuilder`s.
There are derived types from `TypeBuilder`s for each variant of
`AlgebraicType`. `TypeBuilder`s can be converted into a `ColumnBuilder`
which supports creating additional metadata specifying whether a column
is a primary key or should be unique. `ColumnBuilder`s are not allowed
within `TypeBuilder`s, but the opposite is allowed (for composite
types).
`ColumnBuilder`s can only be used at the top level and contain a
`TypeBuilder`.
`TypeBuilder` stores three pieces of information:
1. A phantom typescript type that the SATS type should be interpreted as
in TS
2. A phantom typescript type which is the type of the SATS type
3. A runtime AlgebraicType which stores the information to be reported
to host.
`ColumnBuilder` stores three pieces of information:
1. A `TypeBuilder`
2. A phantom typescript type with the column metadata
3. A runtime `ColumnMetadata` which stores the information to be
reported to the host.
Additionally, it is only possible to add metadata to `ColumnBuilder` or
`TypeBuilder` types which are compatible with that metadata. e.g. It is
a compile-time error to add `isAutoIncrementing` to a `String`
type/column.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I've created a set of type tests in `type_builders.test-d.ts`
- [x] I also created a set of runtime tests in `index.test.ts`
- [x] I have modified CI so that these tests run on every PR
# Description of Changes
Previously all TypeScript generated code had a `// @ts-ignore` directive
at the top to suppress unused variable errors that users may have run
into with their generated code. This worked to suppress unused, but it
was too broad and suppressed actual errors with our generated code. This
PR removes that directive and fixes the latent errors.
Most notably this fixes the issue in the quickstart where we were
getting `serialize` is not a function.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] This PR increases the test surface only by now typechecking the
typescript generated code within our repository
# Description of Changes
Closes https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1987
# API and ABI breaking changes
This does not affect any APIs but it does affect the code generated
# Expected complexity level and risk
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Snap tests
# Description of Changes
This PR:
- standardizes the prettier config across all TypeScript projects
- adds a root level package.json
- standardizes all `pnpm` commands to be the same
- updates documentation accordingly
- adds some additional typescript testing for serialization and
deserialization
**IMPORTANT!** Once this PR merges we will need to change the
`compile-and-test` required check to `build-and-test`
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
2 - It in principle doesn't change any code, but could affect deploy
processes.
# Testing
- [x] Just the automated testing that we had previously
- [x] I added additional automated tests
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Please note, much of the code changed in this PR is generated code.
This change updates the TypeScript SDK to use a new `spacetimedb`
TypeScript library which lives under the `/crates/bindings-typescript`
folder alongside `/crates/bindings-csharp`. Just like with the C#
bindings library, the types for `AlgebraicType` and `RawModuleDef` are
now code generated with a script in `/crates/codegen/examples`.
Pulling this out into a library allows us to use the same types and
serialization code on both the server and the client for TypeScript
modules.
In the process of making this change I also found and fixed several
issues with the TypeScript code generation. Namely an issue with
recursive types and an issue with the `never` type. I also removed any
use of `namespace`s since those are a TypeScript only feature, and we
want to have JavaScript + types so that we can use the generated code
with ESBuild without TSC.
I have also improved the npm/pnpm scripts to be able to generate
TypeScript code for us automatically by running `pnpm generate` for any
place that we have to generate typescript code. Namely:
- Quickstart module bindings
- AlgebraicType/ModuleDef for TypeScript module library
- Client API messages for the TypeScript API
- TestApp module bindings
# API and ABI breaking changes
IMPORTANT! This is an API breaking change for clients, as such it should
be a major version change. However, I am going to see if I can shim in
the old API as best as possible to make it compatible.
Notably, we were previously exporting APIs that end users do not need,
and I don't think it would ever occur to them to use, namely the whole
`AlgebraicValue` API and also the `AlgebraicType` API. In principle, no
one should have a need for these, although it was technically possible
for them to use it.
Indeed, we could potentially even just remove AlgebraicType completely
from the API by directly code generating the serialization code.
Listed below are all of the **BREAKING** changes to the API and their
effect:
- `AlgebraicType` is now a structural union literal type instead of a
class (nominal type), this is a consequence of generating the type with
our code gen. Users did not have a reason to use `AlgebraicType`
directly.
- The `AlgebraicValue` type has been removed entirely. This was
previously a class that was exported from the SDK, but very unlikely to
be used by users.
- The `ProductValue` type has been removed.
- The `ReducerArgsAdapter` and `ValueAdapter` types, which were used
with AlgebraicValues have been removed.
- Generated code has changed incompatibly so users will have to
regenerate code when upgrading to this version. Technically a breaking
change, but pretty easy to fix.
Listed below are the non-breaking API changes:
- I am now exporting generated product types as the default export in
addition to how I was exporting them before
- For generated sum type `T`, I am now exporting a `TVariants` namespace
which has the types of all the variants for `T`. This was previously
exposed on the namespace `T`, but was inaccessible in modules other than
the one it was defined in because I also export a type `T` in addition
to namespace `T` and in the type position `T` was being interpreted as a
type rather than a namespace. It's unclear why TypeScript resolved it as
the `T` namespace within the module in which is was defined previously.
Anyway, since the old types were apparently unobservable to users, I've
replaced them with the types in `TVariants`. (Open to other names for
this namespace).
- I fixed a bug where never types (sum types with no variants) were not
correctly generated.
# Expected complexity level and risk
3. The most complex thing about the PR are the potential impacts to the
API. I am reasonably certain, but not 100% certain that I have accounted
for everything above.
# Testing
- [x] I ran `pnpm test` which runs all the tests in the repo including
an integration test which tests the connection to SpacetimeDB. I also
fixed broken tests.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Simple patch version bump. Because we forgot to bump the version
number for 1.3.1 this is bumping from 1.3.0 => 1.3.2
# API and ABI breaking changes
0
# Testing
- Verified that the version number in the license has been updated
- Verified the output of `spacetime --version`:
```
$ spacetime --version
spacetime Path: C:\Users\boppy\AppData\Local\SpacetimeDB\bin\current\spacetimedb-cli.exe
Commit: 0027d094e1
spacetimedb tool version 1.3.2; spacetimedb-lib version 1.3.2;
```
---------
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Apparently, I missed several license files in #3002. I'm not sure what
method I was using to find them, but apparently it was insufficient.
**This replaces all empty `LICENSE` files with an explicit (symlink to)
BSL license, and all apache licenses with symlinks to the root apache
license.** This PR does not intentionally change any license terms, so
if you see one that changed, **it's a mistake**.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
```bash
$ find . -name '*LICENSE*' -type f | grep -v '\.meta$'
./crates/sqltest/standards/LICENSE # this one is an external library that we are not allowed to re-license
./LICENSE.txt # this is the root license
```
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We recently merged several repos together. This PR clarifies the license
terms for several subdirectories, as well as the relationship between
the licenses.
The licenses in our subdirectories have become symbolic links to
licenses in our toplevel `licenses` directory. For any particular
subdirectory's license file in the diff, you can click `... -> View
file` and then click on the text that says "Symbolic Link" on that page.
This will take you to the license file that it links to.
I have also updated the `tools/upgrade-version` script to update the
change date in the new `licenses/BSL.txt` file.
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None. Only changes to license files.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Fixes a Typescript SDK bug where onclose of WebsocketDecompressAdapter
is never called
# API and ABI breaking changes
None
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
Potential risk as Typescript SDK consumers might be currently relying
only on onConnectError to report disconnection. Users should instead use
onDisconnect, or can simply subscribe to both callbacks with the same
function again to get the same functionality.
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
This is a trivial change, but could have minor repercussions on current
code. I'd still say it's a level 1 change, with consumers able to easily
change to the correct callback.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
# Description of Changes
Added a repo migration notice workflow to each repository that we've
merged into SpacetimeDB.
These need to be mirrored back to the actual repos with a process like:
```bash
git subtree split --prefix=sdks/typescript -b release/typescript
git push -f git@github.com:clockworklabs/spacetimedb-typescript-sdk.git release/typescript:main
git branch -D release/typescript
```
Once they're migrated there, the bot will automatically comment on any
PR or issue opened in those repos.
# API and ABI breaking changes
None. CI-only changes.
# Expected complexity level and risk
2
# Testing
- [x] In a demo repo, it properly commented and closed a PR:
https://github.com/clockworklabs/github-tooling-test/pull/42
- [x] In a demo repo, it properly commented and closed an issue:
https://github.com/clockworklabs/github-tooling-test/issues/43
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
We want to release #2980 for the TS SDK, so I'm bumping our version
number.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None, just a version bump.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
The `ScheduleAt` type appears to have an outdated structure. I've
updated the structure of the `ScheduleAt` type represented in TypeScript
to be in line with the Rust type:
https://docs.rs/spacetimedb/latest/spacetimedb/enum.ScheduleAt.html
Namely, we were missing the inner Spacetime library types of
`TimeDuration` and `Timestamp`.
This is to address #2969.
# API and ABI breaking changes
This is an API breaking change in that it changes the API, but it's
breaking in that it fixes a bug.
# Expected complexity level and risk
2
# Testing
I have not done additional testing, but I thought I would get this PR
started to make it easier for the next person who comes along.
The ideal automated test to add would be one which connects a TypeScript
client to a SpacetimeDB module with a scheduled table, and verifies that
the type is correctly deserialized and represented in TypeScript when
the rows are sent down.
---------
Co-authored-by: Jeffrey Dallatezza <jeffreydallatezza@gmail.com>
# Description of Changes
Bumped all versions to 1.3.0 in preparation for an upcoming minor
release.
# API and ABI breaking changes
No breaking changes.
# Expected complexity level and risk
1
# Testing
None
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Bumping this in preparation for an upcoming release.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
None, just a version bump
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
This PR helps to support React Native in the Typescript SDK. We have
identified two issues:
1. Certain versions of React Native exhibit a bug where the constructor
mistakenly treats a URL object as a string. This causes an error when
the constructor attempts to call `.endsWith()` on the URL object,
leading to runtime errors. This PR adds a .toString() call when passing
a URL instance to the URL constructor.
2. React Native doesn't provide an implementation for TextEncoder and
TextDecoder which are used to read/write strings in our binary reader
and writer. This PR use introduce the usage of a library for these two
types.
Note: this still requires the use of a Polyfill as React Native URL
implementation is missing most methods in version older than 0.79 (>= 6
month old)
* Get it working with base64 encoded row ids, and start adding the better way for indexing.
* Use concat to avoid creating too many arguments for big updates.
* Use primitives as rowIds
* Remove OperationsMap
* Add some table cache tests and regen stuff.
* Cleanup
* revert accidental cargo change.
* Add semver parsing.
* Add a minimum version check.
* Use qmark syntax.
* Regen quickstart-chat bindings with 1.2.0 cli
* Bump minimum codegen version.
* Regen test app with 1.2
* Test quickstart-chat in CI
* fix?
* ignore diffs containing SpacetimeDB commit
* prettier config
* fix?
* [REVERT THIS TEST COMMIT] add diff to module bindings
* revert previous test commit
* [REVERT THIS TEST COMMIT] introduce a typechecking error
* revert previous test commit
* tiny review
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
* Allow spacetimedb sdk to work when spacetimedb url is more than a host name.
* run pnpm format
---------
Co-authored-by: DeveloperChaseLewis <developer.chase.lewis@gmail.com>
* Regen the api types from the server.
* fmt the client api code
* Use the multi versions of subscription messages.
* Add ref counting of rows and delay callbacks.
* WIP debugging tests.
* Regen test app
* A lot of changes.
* Use new subscription api.
* remove commented out code.
* Fix update and cleanup.
* Fix path to spacetimedb
* Run tests before checking diff in CI
* Update packages/sdk/tests/db_connection.test.ts
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
* Update packages/sdk/tests/db_connection.test.ts
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
* Update packages/sdk/tests/db_connection.test.ts
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
* Update packages/sdk/tests/db_connection.test.ts
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
* Various cleanups.
* Comment out the CI test for the generated code being up to date.
* Regen.
* Add sed command.
---------
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Tyler Cloutier