# Description of Changes
Updates views atomically on commit, but before downgrading to a
read-only transaction for subscription evaluation.
What this patch does:
1. Renames `ViewId` to `ViewFnPtr`
2. Renames `ViewDatabaseId` to `ViewId`
3. Removes the `module_rx` module watcher from the subscription manager
4. Refactors read sets to only track table scans (index key tracking
will be added later)
5. Drops read sets and removes rows from `st_view_sub` when dropping a
view in an auto-migrate
6. Re-evaluates and updates views (`call_views_with_tx`) from
`call_reducer_with_tx` for any view whose read set overlaps with the
reducer's write set
7. Does the same for sql dml
# API and ABI breaking changes
None
# Expected complexity level and risk
3
It's a bit of a messy diff.
# Testing
- [x] Integrate with
https://github.com/clockworklabs/SpacetimeDB/pull/3616
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
I allowed chatgpt to mislead me about how to do these format strings.
Apparently this is the wrong syntax. I've now verified the correct
syntax:
https://docs.github.com/en/actions/reference/workflows-and-actions/expressions#format
# API and ABI breaking changes
CI only.
# Expected complexity level and risk
1
# Testing
I honestly don't know how to check what concurrency group something is
running in..
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Disables parameterized views in rust until we can pass arguments from
sql. Note parameters are already disabled in C# and typescript.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Needed to update some tests. Will revert these changes once we re-enable
parameters.
# Description of Changes
`schema_for_table_raw` was hardcoded to return `None` for `ViewInfo`.
This issue was not observable when the module launched for the first
time, but during module re-launches, it caused problems with `impl
CollectViews`, as the implementation was treating view tables as normal
tables.
# API and ABI breaking changes
NA
# Expected complexity level and risk
1, pretty obvious.
# Description of Changes
Following discussions about having our Inkeep's API key being public, I
added a comment to all the places where we use it saying that this is OK
and a link to the documentation.
As per Inkeep's documentation:
> When you embed your AI assistant in websites where the user is not
authenticated, like in your public docs, help center, or marketing site,
the web browser is considered a "public client". Since many of these
clients often don't have a backend, the most practical way to use the AI
assistant is for UI components to talk directly to the Inkeep service.
While it is up to your company's own policies and best practices, in
these scenarios, the Inkeep API key is generally ok to be included in
the source code of your web page that is exposed in the browser. This is
similar to how [Algolia's search
service](https://support.algolia.com/hc/en-us/articles/18966776061329-Can-the-search-API-key-be-public)
or [Sentry's error
logging](https://forum.sentry.io/t/how-does-sentry-prevent-spammers/8188)
works.
https://docs.inkeep.com/cloud/ui-components/public-api-keys#public-clients
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Ran the docs locally to ensure that the comment isn't visible on the
markdown pages
# Description of Changes
This commit adds support to the Rust client SDK for calling procedures.
Similar to reducers, each `DbContext` implementor has a `pub procedures:
RemoteProcedures` field, with methods provided by extension traits for
each procedure.
Unlike reducers, the provided methods are invoke and invoke-then.
Invoke-then takes a `FnOnce` callback to run
when the SDK is notified of the procedure's termination status, while
invoke ignores that notification.
No mechanism is provided for observing procedures invoked by other
clients.
Procedure callbacks are implemented by storing a map from `request_id`
to `ProcedureCallback`, with the callback closure internally knowing how
to deserialize the return value. It's mildly unfortunate to deserialize
within the callback instead of on the preprocess background task, but it
saves significant complexity.
This commit also adds a new sdk-test module, `sdk-test-procedures`, and
a new Rust test client, `procedure-client`.
Together, these are used in two tests of invoking and observing
procedures. I've left TODOs for other tests that we should write as we
implement additional procedure features.
I also had to fix a few minor bugs in the Rust codegen which were not
strictly related to procedures: we previously assumed that the sets of
reducers and of tables were non-empty, which led to wonky invalid
codegen on modules which did not define any reducers or which did not
define any tables.
I'm sneaking a change to the Nix flake into this PR as well - when
initially writing it I had included `cargoArtifacts` (the pre-built and
cached dependencies of our actual builds) in its `packages`, but that
was neither necessary or useful, and just made building the shell take a
long time after dependency changes.
# API and ABI breaking changes
Breaks the internal interface between the Rust client SDK and codegen,
so users will have to re-run `spacetime generate`.
# Expected complexity level and risk
2-ish? Pretty simple change to the Rust SDK overall.
# Testing
- [x] Added new automated integration tests exercising the new
functionality.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
When a new commitlog segment is created, allocate disk space for it up
to the maximum segment size. Also do this when resuming writes to an
existing segment, such that segments created without preallocation will
allocate as well when the database is opened.
Preallocation is gated behind the feature "fallocate", because it is not
always desirable to preallocate, e.g. for local `standalone` users.
The feature can only be enabled on Linux targets, because allocation is
done using the Linux-specific `fallocate(2)` system call.
Unlike `ftruncate(2)` or the portable `posix_fallocate(3)`,
`fallocate(2)`
supports allocating disk space without zeroing. This is currently
required, because the commitlog format does not handle padding bytes.
If not enough space can be allocated, the commitlog refuses writes. For
commitlogs that were created without preallocation, this means that the
commitlog cannot even be opened in this situation.
The local durability impl will crash if it detects that the commitlog is
unable to allocate enough space.
This means that a database will eventually crash and be unable to start
in
an out-of-space situation.
Allocated space is not included in the reported size of the commitlog.
Instead, allocated blocks are reported separately.
# Expected complexity level and risk
3 - Disk size monitoring may need to be adjusted.
# Testing
- [x] Adds a test that demonstrates the crash behavior of
[`spacetimedb_durability::Local`]
when there is insufficient space. The test performs I/O against a loop
device.
- [x] Modified the `repo::Memory` impl so that it can run out of space.
No test currently
utilizes this, but existing tests assuming infinite space still pass.
# Description of Changes
The main thing this does is implement `WasmInstance` for `V8Instance`,
so we can now change all the `vm_call*: impl FnOnce()` args on
`InstanceCommon` to instead just take `&mut impl WasmInstance`.
(Obviously our javascript instance is not a wasm instance, but
`wasmtime::module_host_actor` is desperately in need of a rebrand in
general, so I just left `WasmInstance`'s as is.)
This vastly simplifies the callsites, and also allows @joshua-spacetime
to call views from inside of `call_reducer_with_tx`, which was the main
motivating factor behind this.
If the last commit is a bit much, I'm happy to split it off - I just
don't get explicitly asked to refactor the rough edges of the codebase
very often :3
# Expected complexity level and risk
2 - it's a refactor, but overall pretty straightforward, and Rust lets
you do fearless refactoring etc etc
# Testing
n/a - internal code change
---------
Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: joshua-spacetime <josh@clockworklabs.io>
# Description of Changes
This patch:
1. Materializes views on subscribe and sql calls by invoking `call_view`
on the `ModuleHost`.
2. Downgrades to a read-only transaction after view materialization but
before query execution.
3. Updates the `st_view_sub` system table on both subscribe and
unsubscribe.
4. Makes subscribe methods on the SubscriptionManager async.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
End-to-end tests to be added with atomic view updates
# Description of Changes
For some reason, the Internal Tests have trouble fetching the commit
sha, especially when the job is re-run. This PR switches it to using ref
names rather than commit sha, because the ref names are much more
durable than GitHub's ephemeral commits that it generates during
workflows.
# API and ABI breaking changes
None. CI only.
# Expected complexity level and risk
1
# Testing
- [x] CI still passes
- [x] Ref still gets checked out successfully on re-run.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Currently need changes from both #3548 and #3527, and the latter is
merged into master but the former isn't rebased on top of it. so we stay
silly for the moment
I'd like to pull out my first commit into its own PR, but that's not
really possible until #3548 rebases onto master.
# Expected complexity level and risk
2 - pretty straightforward translation of the wasm/Rust view
implementation to typescript.
# Testing
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
When computing rowId for non primary key tables the typescript sdk
incorrectly used the entire data buffer.
This resulted in rowIds being incorrect when multiple rows were present
within an update.
# Expected complexity level and risk
1
# Testing
Tested this in the context of developing the collaborator feature on the
spacetime.com website.
Without this change, updates to tables without primary keys were
sometimes not applied to the client cache.
With this update no further issues were observed.
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
# Description of Changes
Problem is visible in replication smoketests, which exit and re-lauch
modules, erorring out with:
```
ERROR /app/public/crates/core/src/host/host_controller.rs:818: Failed to open database: DatabaseError: Database is already opened. Path:`/stdb/data/replicas/4`. Error:Resource temporarily unavailable (os error 11).: Database is already opened. Path:`/stdb/data/replicas/4`. Error:Resource temporarily unavailable (os error 11)
```
Commented code will be a blocker for releasing Views but we want CIs to
pass.
# Description of Changes
`call_view` and `call_view_anon` maintains separate `Vec` list for view
inside Module but `ModuleDef` used to have single `Vec`. Hence, Views
index is not same across `ModuleDef` and module itself.
Added a `index` field inside `ViewDef` which stores it index inside
module.
# API and ABI breaking changes
NA
How complicated do you think these changes are? Grade on a scale from 1
to 5,
2,
there could be some existing code relying on poistion in `Vec` and may
not appear as compiler error
# Testing
Future smoketests should cover this.
# Description of Changes
Our `Dockerfile` runs `COPY . .`. If we're in a submodule rather than a
"full" git repo, that causes `.git` to be a in a weird state inside the
docker container, where it thinks it's in a submodule even though the
parent directory context has been lost.
This PR just adds a clear error message early in the process, because
this has bitten us a few times.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
- [x] We appropriately get the failure if we try to `docker build` this
from within a submodule.
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This updates a bunch of types to make sure we generate the right
functions for indexes with multiple columns. Most of the changes are
making the array of column names readonly.
This also has a fix for `AllUnique`, because we were treating composite
indexes as unique when the first field had a unique index. FWIW, we
should also make sure that the range scanning functions still exist in
that case, but I didn't try to add that here.
# API and ABI breaking changes
Any change to these types is probably technically a breaking API change,
but no one should consider these types stable.
# Expected complexity level and risk
1.5. I'm still not super confident with some of these types, but this
shouldn't change any runtime behavior.
# Testing
I added some uses of indexes in `schema.test-d.ts`, so `pnpm run
build:types` would throw errors if this were broken.
# Description of Changes
Host implementation to invoke `call_view` method.
I also covers:
1. API `MutTxId::is_materialized`to check if existing view exisits and
updated.
2. Update in readsets logic to remove stale views.
3. sql caller implmentation.
# API and ABI breaking changes
NA
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
3
# Description of Changes
Following #3498 , run the `gen-client-api.sh` tool to fix one of the
non-required CI checks.
# API and ABI breaking changes
N/a
# Expected complexity level and risk
1
# Testing
Automated testing is fine.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Refactored `st_view_client` and renamed it `st_view_sub` which tracks
the number of clients subscribed to a view. On disconnect, we decrement
the `num_subscribers` column in the appropriate rows. An async task will
be in charge of cleaning up views (and their read sets) whose subscriber
count has gone to zero (not in this patch).
On module init, we clear the entirety of each view table.
# API and ABI breaking changes
None. Technically this updates the schema of a system table, but the
system table was added and modified between releases.
# Expected complexity level and risk
~2
Need to make sure we cover all cases so that we don't leave dangling
data. Making these tables ephemeral in the future should simplify this.
# Testing
Will add tests once we can subscribe to views
# Description of Changes
The merge queue was (partly) getting borked because we were putting all
non-PR CI events into the same concurrency group, which meant they all
non-PR CI jobs would run sequentially instead of running in parallel.
This sometimes caused _painfully_ long delays in the merge queue.
This was due to my misunderstanding in
https://github.com/clockworklabs/SpacetimeDB/pull/3501#discussion_r2466570395,
where I didn't realize that `cancel-in-progress: false` would cause
everything to queue up.
Now, for non-PR events, we append the commit SHA to the concurrency
group. For merge queue events, this should be the SHA of the ephemeral
merge commit that GH creates, so it will never conflict. For push events
or manual workflow dispatch events, the SHA should be a sane way to
recognize/cancel redundant events.
# API and ABI breaking changes
None. CI-only change.
# Expected complexity level and risk
1
# Testing
- [x] PR CI passes on this PR
- [x] PR CI is still canceled on this PR if a new commit is pushed
Unfortunately it's hard to test the behavior for non-PR events without
merging and seeing if it works.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This reverts commit #3496.
# API and ABI breaking changes
Technically maybe yes? But definitely nothing is using the new code yet.
# Expected complexity level and risk
1
# Testing
CI only
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
This patch does the following:
1. Expands views as part of query planning. Views are always assumed to
be materialized by the query planner, however a view's backing table may
have private columns such as the `sender` column. The query planner
needs to filter by this column in order to select the rows pertaining to
a particular caller.
2. Plumbs `AuthCtx` through the query optimizer. This is needed in order
to implement (1).
3. Adds a new operator for views to the query engine that drops a view's
private columns
# API and ABI breaking changes
None
# Expected complexity level and risk
2.5
# Testing
- [x] SQL http tests
- [ ] Subscription tests
- [ ] One off query tests
# Description of Changes
Now running `python -m smoketests` no longer updates
`smoketests/config.toml` with a new identity, and also `replication`
doesn't fail when docker isn't available.
# Expected complexity level and risk
1
# Testing
- [x] those things no longer happen
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
So far, the `--clear-database` option to `publish` has simply dropped
and then re-created the database (if it did exist).
This will no longer work when databases can have "children": because
dropping and re-creating is not atomic, children would either become
orphans, or be dropped as well.
To solve this, `reset_database` is introduced as a separate action that:
- shuts down all replicas
- if a `program_bytes` is supplied, replaces the database's initial
program
- if a `host_type` is supplied, replaces the database's host type
- starts `num_replicas` or the previous number of replicas, which
initialize themselves as normal
As this could be its own CLI command, the action is provided as its own
API endpoint (undocumented).
However, since `publish` has no way of knowing whether the database it
operates on actually exists, the `publish_database` handler will just
invoke the `reset_database` handler if `clear=true` and the database
exists, and return its result. This is to avoid starting the transfer of
the program in the request body, only to receive a redirect.
Some refactoring was necessary to dissect and understand the flow.
# API and ABI breaking changes
Introduces a new, undocumented API endpoint.
We may want to nest it under `/unstable`.
# Expected complexity level and risk
2
# Testing
From the outside, the observed behavior should be as before,
so smoketests should cover it.
# Description of Changes
Not many changes were required for the query compiler to be able to
resolve views. This is because the query engine can always assume a view
is materialized and therefore has a backing table. So from the
perspective of the query engine, a view is just another table with one
small caveat: The physical table in the datastore has two internal
metadata columns - `sender` and `arg_id`. These columns are not user
facing and so should be hidden from name resolution/type checking.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] SQL type checking tests
# Description of Changes
The previous `syscall.rs` is now in `syscall/v1.rs`, and there's proper
parsing of the `sys@X.Y` version number done now. This lets us more
easily extend the ABI in the future.
I also changed how slots are stored in the Context embedder data, to
make it easier to add more in the future and more performant to fetch
them.
# Expected complexity level and risk
1
# Testing
n/a, reorganization
---------
Signed-off-by: Noa <coolreader18@gmail.com>
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Specifically generate client table handles for views.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Added a `#[view]` to `module-test` and generated new client snapshots
# Description of Changes
Bump versions to 1.7.0 in preparation for the release.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
# Expected complexity level and risk
1
# Testing
- [x] CI passes
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes
Inkeep's API key for the search bar is the staging one instead of
production, therefore requests made are failing due to an incorrect
referrer.
I've fixed this on `docs/release` quickly after we migrated the new docs
but haven't made a PR to add it to master, due to this the change was
lost on `docs/release` in the last docs release.
I just cherry-picked the commit that I had on my local branch
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
Cherry-pick of a commit that was already deployed
# Description of Changes
When building under Nix, Git metadata is not available within the
sandbox, as we use `lib.cleanSource` on our source directory. This is
important because it avoids spurious rebuilds and/or determinism
hazards.
The build was broken due to our new `spacetime init` template system
accessing Git metadata in the CLI's build.rs
to filter out non-git-tracked files from the templates. The Flake
sandbox does this automatically (even without `lib.cleanSource`!), so
when building under Nix it's unnecessary to do twice. (I remain
unconvinced that it's necessary to do in non-Nix builds either, as CI
builds should have a clean checkout and local dev builds don't need
clean templates, but the behavior was already in master and I didn't
feel comfortable removing it.)
As an enhancement, I've also found a Nix-ey way to embed our Git commit
hash in builds. Previously, builds under Nix had the empty string
instead of a commit hash, because we included the `git` CLI tool but
scrubbed the necessary metadata. Now, we inject an environment variable
from the Nix flake, and don't make the `git` CLI tool available at all.
This has the convenient upside of allowing Nix builds to reference
`dirtyRev` in builds with a dirty worktree, which should reduce
confusion.
# API and ABI breaking changes
N/a
# Expected complexity level and risk
3? I didn't have a strong understanding of what the CLI build script was
doing, and to what extent it was doing things intentionally versus for
convenience. As such, it's possible that I've inadvertently damaged
something load-bearing.
# Testing
- [x] Built with `nix build`, ran `spacetime init`, chose the
`basic-rust` template, and got a reasonable-looking template
instantiation.
- [ ] Hopefully we have automated tests for this?
# Description of Changes
Based on #3461, which fixes a bug encountered in the quickstart.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
n/a
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Yes
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
# Description of Changes
The GitHub edit links were disabled in the new docs, this reenables
them.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Tested locally that this works.
# Description of Changes
Creates a multi-column index on the `sender` and `arg_id` columns of a
view's backing table so that we can quickly retrieve a view's result set
for each caller.
<!-- Please describe your change, mention any related tickets, and so on
here. -->
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
This is an internal index. No module bindings are generated for it.
Uniqueness of the index name follows from the uniqueness of the table
name since views live in the table namespace.
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
Performance change
# Description of Changes
The `aud` claim is required in the OIDC spec, but the server currently
allows it to be missing, and the spacetime auth tokens we use for the
website don't have an audience.
Previously the module bindings would throw an error if there were no
`aud` claim in a jwt payload (if someone used the `audience` within a
reducer), but this change makes us treat a missing audience as an empty
list.
This also renames the `authCtx` fields to `senderAuth` in the typescript
and csharp module APIs, so they match rust.
# API and ABI breaking changes
This doesn't break any ABIs.
This changes the ReducerContext APIs in typescript and rust, but only by
renaming a field that hasn't been released yet.
# Expected complexity level and risk
1.
# Testing
I've tested accessing the `audience` within a reducer for a token
missing an `aud` claim in Typescript and Rust.
# Description of Changes
This PR implements a new command for the CLI: `spacetime dev`. If run
from outside of a project directory, it will call `init` in interactive
mode to allow user to create a project. If run from a project directory,
it will ask for a module to conect to and it will:
* subscribe to logs
* observe files in the `spacetimedb` directory and automatically publish
to SpacetimeDB
One caveat is that instead of reusing the `logs` code I did some
repetition cause currently the logs code acquires a lock that lives
through await points. It is fixable, but due to limited time I decided
to go with a bit of code duplication. It shouldn't be very hard to fix
later, though.
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
- Fixes an issue in the new TypeScript module quickstart which would
cause users to encounter an error.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] I did the quickstart from start to finish and it works now without
hitting any errors
Manual test output:
```
boppy@geralt MINGW64 ~/clockwork/SpacetimeDB/quickstart-chat (jdetter/fix-typescript-quickstart)
$ spacetime publish --project-path spacetimedb quickstart-chat
Build finished successfully.
Uploading to local => http://127.0.0.1:3000
Checking for breaking changes...
Publishing module...
JavaScript / TypeScript support is currently in BETA.
There may be bugs. Please file issues if you encounter any.
<https://github.com/clockworklabs/SpacetimeDB/issues/new>
Created new database with name: quickstart-chat, identity: c200a4abaa304575600961ee0a39210a536bcd9d52d14580ce56dd51627013f5
boppy@geralt MINGW64 ~/clockwork/SpacetimeDB/quickstart-chat (jdetter/fix-typescript-quickstart)
$ spacetime call quickstart-chat send_message "Hello, World!"
WARNING: This command is UNSTABLE and subject to breaking changes.
boppy@geralt MINGW64 ~/clockwork/SpacetimeDB/quickstart-chat (jdetter/fix-typescript-quickstart)
$ spacetime logs quickstart-chat
2025-10-30T15:50:23.882048Z INFO: Creating table `message`
2025-10-30T15:50:23.882191Z INFO: Creating table `user`
2025-10-30T15:50:23.882713Z INFO: Invoking `init` reducer
2025-10-30T15:50:23.883599Z INFO: Database initialized
2025-10-30T15:50:31.461395Z INFO: send_message spacetimedb_module:3640: User c20093cd8df79879ae6d2b058650ca5112d6bb11334b5003f443cc0776da3d15: Hello, World!
boppy@geralt MINGW64 ~/clockwork/SpacetimeDB/quickstart-chat (jdetter/fix-typescript-quickstart)
$ spacetime sql quickstart-chat "SELECT * FROM message"
WARNING: This command is UNSTABLE and subject to breaking changes.
sender | sent | text
--------------------------------------------------------------------+----------------------------------+-----------------
0xc20093cd8df79879ae6d2b058650ca5112d6bb11334b5003f443cc0776da3d15 | 2025-10-30T15:50:31.460991+00:00 | "Hello, World!"
```
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
Adds two new system tables for views. One tracks view arguments, the
other tracks view subscribers.
`st_view_arg` generates a unique id for each unique argument
instantiation. This id will act as a foreign key that both read sets and
backing tables can reference.
`st_view_client` is needed so that we can drop views when clients
unsubscribe or disconnect. Note this disconnect logic is not implemented
in this patch.
Eventually both of these tables should be ephemeral. There's no reason
they need to write to the commitlog.
Note also that the schema of a view's backing table has been updated. It
no longer stores a view's argument values but rather the `id` from
`st_view_arg`.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
Tests will be added in the patch that computes read sets
---------
Signed-off-by: joshua-spacetime <josh@clockworklabs.io>
Co-authored-by: Shubham Mishra <shivam828787@gmail.com>
# Description of Changes
This commit builds support for executing procedures in WASM modules.
This includes an HTTP endpoint,
`/v1/database/:name_or_address/procedure/:name POST`, as well as an
extension to the WS protocol. These new APIs are not wired up to the CLI
or SDKs, but I have manually tested the HTTP endpoint via `curl`. The
new WS extensions are completely untested.
Several TODOs are scattered throughout the new code, most notably for
sensibly tracking procedure execution time in the metrics.
I also expect that we will want to remove the `procedure_sleep_until`
syscall and the `ProcedureContext::sleep_until` method prior to release.
# API and ABI breaking changes
Adds new APIs and ABIs.
# Expected complexity level and risk
3? 4? Unlikely to break existing stuff, 'cause it's mostly additive, but
adds plenty of potentially-fragile new stuff. Notably is the first time
we're doing anything actually `async`hronous on a database core Tokio
worker, and we don't yet have strong evidence of how that will affect
reducer execution.
# Testing
- [x] Manually published `modules/module-test` and executed procedures
with the following `curl` invocations:
- `curl -X POST -H "Content-Type:application/json" -d '[]'
http://localhost:3000/v1/database/module-test/procedure/sleep_one_second`
- `curl -X POST -H "Content-Type:application/json" -d '[1223]'
http://localhost:3000/v1/database/module-test/procedure/return_value`
- [ ] Need to write automated tests.
---------
Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
This is a draft of the new functionality for `spacetime init`. In order
to run it with built-in templates you have to set the path to the config
file:
```
export SPACETIMEDB_CLI_TEMPLATES_FILE=crates/cli/.init-templates.json
```
In the future it will fetch the list from GH.
A few notes:
* the previous functionality of `spacetime init` does not work at the
moment
* the code needs a bit more cleanup and tests before merging
* there is a bit of a mix in how we generate empty server and client
projects. For Rust we use the existing way of generating. For TypeScript
we clone an empty project from the repo. I wanted to play with both ways
of doing things, and I'm still not sure which is better. Generation in
Rust means that the generated code will match the CLI version and not
necessarily whatever is in Git. On the other hand, for the builtin
templates we will be fetching the newest version from GH, which I guess
might also not what we want, ie. we probably want only stable templates.
More discussion is needed here
* we use `spacetimedb` directory for the server files
* I don't particularly like the inability to disable interactive mode
easily. We discussed disabling it by default if all of the required
arguments are passed, but I don't think it's feature proof. For example,
if someone relies on a non-interactive mode, and we add a new required
argument, instead of printing a message `missing --foo`, we will
automatically launch interactive mode, which is harder to debug. That's
why I think I'd prefer to implement `--non-interactive` argument
* it's kind of hard to keep the legacy behaviour. If you don't pass any
arguments, we go into interactive mode. In the legacy version, we would
print required arguments. If someone passes `--lang` or `--project-path`
explicitly, I guess we could run the legacy workflow, but not sure if
it's worth it, as the command was marked as unstable anyway
* the project path defaults to the project name, but I think we should
probably replace change whitespaces to dashes, or at least ask for the
project path with the project name being the default (or both)
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: = <cloutiertyler@gmail.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Co-authored-by: Tyler Cloutier <cloutiertyler@aol.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
- Expanded on the Unreal tutorial to create the Blueprint version,
~~this uses :::server-cpp and :::server-blueprint which might not work
yet when deploying~~ with using Docusaurus Tabs.
- Updated the server side code to use signed integers as discussed for
both Rust + C# server modules.
# API and ABI breaking changes
None
# Expected complexity level and risk
1 - Documentation update, does update the server side
# Testing
- [x] Manually tested both Rust + C# modules
- [x] Ran through all images and steps
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
The query engine can now execute queries using mutable transactions
because this patch implements the `Datastore` trait for `MutTxId`.
Note this is both a refactor and a feature patch. It is a refactor
because the `Datastore` trait was updated to allow for mutable
transactions.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
1.5
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [x] Current tests pass after refactor
- [ ] Use `MutTxId` for query execution
The commitlog creates new segments atomically, returning EEXIST if the
segment already exists. This is to break a retry loop in case the
filesystem becomes unwritable.
This error did not contain any context about what does not exist, so
this patch adds some.
Also, an unhandled edge case has been discovered:
When opening an existing log, the commitlog will try to resume the last
segment for writing. If it finds a corrupt commit in that segment, it
won't resume, but instead create a new segment at the corrupt commit's
offset + 1.
However, if the first commit in the last segment is corrupted, the
offset will be that of the last segment -- trying to start a new segment
will thus fail with EEXIST.
Without additional recovery mechanisms, it is not obvious what to do in
this case: the segment could contain valid data after the initial
commit, so we certainly don't want to throw it away.
Instead, we now detect this case and return `InvalidData` with some
context.
# Expected complexity level and risk
1
# Testing
- [ ] A (regression) test is included
# Description of Changes
<!-- Please describe your change, mention any related tickets, and so on
here. -->
The `InstanceEnv` will now compute read sets when executing a view and
store them in the `MutTxId`.
These read sets track table scans as well as singular index key scans.
Index key ranges will be tracked in the future, but for now an index
range scan is treated as a full table scan.
These read sets are maintained as part of the `CommittedState`.
TODO: Check write sets against read sets. Re-evaluate views in the case
of overlap and update the read sets.
# API and ABI breaking changes
<!-- If this is an API or ABI breaking change, please apply the
corresponding GitHub label. -->
None
# Expected complexity level and risk
<!--
How complicated do you think these changes are? Grade on a scale from 1
to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex
change.
This complexity rating applies not only to the complexity apparent in
the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning
ways. -->
1
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] Unit tests
# Description of Changes
For core dumps Unreal tries to use setrlimit() for infinity when setting
resource limits, it seems this becomes an issue intermittently when
running tests in parallel. There is an option when running Unreal to
disable this and we can rely on error longs only which makes sense for
CI imo.
# API and ABI breaking changes
None
# Expected complexity level and risk
1 - small flag change
# Testing
- [x] Re-ran all tests locally
# Description of Changes
These tests fail on external PRs, but not for any real reasons - just
because GH secrets are missing. "Skipped" is more informative than
"failed".
# API and ABI breaking changes
None.
# Expected complexity level and risk
1
# Testing
None, but I just copied the logic from the unity testsuite.
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
joshua-spacetime