mirror of
https://github.com/clockworklabs/SpacetimeDB.git
synced 2026-05-06 15:49:35 -04:00
bradleyshep
efa6f382b1
# Description of Changes LLM benchmark updates for local development: - **Local SDK paths**: Templates use relative paths to workspace crates (`crates/bindings`, `crates/bindings-csharp`, `crates/bindings-typescript`) instead of published packages, so the bench runs against local SDK changes. - **NODEJS_DIR support**: On Windows (e.g. nvm4w), if `pnpm` is not on PATH, the bench uses `NODEJS_DIR` to locate `pnpm` and prepends it to PATH for subprocesses. - **Refactor**: Extracted `relative_to_workspace()` in `templates.rs` and removed noisy `NODEJS_DIR` logging in `publishers.rs`. - **Benchmark results**: Updated `docs/llms/llm-comparison-details.json` and `docs/llms/llm-comparison-summary.json`. # API and ABI breaking changes None. # Expected complexity level and risk **2** — Local-only changes to the benchmark tool. Templates now require local SDKs to be built (especially TypeScript: `pnpm build` in `crates/bindings-typescript`). No impact on published SDKs or runtime. # Testing - [ ] Run `cargo llm run --lang rust --modes docs --providers openai` from repo root - [ ] Run TypeScript benchmarks with `pnpm build` in `crates/bindings-typescript` first - [ ] On Windows with nvm4w, set `NODEJS_DIR` if `pnpm` is not on PATH and run TypeScript benchmarks --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>