mirror of
https://github.com/supabase/supabase.git
synced 2026-06-28 11:33:52 -04:00
Pamela Chia
20290c71bd
## Summary Since the guides UA-redirect shipped (GROWTH-811), named LLM bots requesting `/docs/guides/*` get rewritten to the markdown handler, which returns a 404 when no `.md` file exists. About 90K of those 404s per day land on real pages that serve HTML 200 fine: the bot gets nothing on a page that works. The root cause is that the docs middleware hardcoded `hasMarkdownVariant: true` for every guide path, so it never checked whether a `.md` actually existed. I fixed it in two layers: 1. A build-time slug manifest makes `hasMarkdownVariant` truthful. Guide pages with no `.md` now fall through to HTML 200 instead of a 404. This is content-source-agnostic and future-proof: a new content source can never silently regress to a 404. 2. A second generator pass emits real markdown for the troubleshooting collection (the largest source, ~70% of the 404 volume), so those bots get clean markdown rather than just HTML. ## Changes - Add a shared `markdown-sources` module: a single source of truth for which slugs get a `.md` (guides + troubleshooting), so the generator output and the manifest cannot drift. - Generate markdown for the troubleshooting collection (196 pages, TOML frontmatter parsed via `smol-toml`), written under `public/markdown/guides/troubleshooting/`. - Emit a build-time slug manifest (a gitignored generated `.ts` module, regenerated in `prebuild`, `predev`, and `pretypecheck`, mirroring the existing `__generated__/graphql.ts` lifecycle). - Gate the middleware's `hasMarkdownVariant` on the manifest: serve HTML 200 instead of a 404 for guide paths with no markdown variant. This PR intentionally does not generate markdown for the ai-prompts, YAML config, and externally-fetched (splinter) sources. The HTML fallback covers them now; generating their markdown is follow-up work. ## Testing Local verification (deterministic, against the real manifest and the real negotiation function): - Manifest invariant holds: 744 manifest slugs equal 744 generated `.md` files. - Generator emits 196 troubleshooting files with zero warnings, frontmatter stripped, no leaked delimiters. - Negotiation decision matrix, 6/6: covered slug + bot UA to markdown; uncovered real page + bot UA to pass (HTML 200); nonexistent + bot UA to pass; browser to HTML; covered + `.md` suffix to markdown; uncovered + `.md` suffix to pass. Verified on the Vercel preview deploy: - [x] `User-Agent: ChatGPT-User` on a troubleshooting page returns `200 text/markdown` (real markdown body, frontmatter stripped). - [x] `User-Agent: ChatGPT-User` on an uncovered real page (`ai-tools/ai-prompts/code-format-sql`) returns `200 text/html` (was 404). - [x] Browser request to the same uncovered page returns `200 text/html` (unchanged for humans). - [x] `User-Agent: ChatGPT-User` on a covered standard guide returns `200 text/markdown` (no regression). - [x] `User-Agent: ChatGPT-User` on a nonexistent guide URL returns `404` (correct). Known limitation: an explicit `.md`-suffix request on an uncovered page still 404s by design (an explicit markdown request for a page that has no markdown). The ~90K/day volume is plain-URL UA-based, so it is unaffected. Post-deploy, I will re-run the request-grain 404 reclassification in the GROWTH-915 BQ workspace to confirm fixable guide markdown 404s drop to near zero. ## Linear - fixes GROWTH-946 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added generated markdown slug tracking for docs guides, improving markdown availability detection. * Added automated manifest generation and validation during docs build and CI workflows. * **Bug Fixes** * Improved guide markdown negotiation so only supported guide slugs are treated as having a markdown variant. * Standardized markdown source handling for guides and troubleshooting pages. * **Tests** * Added coverage for guide and troubleshooting slug generation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
Reference Docs
Supabase Reference Docs
Maintainers
If you are a maintainer of any tools in the Supabase ecosystem, you can use this site to provide documentation for the tools & libraries that you maintain.
DocSpec
We use documentation specifications which can be used to generate human-readable docs.
- OpenAPI: for documenting API endpoints.
- SDKSpec (custom to Supabase): for SDKs and client libraries.
- ConfigSpec (custom to Supabase): for configuration options.
- CLISpec (custom to Supabase): for CLI commands and usage.
The benefit of using custom specifications is that we can generate many other types from a strict schema (eg, HTML and manpages). It also means that we can switch to any documentation system we want. On this site we use Next.js, but on Supabase's official website, we use a custom React site and expose only a subset of the available API for each tool.
Contributing
To contribute to docs, see the developers' guide and contributing guide.