* refactor: reading markdown docs files
Refactor how Markdown docs files are read:
- Reuses the same logic across search index generation & page generation
- Improves the indexed content for search:
- Stops removing MDX components, which often contain useful
information like Admonitions
- Denormalizes Partials and CodeSamples for more complete content
This is a prerequisite step for implementing the "Copy docs as Markdown"
functionality.
Only touches regular guides for now, not federated ones.
* fix: tailwind build error (#37728)
We changed to default to ESM imports a while ago, which means local
builds are now breaking because the Tailwind uses a require. Changed to
CJS for Tailwind config file. (I have no idea how this has been working
on Vercel all this time.)
* style: prettier
strictNullChecks was off for docs, which lets errors slip through and
leads to incorrect required/optional typing on Zod-inferred types. This
PR enables strictNullChecks and fixes all the existing violations.
Our environment variables are a bit of a mess, so cleaning them up:
- We have duplicate environment variables that resolve to the same thing
but just have different names, deduplicating all of these
- We have a .env.development (for publicly safe variables) and a
.env.example, we
really only need one
- Privately, we have a .env (for running scripts) and a .env.local (for
Next.js). Changing the dotenv source for scripts, so again we only need
one
Scripts currently use CJS, which is causing a bit of a mess when trying
to use shared utilities from the app. Converting everything to ESM so
there are fewer conflicts when adding new scripts going forward.
* feat: llms.txt
* feat: split llms.txt into multiple files
We have too many docs, so the concatenated text file uses an unreasonable amount of tokens. Chunk it up a little so it's more usable.
Improve the way docs are indexed and FTS results are ranked, so results are more relevant.
Also improve debouncing and searching UX so it feels a bit faster and you can scan the results more easily.
Charis