Problem:
Checking the extension of a file is done often, e.g. in Nvim's codebase
for differentiating Lua and Vimscript files in the runtime. The current
way to do this in Lua is (1) a Lua pattern match, which has pitfalls
such as not considering filenames starting with a dot, or (2)
fnamemodify() which is both hard to discover and hard to use / read if
not very familiar with the possible modifiers.
vim.fs.ext() returns the file extension including the leading dot of
the extension. Similar to the "file extension" implementation of many
other stdlibs (including fnamemodify(file, ":e")), a leading dot
doesn't indicate the start of the extension. E.g.: the .git folder in a
repository doesn't have the extension .git, but it simply has no
extension, similar to a folder named git or any other filename without
dot(s).
refactor(lua): add integer coercion helpers
Add vim._tointeger() and vim._ensure_integer(), including optional base
support, and switch integer-only tonumber()/assert call sites in the Lua
runtime to use them.
This also cleans up related integer parsing in LSP, health, loader, URI,
tohtml, and Treesitter code.
supported by AI
Problem:
vim.fs.joinpath treats empty string as a path segment
(it adds a path separator for each empty item):
print(vim.fs.joinpath('', 'after/lsp', '')) -- '/after/lsp/'
print(vim.fs.joinpath('', '')) -- '/'
Especially problematic if the empty segment is the first segment, as
that converts the path to an absolute path.
Solution:
Ignore empty (length of 0) path segments.
Benchmark:
local function test(func)
local t = vim.uv.hrtime()
for _ = 1, 100000, 1 do
func('', 'this/is', 'a/very/long/path', '', 'it', 'really', 'is')
end
print(math.floor((vim.uv.hrtime() - t) / 1e6), 'ms')
end
- with Iter():filter() --> 370 ms
- building new segments table --> 208 ms
- with vim.tbl_filter --> 232 ms
- Instead of gsub split on `/` in all parts --> 1870 ms
Problem: vim.fs.root uses vim.fn.fnamemodify. vim.fn table isn't
available from nvim -ll or thread contexts.
Solution: Swap out vim.fn.fnamemodify for vim.fs.abspath.
This is a temporary workaround and may be reverted since the
long-term plan is to use more fast=true "fn" functions from vim.fs
where possible.
Problem: There is currently no way to check if a given path is absolute or convert a relative path to an absolute path through the Lua stdlib. `vim.fs.joinpath` does not work when the path is absolute. There is also currently no way to resolve `C:foo\bar` style paths in Windows.
Solution: Add `vim.fs.abspath`, which allows converting any path to an absolute path. This also allows checking if current path is absolute by doing `vim.fs.abspath(path) == path`. It also has support for `C:foo\bar` style paths in Windows.
Problem:
`vim.validate()` takes two forms when it only needs one.
Solution:
- Teach the fast form all the features of the spec form.
- Deprecate the spec form.
- General optimizations for both forms.
- Add a `message` argument which can be used alongside or in place
of the `optional` argument.
Problem:
`vim.fs.dirname([[C:\User\XXX\AppData\Local]])` returns "." on
mingw/msys2.
Solution:
- Check for "mingw" when deciding `iswin`.
- Use `has("win32")` where possible, it works in "fast" contexts since
b02eeb6a72.
Problem:
vim.fs.normalize() normalizes too much vim.loader and is slow.
Solution:
Make it faster by doing less. This reduces the times spent in
vim.fs.normalize in vim.loader from ~13ms -> 1-2ms.
Numbers from a relative benchmark:
- Skipping `vim.validate()`: 285ms -> 230ms
- Skipping `path_resolve_dot()`: 285ms -> 60ms
- Skipping `double_slash`: 60ms -> 35ms
Re-normalizing a path after a `joinpath` isn't necessary. Calling
`normalize` on each child directory had quite a bit of impact when
traversing a large directory.
A simple test showed:
Before: ~144ms
After: ~80ms
running the following logic against a dir with 4367 child folders and
25826 files:
local files = {}
local start = uv.hrtime()
for name, type in vim.fs.dir(path, { depth = max_depth }) do
table.insert(files, { name, type })
end
local duration = uv.hrtime() - start
Relates to https://github.com/neovim/neovim/issues/23291
vim.fs.root() is a function for finding a project root relative to a
buffer using one or more "root markers". This is useful for LSP and
could be useful for other "projects" designs, as well as for any plugins
which work with a "projects" concept.
Backslashes are valid characters in unix style paths.
Fix the conversion of backslashes to forward slashes in several `vim.fs`
functions when not on Windows. On Windows, backslashes will still be converted
to forward slashes.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Justin M. Keyes