perf(files): cache path discovery, filter before resolve - #1180
Merged
kristijanhusak merged 2 commits intoSep 17, 2026
Merged
Conversation
A configured path can be a recursive glob like `~/org/**/*`, which matches every file in the tree, not only the org ones. Path discovery resolved all of them and dropped the non-org ones afterwards. On a tree with 2429 entries of which 1028 are org files, that is 1400 pointless resolve calls, and it happens on every call, worth 16-22ms each time. The filter now runs before the resolve. The extension is read from the name the glob produced, which is also how filetype detection decides what an org file is, so a symlink counts by its own name instead of its target.
Every call that walks the file set runs path discovery again. A recursive glob over a large tree costs 70-100ms per call, and opening one org file triggers three of them, so the same unchanged result is paid for several times. The result is now kept until the next load or unload. A load always rediscovers, so a forced reload still sees new files. The exception is the check whether a newly created file belongs to the configured paths: it asks about a file the cache cannot know about, so it rediscovers on its own. That runs once per created file. One property is traded away. A file deleted from disk outside the editor used to drop out of the file list on the next call, because the stat during rediscovery failed. It now stays listed until the next load. The `skip_resolve` argument goes with it. The unresolved variant existed only to avoid a second resolve during load, and `load_file` resolves the name anyway, so both callers can share one result.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OrgFiles:_files()runs on every call that walks the file set, and each run globs, resolves and stats every match. With a recursive glob like~/org/**/*that costs 70-100 ms per call on my ~1000 file tree. org-roam.nvim produces exactly that glob for every configured directory and hands it to its ownOrgFilesinstance, which is where I measured it. Opening one org file triggers three of them.Two commits.
Filter before resolve. The glob matches every file in the tree, org or not, and discovery resolved all of them before dropping the non-org ones. On my tree that is well over a thousand wasted
vim.fn.resolvecalls per run. Cold discovery roughly halves, from about 140 ms to about 75 ms.Keep the result until the next
load()orunload().load()always rediscovers, so a forced reload still sees new files.load_filewithpersistalso rediscovers, because it asks whether a file that did not exist when the cache was built belongs to the configured paths. That runs once per created file. A warmall()goes from about 100 ms to nothing measurable.Changes
_files(refresh?)replaces_files(skip_resolve?). The unresolved variant only existed to avoid a second resolve during load, andload_fileresolves the name anyway, so both callers now share one cached result. The call inload()still reads_files(true), buttruenow means refresh._path_cacheonOrgFiles, cleared inunload().The trade-off is what happens when a file disappears from disk outside the editor. Before, the next
all()dropped it because the stat failed during rediscovery. Now it stays in the list until the next load, so the agenda keeps showing its entries until you pressr. I think that is acceptable: the parsed content inself.fileswas stale either way, and refreshing the agenda is what you do after changing files outside the editor anyway. But it is a behaviour change.Checklist
I confirm that I have:
Conventional Commits
specification (e.g.,
feat: add new feature,fix: correct bug,docs: update documentation).make test.