Skip to content

perf(files): cache path discovery, filter before resolve - #1180

Merged
kristijanhusak merged 2 commits into
nvim-orgmode:masterfrom
seflue:perf/files-path-discovery
Sep 17, 2026
Merged

kristijanhusak merged 2 commits into
nvim-orgmode:masterfrom
seflue:perf/files-path-discovery

Conversation

@seflue

@seflue seflue commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

OrgFiles:_files() runs on every call that walks the file set, and each run globs, resolves and stats every match. With a recursive glob like ~/org/**/* that costs 70-100 ms per call on my ~1000 file tree. org-roam.nvim produces exactly that glob for every configured directory and hands it to its own OrgFiles instance, which is where I measured it. Opening one org file triggers three of them.

Two commits.

  1. Filter before resolve. The glob matches every file in the tree, org or not, and discovery resolved all of them before dropping the non-org ones. On my tree that is well over a thousand wasted vim.fn.resolve calls per run. Cold discovery roughly halves, from about 140 ms to about 75 ms.

  2. Keep the result until the next load() or unload(). load() always rediscovers, so a forced reload still sees new files. load_file with persist also rediscovers, because it asks whether a file that did not exist when the cache was built belongs to the configured paths. That runs once per created file. A warm all() goes from about 100 ms to nothing measurable.

Changes

  • _files(refresh?) replaces _files(skip_resolve?). The unresolved variant only existed to avoid a second resolve during load, and load_file resolves the name anyway, so both callers now share one cached result. The call in load() still reads _files(true), but true now means refresh.
  • _path_cache on OrgFiles, cleared in unload().

The trade-off is what happens when a file disappears from disk outside the editor. Before, the next all() dropped it because the stat failed during rediscovery. Now it stays in the list until the next load, so the agenda keeps showing its entries until you press r. I think that is acceptable: the parsed content in self.files was stale either way, and refreshing the agenda is what you do after changing files outside the editor anyway. But it is a behaviour change.

Checklist

I confirm that I have:

  • Followed the
    Conventional Commits
    specification
    (e.g., feat: add new feature, fix: correct bug,
    docs: update documentation).
  • My PR title also follows the conventional commits specification.
  • Updated relevant documentation, if necessary.
  • Thoroughly tested my changes.
  • Added tests (if applicable) and verified existing tests pass with
    make test.
  • Checked for breaking changes and documented them, if any.

A configured path can be a recursive glob like `~/org/**/*`, which
matches every file in the tree, not only the org ones. Path discovery
resolved all of them and dropped the non-org ones afterwards. On a
tree with 2429 entries of which 1028 are org files, that is 1400
pointless resolve calls, and it happens on every call, worth 16-22ms
each time.

The filter now runs before the resolve. The extension is read from
the name the glob produced, which is also how filetype detection
decides what an org file is, so a symlink counts by its own name
instead of its target.
Every call that walks the file set runs path discovery again. A
recursive glob over a large tree costs 70-100ms per call, and opening
one org file triggers three of them, so the same unchanged result is
paid for several times.

The result is now kept until the next load or unload. A load always
rediscovers, so a forced reload still sees new files. The exception
is the check whether a newly created file belongs to the configured
paths: it asks about a file the cache cannot know about, so it
rediscovers on its own. That runs once per created file.

One property is traded away. A file deleted from disk outside the
editor used to drop out of the file list on the next call, because
the stat during rediscovery failed. It now stays listed until the
next load.

The `skip_resolve` argument goes with it. The unresolved variant
existed only to avoid a second resolve during load, and `load_file`
resolves the name anyway, so both callers can share one result.

@kristijanhusak kristijanhusak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@kristijanhusak
kristijanhusak merged commit a09ed25 into nvim-orgmode:master Sep 17, 2026
32 of 33 checks passed
@seflue
seflue deleted the perf/files-path-discovery branch September 17, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants