Skip to content

Clean up stale cached avatars automatically - #3635

Open
faisalahammad wants to merge 4 commits into
Automattic:trunkfrom
faisalahammad:fix/3583-avatar-cache-cleanup
Open

Clean up stale cached avatars automatically#3635
faisalahammad wants to merge 4 commits into
Automattic:trunkfrom
faisalahammad:fix/3583-avatar-cache-cleanup

Conversation

@faisalahammad

@faisalahammad faisalahammad commented Aug 10, 2026

Copy link
Copy Markdown

Fixes #3583

Proposed changes:

  • Stop duplicate avatar copies from being created in the first place: when a remote actor changes their profile picture, the previous cached version is now removed at the moment the new one is downloaded. Remote actors rotate their icon URL often, and each change previously left an orphaned copy behind, which is how the actors folder kept filling up (the root cause of actors folder clogs up directory #3583).
  • Add a daily cron job that scans the avatar cache directory and drains any backlog already on disk, removing orphaned actor directories that no longer match an actor post plus stale avatar files that no longer match an actor's current icon.
  • Batch the cron cleanup with a resume cursor so a large backlog drains over several runs instead of blocking on one.

Other information:

  • Have you written new tests for your changes, if applicable?

New unit tests added for the hash resolver, the file prune, write-time pruning when a new avatar replaces an old one, and the cron cleanup (orphan handling, batching, and the re-entrancy lock). A changelog entry file is included in the branch.

Testing instructions:

  • Enable the plugin and let a remote avatar render (a page with a federated comment or follower) so files land in wp-content/uploads/activitypub/actors/.
  • Confirm the scheduled event exists: wp cron event list | grep activitypub_cleanup_actor_cache (recurrence daily).
  • Change the actor's avatar URL (or simulate a stale file) and load a page that renders the avatar; confirm only the current hash file remains for that actor after the refresh (no duplicate copies accumulate).
  • Delete a remote actor post and rerun the cron event (wp cron event run activitypub_cleanup_actor_cache --due-now); confirm its directory is removed.

Changelog entry

  • Automatically create a changelog entry from the details below.

Changelog Entry Details

Significance

  • Patch
  • Minor
  • Major

Type

  • Added - for new features
  • Changed - for changes in existing functionality
  • Deprecated - for soon-to-be removed features
  • Removed - for now removed features
  • Fixed - for any bug fixes
  • Security - in case of vulnerabilities

Message

Prevented unused copies of remote profile pictures from accumulating on your server and added automatic cleanup for leftover cached avatars.

@pfefferle

Copy link
Copy Markdown
Member

Hey @faisalahammad 👋

Thanks for your contribution, but I think we should find the source of the issue, to prevent that a second entry is even added, instead of fighting (only) the symptoms.

@faisalahammad

Copy link
Copy Markdown
Author

Thanks for the review. I moved the stale-file cleanup to the moment a new avatar is actually written, so a second copy is never added in the first place.

When a remote actor changes their icon URL, the cache used to leave the previous hash file orphaned. The fix overrides Avatar::cache() so any older avatar version is removed in the same call the new one is downloaded. It is avatar-specific because media entity directories legitimately hold multiple distinct images. The hot render path (a maybe_cache() hit) stays a single get_or_cache() call.

The daily cron now only drains the backlog already on disk and reclaims orphaned actor directories. A regression test (test_new_avatar_url_prunes_previous_version) proves the old version is removed when a new URL is cached.

I pushed the updated branch. Happy to adjust if you would like it done differently.

@pfefferle pfefferle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on @faisalahammad, the cron sweep is a good idea and the test coverage is more than I expected.

I think the write-time half is aimed at the wrong place though.

Avatar::clear_cached_avatar() is already hooked to save_post_ap_actor and calls invalidate_entity(), which deletes the whole actor directory. Remote_Actors::update() goes through wp_update_post(), so an icon change wipes the cache dir today, before the new avatar is downloaded. The new test passes because it calls Avatar::maybe_cache() directly and never updates the actor post.

What I think actually causes the duplicates sits in File::cache(): optimize_image() runs the filename through wp_unique_filename(), so on a name collision you get hash-1.webp, and File::get() globs hash.* and can never find that again. Two requests for the same uncached avatar at the same time both miss get(), both write, and the loser leaves a file that is invisible forever and gets re-downloaded on the next visit. That matches "some profile pics have new versions uploaded daily with the old versions still stored" from #3583 pretty well. Fixing it there would fix Media and Emoji too.

The other thing: #3583 is mostly about actors the site never really interacts with. Remote_Actors keeps every actor it has ever seen, so on the reporter's 750MB site most of those directories still have a live actor post and this cron leaves them alone. There is no age based eviction here. I am not sure we want one, but the changelog entry promises a bit more than the code does.

phpcs is red, composer lint:fix should sort that.

Rest inline.

Comment thread includes/cache/class-avatar.php Outdated
Comment thread includes/cache/class-avatar.php Outdated
Comment thread includes/cache/class-avatar.php Outdated
Comment thread includes/cache/class-avatar.php Outdated
Comment thread includes/cache/class-avatar.php Outdated
Comment thread includes/cache/class-avatar.php Outdated
Comment thread tests/phpunit/tests/includes/cache/class-test-avatar.php
Comment thread tests/phpunit/tests/includes/cache/class-test-avatar.php Outdated
@faisalahammad

Copy link
Copy Markdown
Author

@pfefferle All eight review threads are addressed now — replies are on each thread. Summary of changes:

  • Removed the Avatar::cache() override; avatar-specific pruning now runs through a shared File::after_cache() hook that receives the final written filename, so collision-renamed hash-1.webp files survive their own write.
  • cleanup_actors() now uses a race-safe add_option() lock with update_option() stale takeover, DirectoryIterator + cursor instead of whole-tree glob/sort, one batched get_posts( post__in ) query, empty-string hash for actors with no icon (removes all files), and removes empty actor directories.
  • Test fixes: PHPCS alignment, backslash prefixes, and a batch-cursor test that reads actual DirectoryIterator order instead of assuming literal 1/2 directories.

Full suite green (2923 tests, 21 skipped). Ready for another look when you have a moment.

@pfefferle

Copy link
Copy Markdown
Member

still phpcs issues!

@pfefferle pfefferle closed this Aug 27, 2026
@pfefferle pfefferle reopened this Aug 27, 2026
faisalahammad and others added 4 commits August 28, 2026 01:00
Address review feedback to prevent stale avatar copies at the source
rather than only sweeping them afterwards. Previously the write-time
prune sat on the hot render path inside maybe_cache(), adding a redundant
filesystem check to every cache hit.

Override Avatar::cache() instead so the prune runs only when a fresh
download actually writes a new file. When an actor changes their icon
URL, the previous hash file is removed in the same call, so no second
copy accumulates. Media and Emoji caches are unaffected since they do
not override cache().

maybe_cache() returns to a single get_or_cache() call: cache hits stay
one directory read, with zero prune cost. The daily cron still drains
the existing backlog on live sites and reclaims orphaned actor
directories that write-time pruning cannot reach.

Add a regression test proving that caching a new avatar URL removes the
previous version while keeping the new one.
- prune: drop older cached avatar versions right after a new one is written, via a File::cache() after_cache() hook so a just-written file is always kept even on name collision
- refactor: split get_actor_avatar_hash into a post-taking helper, distinguish missing post (false) from no usable icon (empty string)
- test: add collision-keeps-written-file coverage and alignment fixes

CI phpcs failed on equals alignment in the copied test file; the write-time prune restructure removes the duplicated append logic that triggered it.
- Drop unused off on off off off off off off off off on off on off on off off off on off off off on on off off off on on off off off off off off off on off off off off on off on off off off off off on off on on off off off on off on on off on on off off on on on off on on off on off off off off on off off off on off off on off on off off off on on off on off on off off on off off off on off off off off off off off off on off on off on on on off on on off off on off on on on off on on off on off on off off off off off on on off off on off off off off off on off off on on off off off off on off off off on off off off off on on off on off off off off off on off on off off off off off off off off off off on on off on off off off parameter from Avatar::after_cache override.
- Correct equals-sign alignment flagged by phpcs in avatar cache code and tests.

PHPCS warnings resolved. PHP 7.4 compatible.

Refs Automattic#3635
@faisalahammad
faisalahammad force-pushed the fix/3583-avatar-cache-cleanup branch from 6625420 to 8b951f8 Compare August 27, 2026 19:13
@pfefferle

Copy link
Copy Markdown
Member

Tests and PHPCS still fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

actors folder clogs up directory

2 participants