Clean up stale cached avatars automatically - #3635
Conversation
|
Hey @faisalahammad 👋 Thanks for your contribution, but I think we should find the source of the issue, to prevent that a second entry is even added, instead of fighting (only) the symptoms. |
|
Thanks for the review. I moved the stale-file cleanup to the moment a new avatar is actually written, so a second copy is never added in the first place. When a remote actor changes their icon URL, the cache used to leave the previous hash file orphaned. The fix overrides Avatar::cache() so any older avatar version is removed in the same call the new one is downloaded. It is avatar-specific because media entity directories legitimately hold multiple distinct images. The hot render path (a maybe_cache() hit) stays a single get_or_cache() call. The daily cron now only drains the backlog already on disk and reclaims orphaned actor directories. A regression test (test_new_avatar_url_prunes_previous_version) proves the old version is removed when a new URL is cached. I pushed the updated branch. Happy to adjust if you would like it done differently. |
pfefferle
left a comment
There was a problem hiding this comment.
Thanks for taking this on @faisalahammad, the cron sweep is a good idea and the test coverage is more than I expected.
I think the write-time half is aimed at the wrong place though.
Avatar::clear_cached_avatar() is already hooked to save_post_ap_actor and calls invalidate_entity(), which deletes the whole actor directory. Remote_Actors::update() goes through wp_update_post(), so an icon change wipes the cache dir today, before the new avatar is downloaded. The new test passes because it calls Avatar::maybe_cache() directly and never updates the actor post.
What I think actually causes the duplicates sits in File::cache(): optimize_image() runs the filename through wp_unique_filename(), so on a name collision you get hash-1.webp, and File::get() globs hash.* and can never find that again. Two requests for the same uncached avatar at the same time both miss get(), both write, and the loser leaves a file that is invisible forever and gets re-downloaded on the next visit. That matches "some profile pics have new versions uploaded daily with the old versions still stored" from #3583 pretty well. Fixing it there would fix Media and Emoji too.
The other thing: #3583 is mostly about actors the site never really interacts with. Remote_Actors keeps every actor it has ever seen, so on the reporter's 750MB site most of those directories still have a live actor post and this cron leaves them alone. There is no age based eviction here. I am not sure we want one, but the changelog entry promises a bit more than the code does.
phpcs is red, composer lint:fix should sort that.
Rest inline.
|
@pfefferle All eight review threads are addressed now — replies are on each thread. Summary of changes:
Full suite green (2923 tests, 21 skipped). Ready for another look when you have a moment. |
|
still phpcs issues! |
Address review feedback to prevent stale avatar copies at the source rather than only sweeping them afterwards. Previously the write-time prune sat on the hot render path inside maybe_cache(), adding a redundant filesystem check to every cache hit. Override Avatar::cache() instead so the prune runs only when a fresh download actually writes a new file. When an actor changes their icon URL, the previous hash file is removed in the same call, so no second copy accumulates. Media and Emoji caches are unaffected since they do not override cache(). maybe_cache() returns to a single get_or_cache() call: cache hits stay one directory read, with zero prune cost. The daily cron still drains the existing backlog on live sites and reclaims orphaned actor directories that write-time pruning cannot reach. Add a regression test proving that caching a new avatar URL removes the previous version while keeping the new one.
- prune: drop older cached avatar versions right after a new one is written, via a File::cache() after_cache() hook so a just-written file is always kept even on name collision - refactor: split get_actor_avatar_hash into a post-taking helper, distinguish missing post (false) from no usable icon (empty string) - test: add collision-keeps-written-file coverage and alignment fixes CI phpcs failed on equals alignment in the copied test file; the write-time prune restructure removes the duplicated append logic that triggered it.
- Drop unused off on off off off off off off off off on off on off on off off off on off off off on on off off off on on off off off off off off off on off off off off on off on off off off off off on off on on off off off on off on on off on on off off on on on off on on off on off off off off on off off off on off off on off on off off off on on off on off on off off on off off off on off off off off off off off off on off on off on on on off on on off off on off on on on off on on off on off on off off off off off on on off off on off off off off off on off off on on off off off off on off off off on off off off off on on off on off off off off off on off on off off off off off off off off off off on on off on off off off parameter from Avatar::after_cache override. - Correct equals-sign alignment flagged by phpcs in avatar cache code and tests. PHPCS warnings resolved. PHP 7.4 compatible. Refs Automattic#3635
6625420 to
8b951f8
Compare
|
Tests and PHPCS still fail. |
Fixes #3583
Proposed changes:
Other information:
New unit tests added for the hash resolver, the file prune, write-time pruning when a new avatar replaces an old one, and the cron cleanup (orphan handling, batching, and the re-entrancy lock). A changelog entry file is included in the branch.
Testing instructions:
wp-content/uploads/activitypub/actors/.wp cron event list | grep activitypub_cleanup_actor_cache(recurrencedaily).wp cron event run activitypub_cleanup_actor_cache --due-now); confirm its directory is removed.Changelog entry
Changelog Entry Details
Significance
Type
Message
Prevented unused copies of remote profile pictures from accumulating on your server and added automatic cleanup for leftover cached avatars.