Skip to content

fix: bound clearJobs rm so FUSE hidden files cannot hang auto-delete - #618

Open
BetterAndBetterII wants to merge 1 commit into
C4illin:mainfrom
BetterAndBetterII:fix/clearjobs-fuse-rmsync-loop
Open

fix: bound clearJobs rm so FUSE hidden files cannot hang auto-delete#618
BetterAndBetterII wants to merge 1 commit into
C4illin:mainfrom
BetterAndBetterII:fix/clearjobs-fuse-rmsync-loop

Conversation

@BetterAndBetterII

@BetterAndBetterII BetterAndBetterII commented Aug 20, 2026

Copy link
Copy Markdown

Summary

clearJobs() called unbounded rmSync({ recursive: true, force: true }). On a FUSE mount, unlinking a file that still has an open download fd recreates .fuse_hidden* entries, so delete never returns and the auto-delete timer is never armed.

This bounds directory deletion, isolates a stuck job so the rest still get removed, and always reschedules cleanup from finally.

Fixes #584

Test plan

  • bun test tests/helpers/rmBounded.test.ts tests/helpers/clearJobs.test.ts (6 pass)

Summary by cubic

Prevents job auto-delete from hanging on FUSE mounts by bounding directory removal and always rearming the cleanup timer. Previously unbounded rmSync({ recursive: true, force: true }) could loop when FUSE recreated .fuse_hidden*; now deletion retries are capped, stuck jobs are isolated, and cleanup is rescheduled reliably.

  • Introduces rmBounded (bounded recursive delete): ignores ENOENT, retries directory removal on ENOTEMPTY up to a limit, then throws to prevent infinite loops.
  • Adds clearExpiredJobs and startJobCleanup: deletes output/uploads per job, removes the DB row only on successful deletes, logs and continues on failure, and reschedules from finally (with injectable rm and schedule for tests).
  • Replaces the inline cleanup in src/index.tsx with startJobCleanup when AUTO_DELETE_EVERY_N_HOURS > 0.
  • Tests cover FUSE behavior, real tree removal, missing-path tolerance, per-job isolation, and rescheduling on errors.

Written for commit 4daf77b. Summary will update on new commits.

Review in cubic

rmSync(recursive) never returns when a leaked download fd leaves
.fuse_hidden entries on FUSE, so the cleanup timer is never rearmed.
Retry-cap directory deletes, skip a stuck job, and always reschedule.
@github-actions github-actions Bot added the Fix label Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

clearJobs() spins forever on FUSE mounts when a leaked fd blocks rmSync, permanently disabling auto-delete

1 participant