fix(local): claim WAL before the sidecars race for it - #1781
Conversation
The studio worker died at import on any fresh data directory, so Studio jobs queued with no consumer: the API accepted them, the tab showed processing, and nothing ever ran them. SQLite wants an exclusive lock to switch a file to WAL, and answers SQLITE_BUSY without running the busy handler, so busy_timeout does not cover that one statement. The API and both workers boot in the same tick and each opens huey.db and surfsense.db; on a file nobody had initialised yet, huey's own `pragma journal_mode="wal"` raised inside shared/queue.py at import, which PyInstaller turns into exit 1. Started last, the studio worker was usually the loser. Once a file is in WAL the pragma takes no lock, which is why the next launch always looked fine and why this hid since both files were added on 2026-09-03. shared.sqlite.enable_wal retries the pragma for 5s, matching busy_timeout. queue.py claims WAL on the queue file before constructing the two SqliteHuey instances, and the engine's connect hook now sets busy_timeout ahead of the pragmas that need to wait. Racing the frozen binaries on a fresh data dir, five rounds each: the previous build lost the studio worker once to "database is locked", this one zero times. The unit test pins the mechanism rather than the odds - it holds a write transaction open, asserts SQLite refuses the pragma outright, then asserts enable_wal waits it out and ends in WAL. Also drops the onRetry/retry leftovers from studio-panel.test.tsx. The merge removed both, vitest does not typecheck, so `pnpm dist` was the first thing to notice.
|
@CREDO23 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
The bug
On any fresh data directory the studio worker died at import, so Studio jobs queued with no consumer: the API accepted them, the tab showed processing, and nothing ever ran them.
Cause
SQLite wants an exclusive lock to switch a file to WAL, and answers
SQLITE_BUSYwithout running the busy handler, sobusy_timeoutdoes not cover that one statement. The API and both workers boot in the same tick and each openshuey.dbandsurfsense.db; on a file nobody had initialised yet, huey's ownpragma journal_mode="wal"raised at import, which PyInstaller turns into exit 1. Started last, the studio worker was usually the loser.Once a file is in WAL the pragma takes no lock — which is why the next launch always looked fine, and why this hid since both files were added on 2026-09-03.
Fix
shared/sqlite.py:enable_walretries the pragma for 5s, matchingbusy_timeout.shared/queue.py: claims WAL on the queue file before constructing the twoSqliteHueyinstances, so huey's own pragma only reads the mode back.shared/db.py: the connect hook setsbusy_timeoutahead of the pragmas that need to wait.Verification
Racing the frozen binaries on a fresh data dir, five rounds each: the previous build lost the studio worker once to
database is locked, this build zero times. Afterwards, twelve artifacts generated in one batch (four concurrent) all reachedreadywith no failures.The unit test pins the mechanism rather than the odds: it holds a write transaction open, asserts SQLite refuses the pragma outright, then asserts
enable_walwaits it out and ends in WAL.Also drops the
onRetry/retryleftovers fromstudio-panel.test.tsxthat #1780 left behind — vitest does not typecheck, sopnpm distwas the first thing to notice.High-level PR Summary
This PR fixes a race condition where the studio worker would crash on startup with a
database is lockederror when multiple sidecars attempted to simultaneously switch a fresh SQLite database to WAL mode. The fix introduces a retry mechanism (enable_wal) that waits up to 5 seconds for the exclusive lock needed to enable WAL mode, and pre-claims WAL on queue files before Huey connects. The reordering of pragmas in the database connection hook ensuresbusy_timeoutis set before operations that need to wait for locks.⏱️ Estimated Review Time: 15-30 minutes
💡 Review Order Suggestion
surfsense_local/backend/shared/sqlite.pysurfsense_local/backend/tests/unit/shared/test_sqlite.pysurfsense_local/backend/shared/queue.pysurfsense_local/backend/shared/db.pysurfsense_local/frontend/src/features/studio/studio-panel.test.tsxsurfsense_local/frontend/src/features/studio/studio-panel.test.tsx