Fix yt-dlp stream stalls and unreaped processes - #15
Merged
Conversation
Replace the loose (Popen, kill_stream, classify_stream_error) trio with an AudioStream class that owns the child process end to end. Two problems it fixes: stderr was a pipe nobody read while a track played. classify_stream_error only drained it after playback ended, so a chatty yt-dlp (fragment retries, HTTP warnings, throttling notices) would fill the ~64 KB kernel buffer, block on write, and stop producing audio. FFmpeg then starved and playback stalled with nothing reported to the user. stderr now goes to a temporary file, which never blocks. classify_error() reads only the tail, since the real error is written last and can sit behind megabytes of progress noise. The child was killed but not fully released. Pipes were left for the GC to close and TimeoutExpired was swallowed. close() now kills, reaps, captures the error text while the file is still open, and closes both handles - and is idempotent. The live host had accumulated a yt-dlp zombie that survived 19 days on exactly this path. close() blocks briefly, so the player calls it through an executor rather than from the event loop, in both the playback loop and destroy(). Verified end to end against real yt-dlp: SoundCloud 2.86s to first byte, YouTube 2.42s, 256 KB streamed each, process reaped, pipes closed. The new flood test reproduces the stall directly - 1 MB of stderr, 16x the pipe buffer, which deadlocks under the old code and now delivers audio normally. Closes #6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6
(Replaces #14, which GitHub auto-closed when its base branch was deleted on merge of #13. Same branch, same commit, now rebased onto
main.)The bugs
1. Playback could stall silently
spawn_stream()gave the childstderr=subprocess.PIPE, and nothing read that pipe while the track played —classify_stream_error()only drained it after playback ended.The kernel pipe buffer is ~64 KB. A chatty
yt-dlp(fragment retries, HTTP warnings, throttling notices) fills it, blocks on write, and stops producing audio. FFmpeg starves, playback stops, and nothing is reported to the user.-q --no-warningsreduces the volume of stderr but does not bound it.2. The child was never fully released
kill_stream()sentSIGKILLand waited, but leftproc.stdout/proc.stderrfor the GC to close and swallowedTimeoutExpired.This one was observed in production, not inferred. The live host had a
yt-dlpzombie parented to the bot that had been sitting there for nearly 20 days:The fix
Replaces the loose
(Popen, kill_stream, classify_stream_error)trio with anAudioStreamclass that owns the child end to end.kill(), whole bufferclose()kill+wait(2),TimeoutExpiredswallowedkill+wait(5), a survivor is loggedclose()is idempotentstdoutdeliberately stays a pipe: that backpressure is what keeps memory bounded on a 1 GB host, since yt-dlp blocks as soon as FFmpeg stops reading rather than buffering a whole track in RAM. Only stderr moves to a file.close()blocks briefly (waiting on the child, reading the file), soutils/player.pycalls it throughrun_in_executorin both the playback loop anddestroy()— never straight from the event loop.Verification
Real end-to-end streaming
Ran the real
spawn_streamagainst live sources and confirmed audio flows and the process is reaped:scsearch1:lofi hip hop)YouTube initially returned
HTTP 403on the local box — that turned out to be a five-month-oldyt-dlp(2026.3.3), not this change. Updating to2026.08.19fixed it, which is direct evidence for #11.The regression test that matters
test_a_process_flooding_stderr_still_delivers_audiospawns a child that writes 1 MB to stderr (~16x the pipe buffer) and then writes to stdout. Under the old piped-stderr code the child deadlocks and no audio ever arrives; it now streams normally.test_close_reaps_the_process_leaving_no_zombieassertspoll()returns a code afterclose()— the exact property the 19-day zombie violated.Suite
120 passedin ~0.8s, three consecutive runs, no flakes. Tests useproc.wait()rather than fixed sleeps so they stay deterministic on slow CI.Notes for review
tests/test_media_helpers.pyshrinks: its mock-basedkill_stream/classify_stream_errortests are replaced bytests/test_audio_stream.py, which exercises the same behaviour against real subprocesses. Mocks cannot reproduce a full pipe buffer, which is the whole point..gitignoregains--Frag*/*.part/*.ytdl: streaming to stdout can drop fragment artifacts in the working directory, and one nearly got committed while verifying this.classify_error()is cached byclose(), so the player can ask why a stream failed after teardown without touching the disk again.