Use YouTube player clients that are not bot-checked - #25
Merged
Conversation
Five of every six YouTube tracks failed to load. The error was "sign in to confirm you're not a bot", and it reads to a user as the bot being slow: a track fails, the bot reports it and skips, and they retry until one works. The chain was "default,android_vr,tv_embedded". Testing every client yt-dlp offers against the same video from the production host, all three of those are now bot-checked - so yt-dlp exhausted the chain and gave up. Only two clients still work: web_embedded 6/6 tracks, 3.2s mean mweb 6/6 tracks, 9.3s mean Order is latency, not preference: yt-dlp tries each client in turn, so a blocked client at the front costs a full round trip before anything plays. web_embedded goes first, mweb behind it, and tv_embedded stays last because it needs no JS runtime and is the only option on a host where Deno is missing. The list was also duplicated - once as a comma-joined string for the streaming subprocess, once as a list for metadata extraction - with nothing keeping them in sync. Both now derive from one definition, and a test asserts they agree. Verified against the real bot code on the host: 6/6, 4.09s mean. Also corrects an overstated claim in the previous comments: a residential IP reduces YouTube's bot-checking but does not eliminate it. Cookies remain supported; a correct client chain avoids needing them. Closes #24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24
The problem
Five of every six YouTube tracks failed to load. Measured on the production host:
yt-dlp's real error:
sign in to confirm you're not a bot.This is not experienced as "tracks fail" — it is experienced as the bot being slow. A track fails, the bot reports it and skips, and you retry until one works.
Root cause
The chain was
default,android_vr,tv_embedded. Testing every client yt-dlp offers, against the same video, from the same host:default,web,android_vr,tv_embedded,tv,ios,android_musicmwebweb_embeddedEvery entry in the configured chain was blocked, so yt-dlp exhausted it and gave up.
The fix
Validated across all six videos:
mwebweb_embeddedOrder is latency, not preference. yt-dlp tries each client in turn, so a blocked client at the front costs a full round trip before anything can play.
web_embeddedfirst,mwebbehind it, andtv_embeddedstays last because it needs no JS runtime — making it the only thing that can work on a host where Deno failed to install.The duplication that let this rot
The client list existed twice: as a comma-joined string for the streaming subprocess, and as a list inside
YTDL_OPTIONS["extractor_args"]for metadata extraction. Nothing kept them in sync, so an edit to one could silently change search without changing playback.Both now derive from a single tuple, and
test_the_two_client_usages_cannot_driftasserts it.Verification
Against the real bot code, on the production host:
243 passedlocally. The suite cannot test YouTube itself — CI runs on a datacenter IP where everything is bot-checked — so the new tests pin the configuration: that the two usages agree, that a working client is first, that each client measured as blocked is absent, and that a JS-free fallback remains.A correction this PR carries
The comments previously implied a residential IP defeats YouTube's bot-checking. That claim came from three successful samples and is too strong — six samples show 1/6. A residential IP reduces bot-checking but does not eliminate it. The comments now say so, cookies remain supported, and getting the client chain right avoids needing them.
Not changed, and why
Two other hypotheses were measured and rejected:
performancevspowersavemade no difference (4.76s vs 4.78s). The bottleneck is round trips to YouTube, not clock speed. Left alone.