yt-dlp on a Server: Fix the Bot Check and Empty Captions

On a server, yt-dlp fails captions for two unrelated reasons. YouTube refuses the datacenter egress IP with a bot check, and yt-dlp cannot present a browser TLS fingerprint without curl_cffi, so caption bodies come back empty. A proxy fixes the first. The pip install line fixes the second.

Two separate gates decide whether a caption request succeeds, and the error messages do not say which one closed.

The first gate is the egress IP. YouTube answers requests from cloud and datacenter ranges with Sign in to confirm you're not a bot, or with HTTP Error 429, before it evaluates the request itself. The application code that consumes yt-dlp treats both strings as one condition: prepublish-be/internal/infrastructure/youtube/extractor.go returns a typed ErrCaptionsBlocked when stderr contains Sign in to confirm or HTTP Error 429, because a refusal at the network level is a problem on the caller's side and never the channel's.

The second gate is the TLS fingerprint of the connection. YouTube's caption endpoint, timedtext, answers HTTP 429 to a client whose TLS handshake does not look like a browser. yt-dlp can present a browser fingerprint, but only when the curl_cffi dependency is installed beside it.

The two faults get conflated because they can produce the same status code and the same absence of captions. They need opposite fixes. A proxy changes the address the request leaves from and does nothing about the handshake. curl_cffi changes the handshake and does nothing about the address. Run a build with neither and the error does not say which one is missing, which is why the failure states come before any change.

Each row below was produced by the same probe against a public video. The third column is what the result means, rather than what the message says.

The second and third states share a result word because the response is the same shape: replace the build. The first state never means the video lacks captions, and updating yt-dlp cannot clear it.

All four rows were measured on the production host on 19 September 2026, before the fix.

Row three is the one worth reading twice. That container had a proxy, so the address was not the problem, and it still returned zero caption characters. It was also running the same yt-dlp version as the API container that returned 18,143 characters on the same video. The only difference was the missing curl_cffi. The proxy was a real fix for a different failure, and it did not touch this one.

curl_cffi is a Python binding for curl-impersonate. The yt-dlp README lists it under networking dependencies as providing impersonation targets for Chrome, Edge, and Safari, and notes that it may be required for sites that employ TLS fingerprinting (yt-dlp README, retrieved 19 September 2026). The timedtext caption endpoint is one of those sites. The comment in extractor.go states the same constraint from the application side: the caption request is one that yt-dlp, with curl_cffi available, is the only practical tool to make.

The standalone builds are the trap. The same README lists curl_cffi as included in most builds except the Unix zipimport binary and the Windows 32-bit build. The Dockerfile in this repository says why that matters, and the comment is the clearest statement of the failure:

#   - curl-cffi gives yt-dlp browser TLS impersonation. YouTube's timedtext
#     (caption) endpoint answers HTTP 429 to non-browser fingerprints, so
#     WITHOUT curl_cffi every caption download fails. The zip-app build has
#     no way to bundle it; pip pulls the musllinux wheel.

The install line that carries it, from the same file:

pip install --no-cache-dir --break-system-packages -U ${PIP_CHANNEL_FLAG} "yt-dlp[default,curl-cffi]"

${PIP_CHANNEL_FLAG} is --pre when the image tracks nightlies, which is how pip selects a pre-release rather than the stable build. The layer ends with python3 -c "import curl_cffi", so a missing wheel fails the image build instead of failing a caption pull weeks later. On a running container, the equivalent check is docker exec <container> python3 -c "import curl_cffi": silence means the module imports, and a ModuleNotFoundError means the build cannot impersonate.

For a blocked result, no. The new binary leaves from the same address and meets the same gate.

Updating addresses the third failure state, where the binary is old enough that YouTube has moved. The cadence read from the PyPI release index on 18 September 2026 makes the window concrete.

yt-dlp's README describes the stable channel as often stale and prone to external breakage, meaning sites change on their end and break it, and calls nightly the recommended channel for regular users (retrieved 19 September 2026). A monthly stable pin therefore guarantees a window, up to a month wide, in which YouTube has changed something and the extractor has not.

One formatting trap sits here. PyPI names a nightly 2026.9.16.232951.dev0 while yt-dlp --version prints 2026.09.16.232951. A rollback that pins the printed string matches nothing on PyPI, so the rollback target has to be the version that pip show yt-dlp recorded.

The pattern is a loop around the install, not an install on its own. The updater in prepublish-be/scripts/ytdlp-update.sh documents its own contract:

#   1. skip it when it is not running (a stopped container is not a failure);
#   2. read the current version and run the canary for a baseline;
#   3. pip install --no-cache-dir --break-system-packages -U [--pre]
#      "yt-dlp[default,curl-cffi]" as root inside the container;
#   4. run the canary again. ok: done. broken: roll back to the recorded version,
#      re-verify and alert. blocked: leave the build alone (the egress IP is the
#      problem, not the binary) and alert.

Three properties make that loop safe to run unattended. The baseline canary distinguishes a build this update broke from an environment that was already failing. The rollback lands on the version pip recorded, so the container returns to the exact build it had. And a blocked result leaves the binary alone, because replacing a working extractor does not change the address.

Timing follows the upstream cadence. yt-dlp publishes nightlies in the small hours UTC, so the systemd timer that drives this runs at 03:30 UTC with Persistent=true and RandomizedDelaySec=900 (ops/systemd/prepublish-ytdlp-update.timer). A host that was powered off at 03:30 catches up on the next boot rather than skipping the day.

Two checks are enough, and each one proves something the other cannot.

Both calls use the flags production uses, so the probe and the real fetch cannot drift: --extractor-args youtube:player_client=tv_simply,web_safari,mweb, --js-runtimes node, and --ignore-no-formats-error. The extractor argument list matters because YouTube changes which player clients dodge the bot check, and a canary using a different list would report a healthy tool that the application cannot use.

The output contract is one JSON line on stdout and nothing else, with the human-readable copy on stderr. A real passing run in the API container on 18 September 2026:

{"result":"ok","version":"2026.08.19","title_ok":true,"caption_chars":18143,"detail":"metadata and captions ok"}

Exit codes carry the same three outcomes: 0 for ok, 2 for blocked, 1 for broken. Wire the exit code to your alerting rather than the JSON, because the exit code is what a shell, a systemd unit, or a CI step reads without parsing. Alert on the second consecutive failure rather than the first, so a single slow call does not page anyone, and deduplicate repeated alerts so a container stuck in one state does not page every day. Running the same probe on a separate schedule, independent of updates, is what catches YouTube breaking extraction while yt-dlp is current.

One ordering detail inside the canary is worth copying. It checks yt-dlp's no impersonate target is available warning before it checks for a block pattern, because a 429 from a build without curl_cffi looks exactly like a 429 from a blocked address. Reporting that case as blocked would send the reader to look at the network when the fix is one line in the image.

None of this is documented behaviour. The timedtext endpoint has no public contract. The player-client list that dodges the bot check changes on YouTube's schedule rather than yours, and the extractor argument value in the repository carries a comment saying to try other client names when extraction starts failing. Any of it can change without notice, which is why the canary is not optional equipment.

A few costs sit outside the code. Residential proxies cost money, and a datacenter proxy mostly moves one refused address to another. Cookies files expire, so a working setup fails later without anything changing in the deployment. A canary video can pass while a specific video fails, because that video's track may be absent, restricted, or in a language the request did not name. Caption text also varies with YouTube's own revisions, so a caption character count is a floor to compare against, not a fixed number.

The version string is a smaller trap with a long reach. Anything that parses versions has to normalize between PyPI's 2026.9.16.232951.dev0 and the printed 2026.09.16.232951, and a stable-looking 2026.08.19 is still a nightly if that is what was installed. Comparison code that reads the missing .dev0 as proof of stability will make the wrong call about which channel a container is on.

If running this path is not the point, the transcript endpoint runs the yt-dlp route server-side and keeps the result in a 30-day cache, so a cold pull becomes a stored row. The MCP server exposes the same extraction as a tool for an assistant. Prepublish runs the path described here in the open, which is why the measurements in this article have dates.

Everything above concerns caption tracks that a creator published on a public video and that the player serves to any signed-out viewer. None of the measurements came from private, members-only, or otherwise restricted content.

If a track is not served to a signed-out browser, there is no player request to observe and nothing to replay. Reaching one would mean working around an access control, which is a different problem from the one this article describes.