Make deployment reproducible and self-updating

.gitattributes

@@ -0,0 +1,12 @@
+# Line endings.
+#
+# The deploy scripts are checked out and run on Linux. With CRLF endings bash
+# reads the shebang as `/usr/bin/env bash\r` and refuses the file, so these must
+# stay LF regardless of the platform they were committed from. Same for the
+# systemd unit bodies embedded in them and for the CI workflow.
+*.sh        text eol=lf
+*.yml       text eol=lf
+*.yaml      text eol=lf
+
+# Everything else: normalise to LF in the repository, let git decide locally.
+* text=auto

config.py

@@ -3,6 +3,9 @@
 import os
 import sys
 import logging
+import platform
+import subprocess
+from typing import Optional
 
 from dotenv import load_dotenv
 
@@ -38,6 +41,59 @@ def configure_logging() -> None:
     logging.getLogger("discord").setLevel(logging.WARNING)
 
 
+def _first_line(cmd: list[str], cwd: Optional[str] = None) -> Optional[str]:
+    """First line of a command's output, or None if it can't be run."""
+    try:
+        result = subprocess.run(
+            cmd, cwd=cwd, capture_output=True, text=True, timeout=5,
+        )
+    except (OSError, subprocess.SubprocessError):
+        return None
+    if result.returncode != 0:
+        return None
+    lines = result.stdout.strip().splitlines()
+    return lines[0].strip() if lines else None
+
+
+def _ffmpeg_version() -> str:
+    # "ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 ..."
+    line = _first_line(["ffmpeg", "-version"])
+    parts = line.split() if line else []
+    return parts[2] if len(parts) > 2 and parts[1] == "version" else "unknown"
+
+
+def runtime_versions() -> dict:
+    """
+    What this instance is actually running.
+
+    Every lookup degrades to "unknown" rather than failing the boot — a host
+    deployed without git, or without FFmpeg on PATH, should still start and say
+    so plainly.
+    """
+    root = os.path.dirname(os.path.abspath(__file__))
+    return {
+        "commit": _first_line(["git", "rev-parse", "--short", "HEAD"], cwd=root) or "unknown",
+        "yt_dlp": _first_line([sys.executable, "-m", "yt_dlp", "--version"]) or "unknown",
+        "ffmpeg": _ffmpeg_version(),
+        "python": platform.python_version(),
+    }
+
+
+def log_runtime() -> None:
+    """
+    Record the running versions at startup.
+
+    Without this there is no way to tell from `journalctl` which commit or which
+    yt-dlp was live when something broke — and a stale yt-dlp is the single most
+    common cause of YouTube failing.
+    """
+    v = runtime_versions()
+    logging.getLogger("loopify").info(
+        "Running commit %s — yt-dlp %s, FFmpeg %s, Python %s",
+        v["commit"], v["yt_dlp"], v["ffmpeg"], v["python"],
+    )
+
+
 def validate() -> None:
     """Fail fast with a clear message if required config is missing."""
     missing = [name for name, val in {

deploy/README.md

@@ -26,20 +26,22 @@ configured, or follow it step by step.
 
 ## 2. Provision it
 
-Copy the project to the instance (excluding secrets/venv) and run the setup
-script, which installs `ffmpeg` + Python, builds a venv, and installs the
-`loopify-bot` systemd service:
+**Clone** the repo on the instance — do not copy the files over. A git checkout
+is what makes `deploy/update.sh` work later and lets the bot report which commit
+it is running:
 
 ```bash
-# from your laptop, in the repo root
-rsync -av --exclude .venv --exclude .git --exclude __pycache__ \
-      -e "ssh -i <key>.pem" ./ ubuntu@<EC2_IP>:~/LoopifyBot/
-
 ssh -i <key>.pem ubuntu@<EC2_IP>
+
+git clone https://github.com/Isma-L154/LoopifyBot.git ~/LoopifyBot
 cd ~/LoopifyBot
 bash deploy/setup.sh
 ```
 
+`setup.sh` installs `ffmpeg` + Python + Deno, builds a venv, and registers two
+systemd units: the `loopify-bot` service and a daily `loopify-ytdlp-update`
+timer. It is idempotent, so re-running it is safe.
+
 ## 3. Add secrets and start
 
 Secrets are **never** committed. Create the `.env` directly on the instance:
@@ -56,9 +58,63 @@ sudo journalctl -u loopify-bot -f      # live logs — look for "Logged in as ..
 
 ```bash
 ssh -i <key>.pem ubuntu@<EC2_IP>
-cd ~/LoopifyBot && git pull            # or rsync again
-sudo systemctl restart loopify-bot
+cd ~/LoopifyBot && bash deploy/update.sh
+```
+
+`update.sh` pulls, reinstalls dependencies **only if `requirements.txt`
+changed**, restarts the service, and then verifies it actually came back up —
+printing recent logs and failing loudly if it did not.
+
+### If the host was deployed by copying files instead of cloning
+
+`update.sh` refuses to run and tells you how to convert it in place. The short
+version, from the app directory:
+
+```bash
+git init -b main
+git remote add origin https://github.com/Isma-L154/LoopifyBot.git
+git fetch origin
+git branch --set-upstream-to=origin/main main
+git reset --hard origin/main     # discards local edits — check first
+```
+
+The `--set-upstream-to` line matters: without it `update.sh` has nothing to pull
+from and stops with an explanation.
+
+`.env` and `cookies.txt` are gitignored, so they survive this untouched.
+
+### Knowing what is actually running
+
+The bot logs its versions at startup, so `journalctl` answers this directly:
+
 ```
+Running commit 0a90877 — yt-dlp 2026.08.19, FFmpeg 6.1.1-3ubuntu5, Python 3.12.3
+```
+
+```bash
+sudo journalctl -u loopify-bot | grep "Running commit" | tail -1
+```
+
+## Keeping yt-dlp current — automatically
+
+`yt-dlp` is the only dependency deliberately left unpinned. YouTube changes its
+player and extractors constantly, so a stale build starts failing to resolve
+videos within weeks and fails outright within months — a `2026.3.3` build
+returned `HTTP 403` on **every** YouTube URL until it was updated.
+
+`setup.sh` installs a timer that handles this:
+
+```bash
+systemctl list-timers 'loopify-ytdlp-update*'      # when it next runs
+sudo systemctl start loopify-ytdlp-update.service  # force a refresh now
+sudo journalctl -u loopify-ytdlp-update -n 20      # what it did last time
+```
+
+It runs daily with a randomised delay, restarts the bot **only when the version
+actually changed**, and leaves the working version installed if the upgrade
+fails — a newer dependency is never worth trading a running bot for.
+`Persistent=true` means it catches up after downtime rather than silently
+skipping, which matters on a machine that is not on 24/7.
 
 ## 🎬 YouTube from cloud IPs — how it's made to work
 

deploy/setup.sh

@@ -106,11 +106,48 @@ TasksMax=256
 WantedBy=multi-user.target
 UNIT
 
+# ── 5. yt-dlp auto-update timer ───────────────────────────────────────
+# yt-dlp is the one dependency that must NOT be pinned: YouTube changes its
+# player constantly and a stale build stops resolving videos within weeks.
+echo "==> Installing yt-dlp auto-update timer..."
+chmod +x "$APP_DIR/deploy/update-ytdlp.sh"
+
+sudo tee "/etc/systemd/system/${SERVICE_NAME%-bot}-ytdlp-update.service" >/dev/null <<UNIT
+[Unit]
+Description=Refresh yt-dlp for LoopifyBot
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+# Runs as root to restart the service; drops to $APP_USER for the pip install.
+ExecStart=$APP_DIR/deploy/update-ytdlp.sh
+UNIT
+
+sudo tee "/etc/systemd/system/${SERVICE_NAME%-bot}-ytdlp-update.timer" >/dev/null <<UNIT
+[Unit]
+Description=Daily yt-dlp refresh for LoopifyBot
+
+[Timer]
+OnCalendar=daily
+# Spread the load rather than hitting PyPI at midnight with everyone else.
+RandomizedDelaySec=2h
+# Catch up after downtime — important on a machine that isn't on 24/7.
+Persistent=true
+
+[Install]
+WantedBy=timers.target
+UNIT
+
 sudo systemctl daemon-reload
 sudo systemctl enable "$SERVICE_NAME"
+sudo systemctl enable --now "${SERVICE_NAME%-bot}-ytdlp-update.timer"
 
 echo ""
 echo "==> Done. Manage the bot with:"
 echo "    sudo systemctl start   $SERVICE_NAME"
 echo "    sudo systemctl status  $SERVICE_NAME"
 echo "    sudo journalctl -u $SERVICE_NAME -f   # live logs"
+echo ""
+echo "    bash deploy/update.sh                 # pull latest code & restart"
+echo "    systemctl list-timers '*ytdlp*'       # when yt-dlp refreshes next"

deploy/update-ytdlp.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+#
+# Keep yt-dlp current. Run by loopify-ytdlp-update.timer, not by hand.
+#
+# YouTube changes its player and extractors constantly, so a yt-dlp that is a
+# few weeks old starts failing to resolve videos and a few months old fails
+# outright (a 2026.3.3 build returned HTTP 403 on every YouTube URL). Nothing
+# else in the venv is upgraded here — the rest is pinned on purpose.
+#
+# Runs as root so it can restart the service, but drops to the app user for the
+# pip install so the venv does not end up owned by root.
+#
+set -euo pipefail
+
+APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+VENV_DIR="$APP_DIR/.venv"
+SERVICE_NAME="loopify-bot"
+# The venv's owner is the user the bot runs as.
+APP_USER="$(stat -c '%U' "$VENV_DIR")"
+
+version() {
+    runuser -u "$APP_USER" -- "$VENV_DIR/bin/python" -m yt_dlp --version 2>/dev/null || echo "none"
+}
+
+before="$(version)"
+
+# A failed upgrade (offline, PyPI down, a broken release) must leave the working
+# version in place. Never trade a running bot for a newer dependency.
+if ! runuser -u "$APP_USER" -- "$VENV_DIR/bin/pip" install --quiet --upgrade "yt-dlp[default]"; then
+    echo "yt-dlp upgrade failed; keeping $before"
+    exit 0
+fi
+
+after="$(version)"
+
+if [[ "$before" == "$after" ]]; then
+    echo "yt-dlp already current ($after)"
+    exit 0
+fi
+
+# Only restart when something actually changed, so playback is never
+# interrupted for a no-op.
+echo "yt-dlp $before -> $after; restarting $SERVICE_NAME"
+systemctl restart "$SERVICE_NAME"

deploy/update.sh

@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+#
+# Update a deployed bot to the latest committed code and restart it.
+#
+# Usage, on the host:
+#   cd ~/LoopifyBot && bash deploy/update.sh
+#
+set -euo pipefail
+
+APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+VENV_DIR="$APP_DIR/.venv"
+SERVICE_NAME="loopify-bot"
+
+cd "$APP_DIR"
+
+if [[ ! -d .git ]]; then
+    cat >&2 <<'MSG'
+!! This deployment is not a git checkout, so it cannot be updated with git.
+
+   That happens when the code was copied over with rsync. To convert it in
+   place, from the app directory:
+
+     git init -b main
+     git remote add origin https://github.com/Isma-L154/LoopifyBot.git
+     git fetch origin
+     git branch --set-upstream-to=origin/main main
+     git reset --hard origin/main      # discards local edits — check first!
+
+   Your .env and cookies.txt are gitignored and will not be touched.
+MSG
+    exit 1
+fi
+
+if ! git rev-parse --abbrev-ref '@{upstream}' >/dev/null 2>&1; then
+    branch="$(git rev-parse --abbrev-ref HEAD)"
+    cat >&2 <<MSG
+!! Branch '$branch' has no upstream, so there is nothing to pull from.
+
+   This happens on a host converted from a file copy rather than cloned. Set it
+   once:
+
+     git branch --set-upstream-to=origin/main $branch
+
+MSG
+    exit 1
+fi
+
+# Record where we were, so the comparison below survives fast-forwards,
+# no-op pulls and anything that rewrites the reflog.
+previous="$(git rev-parse HEAD)"
+
+echo "==> Fetching..."
+git pull --ff-only
+
+if [[ "$previous" == "$(git rev-parse HEAD)" ]]; then
+    echo "==> Already up to date."
+fi
+
+# Only rebuild the venv when the pinned set actually changed.
+if git diff --quiet "$previous" HEAD -- requirements.txt; then
+    echo "==> requirements.txt unchanged; skipping dependency install."
+else
+    echo "==> requirements.txt changed; reinstalling dependencies..."
+    "$VENV_DIR/bin/pip" install --quiet --upgrade -r requirements.txt
+fi
+
+echo "==> Restarting $SERVICE_NAME..."
+sudo systemctl restart "$SERVICE_NAME"
+
+sleep 2
+if sudo systemctl is-active --quiet "$SERVICE_NAME"; then
+    echo "==> Running commit $(git rev-parse --short HEAD)."
+else
+    echo "!! Service is not active. Recent logs:" >&2
+    sudo journalctl -u "$SERVICE_NAME" -n 30 --no-pager
+    exit 1
+fi

main.py

@@ -9,6 +9,7 @@
 from utils import errors
 
 config.configure_logging()
+config.log_runtime()
 config.validate()
 
 log = logging.getLogger("loopify")

requirements.txt

@@ -1,7 +1,16 @@
-discord.py[voice]>=2.3.0
-yt-dlp[default]>=2024.1.0
-lyricsgenius>=3.0.1
-PyNaCl>=1.5.0
-python-dotenv>=1.0.0
-aiohttp>=3.9.0
-curl_cffi>=0.7.0
+# Pinned so a deploy today installs what was actually tested. Bump these
+# deliberately and re-run the suite; do not let them float.
+discord.py[voice]==2.7.1
+lyricsgenius==3.12.2
+PyNaCl==1.5.0
+python-dotenv==1.2.2
+aiohttp==3.14.3
+curl_cffi==0.15.0
+
+# Deliberately NOT pinned. YouTube changes its player, signature challenge and
+# extractors constantly, so a pinned yt-dlp stops resolving videos within weeks
+# — a five-month-old build (2026.3.3) returned HTTP 403 on every YouTube URL
+# until it was updated. deploy/setup.sh installs a systemd timer that keeps this
+# current on the server; the floor below is simply the oldest build known to
+# work with the current YouTube player.
+yt-dlp[default]>=2026.8.19