Reproducible/deployable environment for LiteLLM proxy/admin server

.gitattributes

@@ -0,0 +1,2 @@
+# SCM syntax highlighting & preventing 3-way merges
+pixi.lock merge=binary linguist-language=YAML linguist-generated=true -diff

.gitignore

@@ -24,3 +24,8 @@ notifiers/*
 # This lab's own settings: what it runs, its models, its channels.
 lab.yaml
 litellm/config.yaml
+# This installation's own settings: gateways, credentials, what is available here.
+lab.env
+# pixi environments
+.pixi/*
+!.pixi/config.toml

AGENTS.md

@@ -108,6 +108,54 @@ case to stop the agent if it has not met the users goal such as total tasks or c
 wallclock. `docs/settings.md` has every setting. If their files already indicate these,
 use that and tell them what you set. Otherwise ask, and offer the defaults.
 
+Then ask which model should run the agent. If it is an OpenAI GPT model, LiteLLM or
+another gateway that translates the Claude Agent SDK's Anthropic Messages API is
+required, even when the campaign's jobs run locally. If it is another non-Claude model
+behind LiteLLM, ask whether the proxy is already running or should this lab start it.
+LiteLLM is not needed for a local example when the agent uses Claude, unless they
+specifically want to test that integration. Ask about a critic separately: it is optional
+and does not determine whether the main agent uses LiteLLM.
+
+### LiteLLM proxy notes
+
+When LiteLLM fronts an OpenAI-compatible backend for the Claude Agent SDK:
+
+- The SDK sends Anthropic Messages API requests to `/v1/messages`. Set
+  `use_chat_completions_url_for_anthropic_messages: true` so LiteLLM translates them to
+  the backend's Chat Completions API.
+- Keep credentials separate. `LITELLM_MASTER_KEY` authenticates callers to the proxy;
+  `ARGO_API_KEY` (or the backend-specific secret) belongs in the proxy config as the
+  upstream `api_key`. Never commit either secret.
+- Ask which models the user wants campaigns to use. With the repository's current
+  configuration, each model needs a `model_list` entry in `litellm/config.yaml`; the
+  proxy exposes the entry's `model_name` as the name the SDK requests and maps it to the
+  backend `litellm_params.model`. Ask for the exact backend model IDs and choose stable,
+  readable aliases. Add one entry per model, reusing the endpoint and environment-backed
+  credential where appropriate; set the selected alias as `AGENT_MODEL` in the campaign's
+  `run.sh` or environment, and restart the proxy after changing the file.
+- A model not listed in `model_list` is not available through the normal configured proxy
+  route. LiteLLM has pass-through modes, but they are not enabled here because they make
+  model exposure and backend routing less explicit; do not rely on them during setup.
+- A user's `~/.claude/settings.json` can point the SDK directly at another gateway and
+  override the intended route. For a campaign that must use LiteLLM, set
+  `CLAUDE_CONFIG_DIR` in its `run.sh` and provide a campaign-local `claude/settings.json`
+  with the proxy URL and model.
+- Anthropic's `context_management` request field is server-side context-clearing control,
+  not context usage information consumed by the workflow. An OpenAI-compatible backend
+  cannot implement it. Drop only that field with the model's
+  `additional_drop_params: [context_management]`; do not enable global `drop_params`,
+  which can hide other incompatible SDK parameters. `client.get_context_usage()` is a
+  separate CLI query for token reporting and does not require `context_management`.
+- LiteLLM's packaged distribution may include `schema.prisma` without migration files.
+  The local Postgres database must be initialized with `prisma db push --schema=<schema>
+  --skip-generate` after `DATABASE_URL` is exported and before starting the proxy;
+  otherwise dashboard or key-management requests can fail with a missing relation such
+  as `LiteLLM_VerificationToken`.
+- The repository launcher owns this setup: use `pixi run litellm-proxy-start` and
+  `pixi run litellm-proxy-stop`. It uses Postgres port 5433, proxy port 4000, and binds
+  the proxy to `127.0.0.1` by default. Restart the proxy after changing
+  `litellm/config.yaml`.
+
 ### 5. The machine
 
 Ask which system. If `systems/<system>.json` exists you have its module line, proxy,

bin/list_agents.sh

@@ -25,17 +25,23 @@ fi
 RUNS_DIR="${WORKSPACE_DIR:-$LAB_DIR/workspace/*}/runs"
 STALE_AFTER=300     # s without a heartbeat before a run is presumed dead
 
-# meta.json is read once per run and cached: one python start per run, not per field.
-declare -A META_CACHE=()
+# meta.json is read once per run and cached in scalar variables. macOS ships Bash 3.2,
+# which has indexed arrays but not the associative arrays used by newer Bash versions.
+META_LOADED_DIR=""
+META_status=""
+META_stop_reason=""
+META_handle=""
+META_host=""
+META_pid=""
+META_user_prompt_file=""
+META_session_id=""
 
 meta_load() {   # run_dir -> cache every field we print, as one call
     local d="$1"
-    [ -n "${META_CACHE[$d|_loaded]:-}" ] && return
-    local line key val
-    while IFS=$'\t' read -r key val; do
-        META_CACHE["$d|$key"]="$val"
-    done < <(python3 -c '
-import json, sys
+    [ "$META_LOADED_DIR" = "$d" ] && return
+    META_LOADED_DIR="$d"
+    eval "$(python3 -c '
+import json, shlex, sys
 fields = ("status", "stop_reason", "handle", "host", "pid",
           "user_prompt_file", "session_id")
 try:
@@ -44,22 +50,22 @@ try:
 except Exception:
     meta = {}
 for k in fields:
-    print(k, str(meta.get(k, "") or "").replace("\t", " "), sep="\t")
-' "$d/meta.json")
-    META_CACHE["$d|_loaded"]=1
+    print("META_%s=%s" % (k, shlex.quote(str(meta.get(k, "") or "").replace("\\t", " "))))
+' "$d/meta.json")"
 }
 
 meta_get() {
     meta_load "$1"
-    printf '%s' "${META_CACHE[$1|$2]:-}"
+    eval "printf '%s' \"\${META_$2:-}\""
 }
 
+
 describe() {   # run_dir -> running / stopped (reason) / presumed dead
     local d="$1" status hb age now
     meta_load "$d"
-    status="${META_CACHE[$d|status]:-}"
+    status="$META_status"
     if [ "$status" = "stopped" ]; then
-        echo "stopped: ${META_CACHE[$d|stop_reason]:-}"
+        echo "stopped: $META_stop_reason"
         return
     fi
     if [ ! -f "$d/heartbeat" ]; then
@@ -79,7 +85,7 @@ describe() {   # run_dir -> running / stopped (reason) / presumed dead
 is_running() {   # a run is running only if it is beating now
     local d="$1" hb age
     meta_load "$d"
-    [ "${META_CACHE[$d|status]:-}" = "stopped" ] && return 1
+    [ "$META_status" = "stopped" ] && return 1
     [ -f "$d/heartbeat" ] || return 1
     hb="$(cat "$d/heartbeat" 2>/dev/null || echo 0)"
     age=$(( $(date +%s) - hb ))
@@ -128,7 +134,7 @@ fi
 # Said once, and only when a finished run in this listing has a session to reopen.
 for d in "${show[@]}"; do
     meta_load "$d"
-    if [ -n "${META_CACHE[$d|session_id]:-}" ] && ! is_running "$d"; then
+    if [ -n "$META_session_id" ] && ! is_running "$d"; then
         echo "Reopen a finished run's conversation with: claude -r <session>"
         echo
         break
@@ -137,14 +143,14 @@ done
 
 for d in "${show[@]}"; do
     meta_load "$d"
-    handle="${META_CACHE[$d|handle]:-}"
+    handle="$META_handle"
     printf '%s%s\n    host=%s pid=%s prompt=%s\n    %s\n' \
         "$(basename "$d")" \
         "${handle:+  [$handle]}" \
-        "${META_CACHE[$d|host]:-}" "${META_CACHE[$d|pid]:-}" \
-        "${META_CACHE[$d|user_prompt_file]:-}" \
+        "$META_host" "$META_pid" \
+        "$META_user_prompt_file" \
         "$(describe "$d")"
-    sid="${META_CACHE[$d|session_id]:-}"
+    sid="$META_session_id"
     # Only for runs that have ended. A live agent is still writing its conversation,
     # and reading it back is not what you want from a listing of what is running.
     if [ -n "$sid" ] && ! is_running "$d"; then

docs/llm.md

@@ -30,82 +30,81 @@ SDK's, unchanged.
 format and routes it to any provider it supports. Running it in front of an
 OpenAI-style backend lets a campaign run on that backend through the same SDK path.
 
-Install it in its own environment, since its proxy extra pulls in a large dependency
-set:
-
-```
-python -m venv ~/venvs/litellm && ~/venvs/litellm/bin/pip install "litellm[proxy]" "fastapi<0.140.7"
-```
-
-The FastAPI cap is needed as of LiteLLM 1.97.0. LiteLLM imports
-`fastapi.dependencies.utils.get_flat_dependant`, which FastAPI removed in 0.140.7, and
-LiteLLM declares `fastapi>=0.136.3,<1.0` — no upper bound below the removal — so an
-uncapped install takes a FastAPI the proxy cannot import. The working range is 0.136.3
-to 0.140.6. Drop the cap once a LiteLLM release no longer needs it; keep it inside that
-range rather than pinning further back, since below 0.136.3 is outside what LiteLLM
-supports.
-
-Write the config that names the upstream model, its endpoint, and the key to reach it.
-It belongs at the top of the lab, where anyone can see which models are on offer:
+The repository's pixi environment includes LiteLLM, its proxy dependencies, PostgreSQL,
+and Prisma. Create the local config from the template, then start the proxy and its
+local database from the repository root:
 
 ```
 cp litellm/config.yaml.template litellm/config.yaml
+pixi install
+pixi run litellm-proxy-start
 ```
 
-That copy is not tracked by git, since the keys are in it.
+The config is not tracked because it contains credentials. It names the models, endpoints,
+and keys to reach them. The relevant settings for an OpenAI-style upstream are already in
+the template:
 
 ```yaml
-model_list:
-  - model_name: my-model
-    litellm_params:
-      model: openai/<upstream-model-name>
-      api_base: https://backend.example/v1
-      api_key: <key>
-
 litellm_settings:
   use_chat_completions_url_for_anthropic_messages: true
   drop_params: true
 ```
 
-`use_chat_completions_url_for_anthropic_messages` is required for an OpenAI-style
-upstream. Without it LiteLLM translates `/v1/messages` to the OpenAI Responses API,
-and a backend that implements only `/v1/chat/completions` answers 404.
-
-`drop_params` is required for the same reason from the other direction. The agent sends
-parameters an OpenAI backend has no equivalent for -- `context_management` among them --
-and LiteLLM refuses the request rather than dropping them, so the first round fails with
-`UnsupportedParamsError` on a proxy that otherwise works.
+`use_chat_completions_url_for_anthropic_messages` sends the translated request to
+`/v1/chat/completions` rather than the OpenAI Responses API. It is required when the
+backend implements only chat completions. `drop_params` lets LiteLLM discard parameters
+with no OpenAI equivalent, such as `context_management`; without it the first round can
+fail with `UnsupportedParamsError`.
 
-Start the proxy. `bin/lab.sh start` does it from the `litellm` lines in `lab.yaml`, and
-by hand it is:
+The launcher initializes PostgreSQL under `scratch/litellm-postgres`, uses TCP port 5433
+for the database, and serves the proxy on port 4000. The database is local and gitignored.
+It generates LiteLLM's Prisma client when needed and applies the packaged schema before
+startup. Stop the foreground process with Ctrl-C, or stop both services from another
+terminal with:
 
 ```
-~/venvs/litellm/bin/litellm --config litellm/config.yaml --port 4000
+pixi run litellm-proxy-stop
 ```
 
-Check it before pointing the agent at it:
+Set `LITELLM_MASTER_KEY` and `LITELLM_SALT_KEY` before starting when using this beyond
+local development. The defaults are intentionally only suitable for a local machine.
+Set any upstream credential variables referenced by `litellm/config.yaml` before starting.
+Keep those credentials out of Git. Use `LITELLM_PORT`,
+`LITELLM_DB_PORT`, or `LITELLM_PGDATA` to change the proxy port, database port, or
+database directory. The proxy binds to `127.0.0.1` by default; set `LITELLM_HOST`
+deliberately for remote access and use a strong master key.
+
+To run the proxy as a lab service instead, set `litellm: on`, `litellm-bin`, and
+`litellm-config` in `lab.yaml`, then use `bin/lab.sh start`. The lab launcher starts the
+command configured by `litellm-bin`; it does not manage the local PostgreSQL service.
+For the repository launcher, `LITELLM_CONFIG` selects the config file and defaults to
+`litellm/config.yaml`.
+
+Check the proxy before pointing the agent at it. `LITELLM_MASTER_KEY` authenticates the
+Agent SDK to the proxy; the configured `api_key` authenticates LiteLLM to the upstream:
 
 ```
-curl -s -X POST http://0.0.0.0:4000/v1/messages -H 'content-type: application/json' -H 'x-api-key: <key>' -H 'anthropic-version: 2023-06-01' -d '{"model":"my-model","max_tokens":64,"messages":[{"role":"user","content":"say hi"}]}'
+curl -s -X POST http://127.0.0.1:4000/v1/messages -H 'content-type: application/json' -H "x-api-key: $LITELLM_MASTER_KEY" -H 'anthropic-version: 2023-06-01' -d '{"model":"my-model","max_tokens":64,"messages":[{"role":"user","content":"say hi"}]}'
 ```
 
-Then point the settings at the proxy:
+Then configure the SDK process (or its `~/.claude/settings.json`) with the proxy URL,
+model, and the same proxy master key:
 
-```json
-{
-  "env": {"ANTHROPIC_BASE_URL": "http://0.0.0.0:4000", "ANTHROPIC_API_KEY": "<key>"},
-  "model": "my-model"
-}
+```sh
+export ANTHROPIC_BASE_URL=http://127.0.0.1:4000
+export ANTHROPIC_API_KEY="$LITELLM_MASTER_KEY"
+export AGENT_MODEL=my-model
 ```
 
-LiteLLM passes the caller's credential upstream, so `ANTHROPIC_API_KEY` has to be one
-the backend accepts, not an arbitrary string. Where the backend authenticates by
-username, that username is the value.
+Use `127.0.0.1`, not `0.0.0.0`, as a client URL. `ANTHROPIC_API_KEY` should be the
+LiteLLM master key when the proxy uses its configured upstream `api_key`; use caller
+credentials only when the proxy is deliberately configured for that mode.
 
 ## Offering several models from one proxy
 
-One proxy can front several models, so the people running campaigns choose one by name
-and install nothing. Give each an entry:
+One proxy can front several models, so people running campaigns choose one by name and
+install nothing. Add an entry for each model; the `litellm_settings` from the previous
+example applies to all of them:
 
 ```yaml
 model_list:
@@ -119,27 +118,22 @@ model_list:
       model: openai/<gemini-model-name>
       api_base: https://gateway.example/v1
       api_key: os.environ/GATEWAY_KEY
-
-litellm_settings:
-  use_chat_completions_url_for_anthropic_messages: true
-  drop_params: true
 ```
 
 Where a gateway serves several vendors' models on one OpenAI-compatible endpoint, every
-entry uses the `openai/` handler regardless of who made the model — the handler names
-the wire format, not the vendor. Reserve `gemini/` and `anthropic/` for going to those
-vendors directly.
+entry uses the `openai/` handler regardless of who made the model: the handler names the
+wire format, not the vendor. Reserve `gemini/` and `anthropic/` for going to those vendors
+directly.
 
-Each person then names the model they want:
+Each person then names the model they want and authenticates to the proxy:
 
 ```json
-{"env": {"ANTHROPIC_BASE_URL": "http://<proxy-host>:4000", "ANTHROPIC_API_KEY": "<their key>"}, "model": "gemini"}
+{"env": {"ANTHROPIC_BASE_URL": "http://<proxy-host>:4000", "ANTHROPIC_API_KEY": "<proxy key>"}, "model": "gemini"}
 ```
 
-LiteLLM passes the caller's credential upstream rather than substituting the one in the
-config, so each person's own key reaches the backend and usage is attributed to them.
-That also means the proxy should only be reachable from where those credentials are
-already trusted.
+Each model's configured `api_key` is used upstream by default. To attribute upstream
+usage to each caller instead, configure LiteLLM explicitly for caller-provided credentials
+and limit proxy access to the trusted network.
 
 Mixing an Anthropic-native upstream into the same config is untested here.
 `use_chat_completions_url_for_anthropic_messages` applies proxy-wide, so a Claude model

docs/settings.md

@@ -132,6 +132,7 @@ Less often changed.
 | `TASK_DIR` | the campaign directory | where `task.py` is found |
 | `TASK_MODULE` | task | module name within `TASK_DIR` |
 | `CLAUDE_CONFIG_DIR` | `~/.claude` | directory holding the Claude Code `settings.json` that decides which LLM the agent uses. `docs/llm.md` |
+| `AGENT_MODEL` | unset | explicit model name passed to the Agent SDK; normally set to a proxy alias from `litellm/config.yaml`, otherwise Claude Code's configured model applies |
 | `CLAIM_STALE_SECONDS` | 21600 | before an unfinished claim can be taken over |
 | `ANNOUNCE_POLL` | 2 | seconds between announcement-board checks while waiting |
 

framework/agent.py

@@ -749,8 +749,11 @@ def _gateway_settings_file():
         return None
     os.makedirs(RUN_DIR, exist_ok=True)
     path = os.path.join(RUN_DIR, "gateway_settings.json")
+    env = {"ANTHROPIC_BASE_URL": GATEWAY_URL}
+    if os.environ.get("ANTHROPIC_API_KEY"):
+        env["ANTHROPIC_API_KEY"] = os.environ["ANTHROPIC_API_KEY"]
     with open(path, "w") as f:
-        json.dump({"env": {"ANTHROPIC_BASE_URL": GATEWAY_URL}}, f)
+        json.dump({"env": env}, f)
     return path