rle-harness-opencode

OpenCode as an RLE harness.

RLE benchmarks harnesses × models on a live RimWorld colony. This package lets the OpenCode coding agent be the harness: each tick the environment prompts OpenCode once, OpenCode acts on the colony through the RLE MCP tools (get_brief, work_priority, blueprint, ... end_turn), and the writes that reached the game are scored with the same composite as every other harness.

Install

# OpenCode itself
curl -fsSL https://opencode.ai/install | bash

# RLE core (not on PyPI yet) + this harness
uv pip install "rimworld-learning-environment[mcp] @ git+https://github.com/AppSprout-dev/RLE"
uv pip install git+https://github.com/AppSprout-dev/rle-harness-opencode

Run

From an RLE checkout with RimWorld + RIMAPI running:

python scripts/run_benchmark.py --harness list
python scripts/run_scenario.py crashlanded --harness opencode --model openai/gpt-4o --ticks 10 --tick-interval 30
python scripts/run_benchmark.py --harness opencode --harness felix --model openai/gpt-4o --runs 4

--model is provider/model in OpenCode's naming (e.g. anthropic/claude-sonnet-4-5, openrouter/qwen/qwen3.5-9b); provider credentials are whatever OpenCode already has (opencode auth login).

Options (--harness-opt key=value):

Option	Default	Meaning
binary	opencode	Executable name/path
attach_url	–	Use a running opencode serve instead of spawning one
hostname / port	127.0.0.1 / free	Bind for the spawned server
agent	–	OpenCode agent to use
turn_timeout_s	180	Abort the turn after this many seconds
idle_grace_s	3	Wait for late tool calls after the reply if end_turn was not called
new_session_each_tick	false	Fresh conversation every tick
extra_instructions	–	Appended to every turn prompt

How it works

• Spawns opencode serve with OPENCODE_CONFIG_CONTENT declaring the RLE MCP server (hosted in-process by RLE over streamable HTTP) as a remote MCP server and permission: allow so tool calls never wait for approval.
• Per tick: POST /session/{id}/message with the turn prompt; returns when the agent is done. Token counts from info.tokens feed RLE's cost tracker.
• --smoke-test needs no OpenCode: a scripted agent plays the same MCP round trip.

Development

uv pip install "rimworld-learning-environment[mcp] @ git+https://github.com/AppSprout-dev/RLE"
uv pip install -e ".[dev]"
pytest && ruff check src tests && mypy src

MIT.