feat(examples): add a self-modifying harness - #2207
Conversation
|
⚪ agents import sizesMeasured 287 runtime imports as minified bundles. The primary size is gzip; raw minified size is included for diagnosis. An existing import growing by more than 10% is marked red. This report is informational.
Compared No import sizes changed. All 287 current runtime imports
Reported by agent-think[bot]. |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
…lity Replace the HTTP operator API with a WebSocket protocol on the WebSockets capability: an object snapshot on connect, subscribe to replay-then-tail a turn's Streams log, and submit, write_source, activate, and restore commands. Turn and revision changes broadcast to every connection. The client connects with useAgent from agents/react through a useHarnessSession hook and uses the same Kumo chat layout as the other harness examples, with tool cards per turn and an inspector for the active revision's code, revision history with restore, and the journal. The worker routes with routeAgentRequest. Tests drive the test object over RPC instead of the removed HTTP adapter. Drop the dev-time journal column migration and the duplicated file list from the snapshot. Claude-Session: https://claude.ai/code/session_01KEFnjoMnZBMewsuD9qxGrL
There was a problem hiding this comment.
Devin Review found 7 potential issues.
3 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
| } catch (error) { | ||
| outcome = taskOutcome({ ok: false, error: errorMessage(error) }); | ||
| } |
There was a problem hiding this comment.
🟡 Task retries become terminal failures
When step.do schedules a retry, this catch converts its suspension into a completed failure outcome. The task settles immediately, so retries never run.
Prompt for agents
Allow Tasks control-flow exceptions from step.do to propagate so ReplayStep can park or retry the run. Only convert a genuinely terminal editable-harness outcome into TurnTaskOutcome. The current catch also swallows TaskSuspension, defeating the configured retry policy and settling the step with ok:false.
Was this helpful? React with 👍 or 👎 to provide feedback.
| const ENTRY_SOURCE = `import { WorkerEntrypoint } from "cloudflare:workers"; | ||
| import harness from "./src/index"; |
There was a problem hiding this comment.
🟡 Custom-tool validation can be bypassed
If edited src/index.ts omits the virtual registry, compileHarness never evaluates tool modules. Activation accepts invalid tools, while valid tools disappear.
Prompt for agents
Make the immutable generated entrypoint import the generated custom-tools registry unconditionally, then expose its definitions and dispatcher to editable code through a contract that does not depend on src/index.ts retaining a particular import. Activation must evaluate every src/tools/*.ts module even after the editable entrypoint is reorganized, so malformed, duplicate, and System-shadowing tools still reject the candidate.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if (prompt.trim() === "") throw new Error("Turn prompt must not be empty"); | ||
| const active = this.#store.activeBuild(); | ||
| if (!active) throw new Error("Harness genesis has not been activated"); | ||
| const existing = this.#store.turn(turnId); |
There was a problem hiding this comment.
🟡 Concurrent turns corrupt conversation order
#admit accepts turns while earlier ones remain active. Their prompts and replies interleave, so later model history no longer represents a valid conversation.
Prompt for agents
Serialize turn admission at the server, not only in each browser's busy state. Before accepting a new turn, reject it or queue it behind every queued/running turn. Ensure history for a turn is assembled only after all preceding turns have terminal assistant output, including submissions racing from multiple WebSocket connections.
Was this helpful? React with 👍 or 👎 to provide feedback.
| case "write_source": | ||
| await this.#host.writeSource(message.path, message.content); |
| function absoluteHarnessPath(path: string): string { | ||
| const absolute = path.startsWith("/") ? path : `${HARNESS_ROOT}${path}`; | ||
| if ( | ||
| !absolute.startsWith(HARNESS_ROOT) || | ||
| absolute.includes("/../") || | ||
| absolute.endsWith("/..") | ||
| ) { | ||
| throw new HarnessPathError( | ||
| `Harness source path must remain under ${HARNESS_ROOT}: ${JSON.stringify(path)}` | ||
| ); | ||
| } | ||
| if (absolute === HARNESS_ROOT) { | ||
| throw new HarnessPathError("Harness source path must name a file"); | ||
| } | ||
| return absolute; |
| function isClientMessage(value: unknown): value is HarnessClientMessage { | ||
| return ( | ||
| typeof value === "object" && | ||
| value !== null && | ||
| "type" in value && | ||
| typeof value.type === "string" | ||
| ); |
…tor port Deploy as self-modifying-harness-example, give the Vite dev server its own inspector port so it can run beside the other harness examples, and note CLOUDFLARE_ACCOUNT_ID for logins with several accounts.
agents/react resolved a second React copy from the agents package, which broke every hook call in the browser.
…the client A closed stream ends as soon as it is replayed, and every stream end asked for a snapshot that resubscribed to every turn, so the page kept re-rendering and scrolling to the bottom. Subscribe to each turn once per connection, refresh only after a live tail ends, and scroll only when content is added.
Summary
Adds an example-only
SelfModifyingHarnessunderexamples/next/harnesses/self-modifying. No package export, no changeset.The example composes a plain
DurableObjectwith Lifecycle, Tasks, Streams, WebSockets, a durable Shell Workspace, Worker Bundler, and Worker Loader. The editable TypeScript harness is versioned in the Workspace and loaded into a fresh Dynamic Worker for every chat turn.How it works
SelfModifyingHarnessaccepts an AI SDKLanguageModelV4; the example passes theworkers-ai-provider@4model directly.CustomToolexports under/harness/src/tools/and execute inside the Dynamic Worker. Activation discovers them; a Custom tool cannot shadow a System tool.WebSocketscapability. The client connects withuseAgentfromagents/react; auseHarnessSessionhook replays-then-tails each turn's Streams log. The Kumo chat shows tool calls per turn, and an inspector shows the active revision's code, the revision history with a restore action, and the journal.Review
Start with the README, then
src/self-modifying-harness.ts,src/harness-runtime.ts, andsrc/transport.ts.Worker Loader access is required to run or deploy this example. The example's Workers-runtime tests drive a deterministic
LanguageModelV4through the real Durable Object over RPC.