-
Notifications
You must be signed in to change notification settings - Fork 10
feat: add local Stagehand code-mode tool #91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
c314f9d
feat: add local Stagehand code-mode tool
shrey150 a5503f7
style: format code-mode package
shrey150 9123ecd
refactor: simplify local Stagehand code-mode tool
shrey150 e9edc2c
docs: use conventional skill filename
shrey150 e4b085d
refactor: make stdio server an internal entrypoint
shrey150 14d6268
docs: document Stagehand code mode integration
shrey150 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| name: Stagehand code-mode tool | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - '.github/workflows/stagehand-codemode.yml' | ||
| - 'packages/stagehand-codemode/**' | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - '.github/workflows/stagehand-codemode.yml' | ||
| - 'packages/stagehand-codemode/**' | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| core: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: pnpm/action-setup@v4 | ||
| with: | ||
| version: 10.9.0 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 24.x | ||
|
|
||
| - run: pnpm install --no-frozen-lockfile | ||
| - run: pnpm --filter @browserbasehq/stagehand-codemode run typecheck | ||
| - run: pnpm --filter @browserbasehq/stagehand-codemode run build |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| # Stagehand Code Mode | ||
|
|
||
| Stagehand Code Mode gives an agent one `code_execute` tool for operating a Browserbase browser with | ||
| Stagehand V4 JavaScript. It is framework-neutral: agent frameworks can launch the included local | ||
| MCP server over stdio or wrap `StagehandCodeExecutor` in a native tool binding. | ||
|
|
||
| The executor creates a Browserbase browser on the first valid call, serializes calls, and reuses the | ||
| same browser until its owner closes it. This lets an agent complete multi-step tasks across tool calls | ||
| without managing a browser or session identifier. | ||
|
|
||
| ## Tool contract | ||
|
|
||
| `code_execute` accepts the body of an async JavaScript function: | ||
|
|
||
| ```ts | ||
| type CodeExecuteInput = { | ||
| code: string; | ||
| }; | ||
| ``` | ||
|
|
||
| The generated function receives these objects: | ||
|
|
||
| - `page`: the active Stagehand V4 `Page`; | ||
| - `context`: the shared Stagehand V4 `BrowserContext`; | ||
| - `stagehand`: the Stagehand V4 `act`, `observe`, and `extract` methods; | ||
| - `z`: Zod V4 for structured extraction schemas; and | ||
| - `console`: captured `log`, `warn`, and `error` methods. | ||
|
|
||
| For example: | ||
|
|
||
| ```js | ||
| await page.goto('https://example.com', { waitUntil: 'load' }); | ||
| return { | ||
| title: await page.title(), | ||
| url: await page.url(), | ||
| }; | ||
| ``` | ||
|
|
||
| Calls return a JSON-safe result containing the active page state, the generated function's return | ||
| value, and any captured logs: | ||
|
|
||
| ```ts | ||
| type CodeExecuteResult = | ||
| | { | ||
| ok: true; | ||
| page: { url: string; title: string }; | ||
| value?: unknown; | ||
| logs?: Array<{ level: 'log' | 'warn' | 'error'; text: string }>; | ||
| } | ||
| | { | ||
| ok: false; | ||
| page?: { url: string; title: string }; | ||
| logs?: Array<{ level: 'log' | 'warn' | 'error'; text: string }>; | ||
| error: { | ||
| kind: 'validation' | 'runtime' | 'aborted' | 'closed'; | ||
| name: string; | ||
| message: string; | ||
| }; | ||
| }; | ||
| ``` | ||
|
|
||
| ## Local MCP integration | ||
|
|
||
| Frameworks with local-process MCP support should launch the built stdio server and keep that process | ||
| alive for the complete agent run: | ||
|
|
||
| ```text | ||
| node packages/stagehand-codemode/dist/stdio-server.js | ||
| ``` | ||
|
|
||
| The stdio server is an internal process entrypoint, not a user-facing CLI. The package does not | ||
| publish a `bin` command or accept command-line arguments. | ||
|
|
||
| The framework owns the process lifecycle: | ||
|
|
||
| 1. Launch one stdio server for the agent run. | ||
| 2. Reuse it for every `code_execute` call that should share browser state. | ||
| 3. Terminate and relaunch it if a call stops responding. | ||
| 4. Close it when the agent run finishes. | ||
|
|
||
| Restarting the process creates a new browser, so browser state from the previous process is not | ||
| preserved. | ||
|
|
||
| ## Native tool integration | ||
|
|
||
| Frameworks that do not launch local MCP servers can wrap the executor directly: | ||
|
|
||
| ```ts | ||
| import { | ||
| StagehandCodeExecutor, | ||
| stagehandCodeConfigFromEnv, | ||
| } from '@browserbasehq/stagehand-codemode'; | ||
|
|
||
| const executor = new StagehandCodeExecutor(stagehandCodeConfigFromEnv()); | ||
|
|
||
| try { | ||
| const result = await executor.execute({ | ||
| code: ` | ||
| await page.goto("https://example.com", { waitUntil: "load" }); | ||
| return { title: await page.title() }; | ||
| `, | ||
| }); | ||
| console.log(result); | ||
| } finally { | ||
| await executor.close(); | ||
| } | ||
| ``` | ||
|
|
||
| Create one executor per agent run and close it in `finally` so the Browserbase browser is released | ||
| when the run succeeds, fails, or is cancelled. | ||
|
|
||
| ## Configuration | ||
|
|
||
| Stagehand Code Mode reads configuration from the owning framework's environment: | ||
|
|
||
| | Variable | Required | Purpose | | ||
| | --------------------------------------------------------------------- | ----------------------------------------- | --------------------------------------------------------------------------- | | ||
| | `BROWSERBASE_API_KEY` | Yes, before the first `code_execute` call | Creates the Browserbase browser | | ||
| | `STAGEHAND_MODEL_NAME` | Only for Stagehand AI methods | Provider and model name used by `act`, `observe`, and `extract` | | ||
| | `STAGEHAND_MODEL_API_KEY` | Provider-dependent | Explicit model-provider API key | | ||
| | `STAGEHAND_MODEL_BASE_URL` | No | Custom model-provider base URL | | ||
| | `GEMINI_API_KEY`, `GOOGLE_API_KEY`, or `GOOGLE_GENERATIVE_AI_API_KEY` | No | Selects `google/gemini-2.5-flash-lite` when no explicit model is configured | | ||
|
|
||
| The consuming application must also make a compatible `@browserbasehq/stagehand` V4 package | ||
| available to the executor. | ||
|
|
||
| ## Model syntax guide | ||
|
|
||
| [`SKILL.md`](./SKILL.md) is the canonical Stagehand V4 syntax guide. The MCP server includes the | ||
| complete guide in the `code_execute` tool description. Native integrations should also add the | ||
| exported `STAGEHAND_CODEMODE_SKILL` string to the agent's system instructions or equivalent | ||
| high-priority context. | ||
|
|
||
| The guide covers deterministic page and locator methods, `act`, `observe`, `extract`, Zod schemas, | ||
| multiple pages, cross-call state, and return-value discipline. | ||
|
|
||
| ## Lifecycle and limits | ||
|
|
||
| - Browser creation is lazy; MCP discovery does not create a Browserbase browser. | ||
| - Calls are serialized because they operate on one shared browser context. | ||
| - Pages, cookies, and navigation state persist across successful calls in the same process. | ||
| - JavaScript variables declared inside generated code do not persist between calls. | ||
| - Input code is limited to 100,000 UTF-8 bytes. | ||
| - Captured logs are limited to 64 KiB. | ||
| - Returned values are limited to 256 KiB and are truncated with metadata when necessary. | ||
| - BigInt and byte-array values are converted into JSON-safe representations. | ||
|
|
||
| ## Security | ||
|
|
||
| Generated JavaScript runs directly in the MCP or native-tool process. Stagehand Code Mode is not a | ||
| security sandbox: generated code can access the filesystem, network, environment variables, Node | ||
| globals, and in-process SDK state available to that process. | ||
|
|
||
| Use Stagehand Code Mode only with trusted agents in a trusted execution environment. Applications | ||
| that execute untrusted code must provide a real isolation boundary, such as a restricted container, | ||
| virtual machine, or purpose-built code sandbox. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| # Stagehand V4 code-mode syntax skill | ||
|
|
||
| You have one `code_execute` tool. Its `code` argument is the body of an async JavaScript function, | ||
| not a complete program. Write direct `await` statements and finish with a JSON-serializable return | ||
| value. | ||
|
|
||
| The following objects are already in scope: | ||
|
|
||
| - `page`: the active Stagehand `Page`. | ||
| - `context`: the Stagehand `BrowserContext` shared across calls. | ||
| - `stagehand`: the Stagehand AI methods `act`, `observe`, and `extract`. | ||
| - `z`: Zod V4, for `stagehand.extract` schemas. | ||
| - `console`: captured `log`, `warn`, and `error` methods. | ||
|
|
||
| Do not import packages, read environment variables, construct Stagehand, call `stagehand.init()`, or | ||
| close the page/browser. The tool process owns initialization and cleanup. | ||
|
|
||
| ## Direct browser syntax | ||
|
|
||
| Use deterministic page and locator methods when you know the target: | ||
|
|
||
| ```js | ||
| await page.goto('https://example.com', { waitUntil: 'load' }); | ||
| const heading = await page.locator('h1').innerText(); | ||
| const visible = await page.locator('a').first().isVisible(); | ||
| return { heading, visible, url: await page.url(), title: await page.title() }; | ||
| ``` | ||
|
|
||
| Common page methods include `goto`, `reload`, `goBack`, `goForward`, `click`, `hover`, `scroll`, | ||
| `dragAndDrop`, `type`, `keyPress`, `evaluate`, `waitForLoadState`, `waitForTimeout`, | ||
| `waitForSelector`, `screenshot`, `snapshot`, `url`, `title`, and `locator`. | ||
|
|
||
| Common locator methods include `click`, `hover`, `fill`, `count`, `isChecked`, `inputValue`, | ||
| `isVisible`, `innerText`, `innerHtml`, `textContent`, `scrollTo`, `type`, `selectOption`, `first`, | ||
| and `nth`. | ||
|
|
||
| ## Stagehand AI syntax | ||
|
|
||
| Use `act` for an interaction described in natural language: | ||
|
|
||
| ```js | ||
| const result = await stagehand.act('Click the sign-in button'); | ||
| if (!result.success) throw new Error(result.message); | ||
| return result; | ||
| ``` | ||
|
|
||
| Use `observe` to find candidate actions without performing them: | ||
|
|
||
| ```js | ||
| const actions = await stagehand.observe('Find the checkout button'); | ||
| return { actions }; | ||
| ``` | ||
|
|
||
| Use `extract` with a Zod schema for structured page data: | ||
|
|
||
| ```js | ||
| const product = await stagehand.extract( | ||
| 'Extract the product name and price', | ||
| z.object({ name: z.string(), price: z.string() }) | ||
| ); | ||
| return product; | ||
| ``` | ||
|
|
||
| Pass `{ page: anotherPage }` as the final options object to `act`, `observe`, or `extract` when the | ||
| active page is not the intended target. | ||
|
|
||
| ## Pages and state across calls | ||
|
|
||
| ```js | ||
| const pages = await context.pages(); | ||
| const secondPage = pages[1] ?? (await context.newPage()); | ||
| await context.setActivePage(secondPage); | ||
| return { | ||
| pageCount: (await context.pages()).length, | ||
| activeUrl: await secondPage.url(), | ||
| }; | ||
| ``` | ||
|
|
||
| The same browser, pages, cookies, and navigation state persist across successful tool calls. Local | ||
| JavaScript variables do not persist, so rediscover pages and elements each call. If a call stops | ||
| responding, the owning agent framework should terminate and restart the local MCP process. That | ||
| restart begins a fresh browser and loses the previous browser state. | ||
|
|
||
| ## Return discipline | ||
|
|
||
| Return only the compact evidence needed by the agent. Prefer strings, numbers, booleans, arrays, | ||
| and plain objects. Do not return page, locator, context, Stagehand, or Zod objects. Await asynchronous | ||
| methods before returning. Logs and oversized return values are bounded by the executor. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| { | ||
| "name": "@browserbasehq/stagehand-codemode", | ||
| "version": "0.0.0", | ||
| "private": true, | ||
| "description": "Local stdio MCP tool for Stagehand code mode", | ||
| "type": "module", | ||
| "exports": { | ||
| ".": { | ||
| "types": "./dist/index.d.ts", | ||
| "import": "./dist/index.js" | ||
| } | ||
| }, | ||
| "scripts": { | ||
| "build": "tsdown", | ||
| "typecheck": "tsc --noEmit" | ||
| }, | ||
| "dependencies": { | ||
| "@modelcontextprotocol/sdk": "^1.30.0", | ||
| "zod": "^4.4.3" | ||
| }, | ||
| "peerDependencies": { | ||
| "@browserbasehq/stagehand": "*" | ||
| }, | ||
| "peerDependenciesMeta": { | ||
| "@browserbasehq/stagehand": { | ||
| "optional": true | ||
| } | ||
| }, | ||
| "devDependencies": { | ||
| "@types/node": "^25.0.9", | ||
| "tsdown": "^0.15.4", | ||
| "typescript": "^5.9.3" | ||
| }, | ||
| "engines": { | ||
| "node": ">=22.18.0" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import type { StagehandCodeConfig } from './types.js'; | ||
|
|
||
| export function stagehandCodeConfigFromEnv( | ||
| env: NodeJS.ProcessEnv = process.env | ||
| ): StagehandCodeConfig { | ||
| const explicitModelName = nonEmpty(env.STAGEHAND_MODEL_NAME); | ||
| const explicitModelApiKey = nonEmpty(env.STAGEHAND_MODEL_API_KEY); | ||
| const inferredGoogleKey = | ||
| nonEmpty(env.GEMINI_API_KEY) ?? | ||
| nonEmpty(env.GOOGLE_API_KEY) ?? | ||
| nonEmpty(env.GOOGLE_GENERATIVE_AI_API_KEY); | ||
| const modelName = | ||
| explicitModelName ?? | ||
| (inferredGoogleKey ? 'google/gemini-2.5-flash-lite' : undefined); | ||
| const modelApiKey = explicitModelApiKey ?? inferredGoogleKey; | ||
| return { | ||
| browserbaseApiKey: nonEmpty(env.BROWSERBASE_API_KEY), | ||
| ...(modelName | ||
| ? { | ||
| model: { | ||
| modelName, | ||
| ...(modelApiKey ? { apiKey: modelApiKey } : {}), | ||
| ...(nonEmpty(env.STAGEHAND_MODEL_BASE_URL) | ||
| ? { baseURL: nonEmpty(env.STAGEHAND_MODEL_BASE_URL) } | ||
| : {}), | ||
| }, | ||
| } | ||
| : {}), | ||
| }; | ||
| } | ||
|
|
||
| function nonEmpty(value: string | undefined): string | undefined { | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is there not a utility function for this from lodash or a similar package? |
||
| const trimmed = value?.trim(); | ||
| return trimmed ? trimmed : undefined; | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's embed a line saying that the most up-to-date knowledge can be found by grepping through the embedded docs package in
@browserbasehq/stagehandThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should probably also add a
REFERENCE.mdwith exhaustive documentation, or just point the agent to the docs for this