Blob storage: `Blob` state machine with a pluggable gRPC data plane by rjhuijsman · Pull Request #109 · reboot-dev/reboot

rjhuijsman · 2026-07-23T15:41:41Z

Before this change, Reboot had no first-class way to store large binary objects: state machines hold protobuf state, which is unsuited to multi-megabyte payloads, so applications had nowhere to put user uploads like images or videos.

This PR introduces the Blob state type as the control plane for one immutable-once-committed binary object: its state holds only metadata (content type, size, upload progress, lifecycle, authorization) while the bytes travel directly between the client and a data plane. Data is uploaded to the data plane in parts, each via PUT to a URL minted by the control plane - compatible with S3 as a data plane, if we ever want that. Uploads are resumable, sizes are enforced against the real bytes at commit time, and never-committed blobs expire automatically.

The data plane is a plain gRPC service, BlobDataPlane (data_plane.proto — no Reboot options, implementable by anything), discovered via REBOOT_BLOB_DATA_PLANE_URL. Locally (rbt dev run / rbt serve run / unit tests) a filesystem-based data plane server is run out-of-the-box.

Authorization: Blob has two mechanisms that combine:

Blob creation is always application-mediated; it can't be done directly from a frontend, creation must go through a backend call. The backend can then use whatever existing auth mechanism they'd like. Assuming the Blob is given a random ID, knowing that state ID acts as a capability: to upload or download you must first know the ID.
Blobs can be created with an uploader_id and downloader_ids; if given these will limit uploads and downloads to only users whose IDs are in those lists.

Reviewer hint: review commits in-order.

TESTED: with new unit tests, and by manually running rbt dev run.

aviator-app · 2026-07-23T15:41:44Z

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This pull request is currently open (not queued).

How to merge

To merge this PR, comment /aviator merge or add the mergequeue-ready label.

See the real-time status of this PR on the Aviator webapp.

Use the Aviator Chrome Extension to see the status of your PR within GitHub.

Copilot

Pull request overview

Introduces first-class blob storage with a Reboot control plane, pluggable gRPC data plane, local filesystem implementation, and browser SDK.

Changes:

Adds blob protocols, lifecycle, authorization, multipart storage, and tests.
Integrates local data-plane startup into CLI and test harnesses.
Demonstrates attachments through the chat-room example and React hooks.

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
`tests/reboot/std/blob/v1/BUILD.bazel`	Configures blob tests.
`tests/reboot/std/blob/v1/blob_tests.py`	Tests blob lifecycle and authorization.
`tests/reboot/examples/chat-room/serve_expected_output.txt`	Updates structured message output.
`tests/reboot/examples/chat-room/BUILD.bazel`	Adds frontend blob packages.
`reboot/std/react/package.json`	Exports React blob helpers.
`reboot/std/react/BUILD.bazel`	Builds React blob helpers.
`reboot/std/react/blob/package.json`	Configures the blob submodule.
`reboot/std/react/blob/index.tsx`	Implements browser upload/download helpers.
`reboot/std/react/blob/BUILD.bazel`	Builds the React blob module.
`reboot/std/BUILD.bazel`	Includes the blob TypeScript API.
`reboot/std/blob/v1/package.json`	Configures the blob module.
`reboot/std/blob/v1/index.ts`	Exposes Node.js blob integration.
`reboot/std/blob/v1/BUILD.bazel`	Builds blob Python and TypeScript libraries.
`reboot/std/blob/v1/blob.py`	Implements the Blob control plane.
`reboot/std/blob/v1/_store.py`	Implements filesystem blob storage.
`reboot/std/blob/v1/_proxy.py`	Proxies application byte routes.
`reboot/std/blob/v1/_http.py`	Serves filesystem upload/download requests.
`reboot/std/blob/v1/_filesystem_server.py`	Hosts the local data plane.
`reboot/std/blob/v1/_data_plane.py`	Creates data-plane connections.
`reboot/routing/cors_settings.py`	Exposes upload ETags through CORS.
`reboot/examples/chat-room/frontend/web/src/App.tsx`	Adds attachment UI and uploads.
`reboot/examples/chat-room/frontend/web/src/App.module.css`	Styles attachment UI.
`reboot/examples/chat-room/frontend/web/package.json`	Adds blob frontend dependencies.
`reboot/examples/chat-room/frontend/reboot-non-react-web/src/main.ts`	Handles structured messages.
`reboot/examples/chat-room/frontend/mobile/src/App.tsx`	Handles structured messages.
`reboot/examples/chat-room/frontend/.tests/type_check.sh`	Installs local blob packages.
`reboot/examples/chat-room/backend/tests/chat_room_servicer_test.py`	Updates message assertions.
`reboot/examples/chat-room/backend/src/main.py`	Registers the blob library.
`reboot/examples/chat-room/backend/src/chat_room_servicer.py`	Creates attachment blobs.
`reboot/examples/chat-room/api/chat_room/v1/chat_room.proto`	Defines attachment APIs.
`reboot/examples/chat-room/.tests/serve_test.sh`	Updates response validation.
`reboot/cli/common/BUILD.bazel`	Builds data-plane startup support.
`reboot/cli/common/blob_data_plane.py`	Spawns the local data plane.
`reboot/cli/commands/serve.py`	Starts blobs under `serve run`.
`reboot/cli/commands/dev.py`	Starts blobs under `dev run`.
`reboot/cli/commands/BUILD.bazel`	Adds CLI blob dependencies.
`reboot/BUILD.bazel`	Packages blob runtime artifacts.
`reboot/aio/tests.py`	Runs a data plane in tests.
`reboot/aio/http.py`	Adds PUT route support.
`reboot/aio/BUILD.bazel`	Adds test-harness blob dependencies.
`rbt/std/BUILD.bazel`	Packages generated blob APIs.
`rbt/std/blob/v1/package.json`	Configures generated blob modules.
`rbt/std/blob/v1/data_plane.proto`	Defines the gRPC data-plane contract.
`rbt/std/blob/v1/BUILD.bazel`	Generates blob protocol bindings.
`rbt/std/blob/v1/blob.proto`	Defines Blob state and methods.
`documentation/docs/learn_more/testing.md`	Updates testing examples.
`documentation/docs/learn_more/define/protobuf.mdx`	Updates protobuf examples.
`documentation/docs/learn_more/call/from_within_your_app.mdx`	Updates internal-call examples.
`documentation/docs/learn_more/call/from_react.mdx`	Updates React examples.
`documentation/docs/learn_more/applications.mdx`	Documents blob registration and PUT routes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Before this change a job changed name at each layer it passed through: `BlobDataPlane.CompleteUpload` called the store's `complete()`, `GetPartUploadInstructions` its `part_put_url()`, the byte routes signed with `signature_for_put()` and `signature_for_get()`, the browser fetched `instructions()` before a part upload, the metadata half of `Blob.Remove` was `StoredBlob.Forget`, and the workflow that `Blob.Commit` schedules was `Blob.CompleteUpload` -- the data plane's name for a different job. What `Blob.Create` schedules was `BeginUpload`, on every layer down to the `StoredBlob` constructor, though nothing is begun from the client's point of view: the blob is brought into existence, ready for parts, and the client's upload starts later with its first `PUT`. A reader following one job across the files learned a new name for it at every step. Every layer now uses one name per job. Creating a blob is `Create` everywhere: `Blob.Create` schedules `Blob.CreateWorkflow`, which calls `BlobDataPlane.Create` (S3's own name for it is `CreateMultipartUpload`), which delegates to the store's `create()`, which constructs `StoredBlob` through its `Create`; the filesystem store's own factory moves from `create` to `open` to free the name. Finishing an upload is a commit everywhere: `Blob.Commit` schedules `Blob.CommitWorkflow`, as `Remove` schedules `RemoveWorkflow`, which calls `BlobDataPlane.Commit`, which delegates to the store's `commit()`, which drives `StoredBlob.Commit`; the data plane had been the one layer to say "complete upload". The store's other methods are named after the RPC each one serves (`part_upload_url`, `download_url`, `delete`), so the shared servicer delegates every RPC to its namesake, and the byte routes sign a part upload and a download with `signature_for_part_upload()` and `signature_for_download()`. The browser's `partUploadInstructions()` is named after the RPC it calls. `StoredBlob.Forget` is `StoredBlob.Remove`, the metadata half of `Blob.Remove`; only the plain gRPC data plane can say `Delete`, since Reboot reserves that method name. The workflow half of a job on `Blob` is named after the writer that schedules it, with the `Workflow` suffix Cloud uses (`UpdateCard` and `UpdateCardWorkflow`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…o its store

`tests/reboot/std/react`: unit-test the part `PUT`'s retries Before this change nothing in the repository exercised `reboot/std/react` on its own: the package was type-checked and driven through the chat-room example, so a retry policy could be wrong in ways no test would say, and a test of it would have had to mock the generated client and the browser both. The policy now lives in `reboot/std/react/blob/put.ts`, with the `fetch` it uses and the sleep between attempts handed in, and `index.tsx` calls `putPartWithRetries` with the real ones. The package's `exports` map keeps the module internal. A plain `js_test` (no Reboot backend, so no wheel) under `tests/reboot/std/react/blob` scripts `fetch` one answer per call and checks each branch of the policy: a first-time success, a 5xx followed by a dropped connection, a 408 followed by a 429, a 5xx and a 403 whose bodies never arrived, a 403 retried on a freshly minted URL, giving up after the last attempt with each wait double the one before, each status refused for good, a success without an `ETag`, and an abort mid-wait. Writing the tests found two things. Reading an error response's body sat outside the retry's `try`, so a connection dropped after a 503's headers rejected the upload instead of retrying it; the body is now read best-effort, since the status is the verdict and the body only explains it. And the cap on the backoff could never bind: four attempts wait at most 500, 1000 and 2000 milliseconds, so the cap and its constant are gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…retried `reboot/std/react`: retry a part's `PUT` the way its RPCs are retried An upload's control-plane calls go through the generated web client and get its retries, but before this change the bytes went out as a bare `fetch`, so one dropped connection or one 503 from a store failed the whole upload from the user's point of view, however many parts were already confirmed. A part's `PUT` is now attempted up to four times, with a doubling, jittered delay in between: after a request that never got an answer, and after a store's 408, 429 or 5xx. A 403 is retried with a freshly minted URL, since on either store that is what an expired URL earns, and a part that starts late in a slow upload can outlive the minutes its URL was minted for. A 400, 404, 409 or 413 is refused for good, as is anything once the caller has aborted. What no attempt can fix is still a rejection of `upload()`, naming the part and the reason, and the rest of that part's window is stopped rather than left to run out its own retries against an upload already rejected; the confirmed parts stay confirmed, so calling `upload()` again for the same blob resumes it. The hook's documentation now says so, and that a blob never committed is removed by the backend after a day, which is all the cleanup an abandoned upload needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

`rbt/std/blob`: say on `BlobDataPlane` who retries what Before this change the interface said every method had to be safe to call more than once, and left the reader to work out why: the retries live in the caller. A data plane retries nothing itself. The one failure a method declares is `CompleteUpload`'s `error`, for a completion that can never succeed; every other failure is an undeclared gRPC error, which the contract now says the caller retries until the call succeeds -- the control plane from its workflows, a client of the presigning methods through the control plane's client. One place for every retry keeps them understandable and lets a data plane stay stateless. The same contract is stated where a store implementer meets it, on `BlobStore`, and `CompleteUpload`'s `error` is documented as final for that commit: retrying the call would only repeat it. The browser hook now says what that means for it: `upload()` again would skip every part the blob already has and repeat the verdict, so upload into a new blob, or replace parts through `BlobUploader` and commit again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

… layer

Before this change the contract a store implements was split across two modules: `BlobStore` lived beside the shared servicer, but the `UploadedPart` its `commit()` takes, the `BlobStoreError` it raises for a final failure and the `DEFAULT_PART_SIZE_BYTES` its `part_size` should answer lived in `_store.py`, the filesystem store. A store elsewhere could only raise the error the shared servicer recognizes by importing it from another store's private module, and the servicer, meant to serve any store, depended on that one. `BlobStoreError`, `UploadedPart` and `DEFAULT_PART_SIZE_BYTES` are now defined next to `BlobStore` in `_data_plane_servicer.py`, and the filesystem store imports them from there like any other store would; the servicer no longer imports the filesystem store at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…retried

…o its store

Before this change `upload()` began by asking the blob for its part size through a reactive `GetPartUploadInstructions`, which a blob that is no longer uploading refuses with a declared error -- and the web client's reactive read retries a refusal forever, so `upload()` hung, until the caller's signal aborted, on any blob past `UPLOADING`. That is where the hook's promise that calling `upload()` again resumes an interrupted upload broke: an upload interrupted while waiting for its commit's verdict, or simply repeated after it succeeded, never came back. The same read stood behind the fresh URL a part asks for after a 403, so a part of a blob removed meanwhile waited forever instead of failing. `upload()` now reads the blob's status first and answers what it finds: a committed blob's ETag at once, the verdict of a commit already under way (`Commit` is idempotent while pending), and a rejection for a removed blob; only a blob still uploading goes on to its parts. `partUploadInstructions()` asks plainly before it watches, so a refusal surfaces as the declared error, and only a blob whose upload session is still being provisioned is watched for the moment it is ready. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…ploading

…tract

Before this change the two readers that hand a client something minted by the data plane -- `GetPartUploadInstructions` and `GetDownloadUrl` -- let a failure of that data-plane call escape as it was, and Reboot propagates an exception a method does not handle as `Unknown`, which a client treats as final. So a data plane that was unreachable for a moment failed an upload before its first part, or a part's fresh URL after a 403, although the contract on `BlobDataPlane` says the client of a presigning method retries through the control plane's own client: that client retries `Unavailable`, and never saw it. Both readers now turn a data-plane call's failure into `Unavailable`, which Reboot propagates as it is and the generated clients retry with backoff, so a blip behind a presigned URL is ridden out the way one behind a workflow is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

Before this change the browser knew a commit had failed by the `commit_error` on the blob being a non-empty string, but nothing made it one: the shared servicer forwarded a `BlobStoreError`'s text as it was, and the store contract never asked for any. A store that raised `BlobStoreError()` bare had the blob revert to uploading with an empty verdict, which the browser did not count as one, so an `upload()` waited for the verdict it had already been given, until its caller gave up. The servicer now reports a verdict with words in it however the store raised, and the browser reads the verdict off the status -- the blob reverting to uploading is the failure, as `blob.proto` says; the message only explains it -- so neither side depends on the other remembering to say something. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

… layer

… a URL

…ploading

…ploading `reboot/std/react`: answer an `upload()` of a blob no longer uploading Before this change `upload()` began by asking the blob for its part size through a reactive `GetPartUploadInstructions`, which a blob that is no longer uploading refuses with a declared error -- and the web client's reactive read retries a refusal forever, so `upload()` hung, until the caller's signal aborted, on any blob past `UPLOADING`. That is where the hook's promise that calling `upload()` again resumes an interrupted upload broke: an upload interrupted while waiting for its commit's verdict, or simply repeated after it succeeded, never came back. The same read stood behind the fresh URL a part asks for after a 403, so a part of a blob removed meanwhile waited forever instead of failing. `upload()` now reads the blob's status first and answers what it finds: a committed blob's ETag at once; the verdict of a commit already under way, watched for rather than asked for again, since a second `Commit` would find a blob committed meanwhile, or restart one that has just failed; and a rejection for a removed blob, which is what a removal during the wait for a verdict is too. Only a blob still uploading goes on to its parts. `partUploadInstructions()` asks plainly before it watches, so a refusal surfaces as the declared error, and only a blob whose upload session is still being provisioned is watched for the moment it is ready. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

… a URL `reboot/std/blob`: let a client retry a data-plane blip behind a URL Before this change the two readers that hand a client something minted by the data plane -- `GetPartUploadInstructions` and `GetDownloadUrl` -- let a failure of that data-plane call escape as it was, and Reboot propagates an exception a method does not handle as `Unknown`, which a client treats as final. So a data plane that was unreachable for a moment failed an upload before its first part, or a part's fresh URL after a 403, although the contract on `BlobDataPlane` says the client of a presigning method retries through the control plane's own client: that client retries `Unavailable`, and never saw it. Both readers now turn a failure to reach the data plane into `Unavailable`, which Reboot propagates as it is and the generated clients retry with backoff, so a blip behind a presigned URL is ridden out the way one behind a workflow is. Only that kind of failure: a presigning call does nothing but reach the data plane and sign, so a refusal, or a data plane that cannot sign, stays the final error it was, rather than being retried until the caller gives up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…retried `reboot/std/react`: retry a part's `PUT` before failing the upload An upload's control-plane calls go through the generated web client and get its retries, but before this change the bytes went out as a bare `fetch`, so one dropped connection or one 503 from a store failed the whole upload from the user's point of view, however many parts were already confirmed. A part's `PUT` is now attempted up to four times, with a doubling, jittered delay in between: after a request that never got an answer, and after a store's 408, 429 or 5xx. A 403 is retried with a freshly minted URL, since on either store that is what an expired URL earns, and a part that starts late in a slow upload can outlive the minutes its URL was minted for. A 400, 404, 409 or 413 is refused for good, as is anything once the caller has aborted. What no attempt can fix is still a rejection of `upload()`, naming the part and the reason, and the rest of that part's window is stopped rather than left to run out its own retries against an upload already rejected; the confirmed parts stay confirmed, so calling `upload()` again for the same blob resumes it. The hook's documentation now says so, and that a blob never committed is removed by the backend after a day, which is all the cleanup an abandoned upload needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…ploading

…o its store `reboot/std`: move the filesystem data plane's bookkeeping into its store The S3 data plane's servicer authorizes a call and hands it to its store, because S3 keeps the metadata: the multipart session, the parts it holds, the object once completed. Before this change the filesystem data plane's servicer did that bookkeeping itself -- beginning a `StoredBlob` session, checking reported parts against written ones, committing a manifest, reclaiming what it did not name, forgetting a blob before removing its bytes -- and the byte routes drove `StoredBlob` directly as well, so three modules spoke the metadata protocol and the two servicers looked nothing alike. `FilesystemBlobStore` now drives `StoredBlob` itself and offers the surface `S3BlobStore` has: `begin_upload`, `part_put_url`, `complete`, `download_url` and `delete`, plus `publish_part` and `read` for the byte routes. Its methods take the context they reach `StoredBlob` with, which is the one honest difference from a store whose metadata lives in an object store. The servicer keeps the caller check and one delegating call per RPC, and `_http.py` no longer imports `StoredBlob`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D8vHXG2KLHDUruv8642AXY

…ploading

…retried

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 1574235 to 6f904f0 Compare August 3, 2026 15:49

rjhuijsman commented Aug 3, 2026

View reviewed changes

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 842a135 to b0c684d Compare August 24, 2026 15:47

rjhuijsman commented Aug 25, 2026

View reviewed changes

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 263b706 to 23d4ee1 Compare August 25, 2026 12:17

rjhuijsman commented Aug 25, 2026

View reviewed changes

Comment thread documentation/docs/learn_more/call/from_react.mdx Outdated

rjhuijsman force-pushed the rjh.blob-storage-v2 branch 2 times, most recently from ef66db2 to c306853 Compare August 25, 2026 13:10

rjhuijsman marked this pull request as ready for review August 25, 2026 13:17

rjhuijsman self-assigned this Aug 25, 2026

rjhuijsman requested review from benh and a balanced review from Copilot August 25, 2026 13:18

Copilot started reviewing on behalf of rjhuijsman August 25, 2026 13:18 View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes