Skip to content

Persist binary media for session suspend and resume #153

Description

@Sewer56

Status

Not planned. Unless I start using with TUI/GUI/non-headless.

We cannot currently suspend and resume sessions (across process boundaries) because the framework discards all binary media after a run completes.

This issue proposes an opt-in way to persist and retrieve this media, so a resume
can happen.

Motivations

A run's binary media exists only for the run's lifetime:

  • Distillation redacts user-to-LLM inline binary (images, audio,
    video, documents, files) to a media type plus byte count in audit
    text, then drops it.
  • Distillation skips LLM-to-user binary (ModelResponsePart::File)
    entirely. The caller can neither observe nor retrieve it.

Zero copies survive RunComplete processing. That is deliberate for
the audit trail, but it makes suspend and resume impossible:

  • Resume needs the raw transcript, including binary, to rebuild the
    provider request prefix.
  • RunConfig::preamble_messages cannot substitute: it flattens text
    into the prompt string, losing tool-call structure and media.

A session that stops loses every image it ever held.

Solution

Opt-in session persistence:

  1. Capture: an opt-in way to obtain the raw vendor transcript
    (e.g. serdesAI's Vec<ModelRequest>).
  2. Persist: serialize that transcript to caller-chosen storage.
    Binary survives the round trip.
  3. Resume: start a new run seeded with the persisted transcript via
    the vendor's message_history.
  4. Retrieve: a way for the caller to read LLM-generated
    ModelResponsePart::File content instead of losing it.

The caller decides where files live (single session file, or a
content-addressed blob store). etc.

Acceptance criteria

  • A persisted-then-resumed run reproduces the original serialized
    request prefix, so provider prompt caches still hit.
  • Binary user parts resume with full fidelity: the resumed model sees
    the same media bytes.
  • The caller can retrieve LLM-generated ModelResponsePart::File
    content.
  • Persistence is opt-in; runs without it keep the current
    zero-copy-after-completion behavior.
  • Resumed runs do not duplicate the static system prompt.

Additional context

Memory footprint is a stated project selling point (~10 MiB, README).
Persisted transcripts live on disk, not in memory, so the default
path stays unchanged.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions