Skip to content

fix: preserve split UTF-8 in JSON-RPC reader - #397

Draft
zhuwenzhuang wants to merge 1 commit into
agentclientprotocol:mainfrom
zhuwenzhuang:codex/preserve-split-utf8
Draft

fix: preserve split UTF-8 in JSON-RPC reader#397
zhuwenzhuang wants to merge 1 commit into
agentclientprotocol:mainfrom
zhuwenzhuang:codex/preserve-split-utf8

Conversation

@zhuwenzhuang

Copy link
Copy Markdown

Summary

  • decode Codex app-server stdout with a stateful UTF-8 decoder
  • preserve compatibility when the readable already emits strings
  • add a regression test that delivers one Chinese character as three separate byte chunks

Root cause

Calling Buffer.toString() independently for every stdout chunk replaces an incomplete multi-byte UTF-8 sequence with U+FFFD. Since stream chunk boundaries are arbitrary, valid JSON text could therefore be corrupted before JSON parsing.

Validation

  • npm test: 401 passed, 28 skipped
  • npm run typecheck
  • npm run build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant