fix(chat_logger): reroute leaked think drafts, rescue truncated tool calls (#8) - #10
Merged
Conversation
Replicates two production failure modes of Qwen3.6 think-tag handling, verified against proxy traces and ~33k logged assistant turns: - content-channel </think> leaks (double tag emission, and thinking that mentions the tag in backticks being consumed as the real close tag) - thinking-only stalls, including tool calls trapped in an unclosed think block (complete and truncated </tool_call XML tails) Shared SSE fakes extracted from test_toolcall_rescue.py into tests/_helpers.py; control tests pin passthrough behaviour for legit tag mentions (inline and fenced), pure stalls, and normal streams. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…calls (#8) Two stream-repair mechanisms in SSEChunkLogger: - Content-side think-leak window: buffer the first 4KB of content deltas; if a standalone </think> line appears outside a code fence, emit the pre-tag text as reasoning_content, drop the tag, and stream the remainder as content. No trigger: flush and pass through. - EOF rescue of truncated tool calls: when the stream ends with a pending capture whose <function=...>...</function> block is complete and only the closing </tool_call> is cut off, synthesize the tool call and rewrite finish_reason to tool_calls. Held state (buffered content, held finish events, pending rescue) is resolved before [DONE] is forwarded, and empty transform output no longer terminates the read loop early (readany's b"" contract). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the two production failure modes of Qwen3.6 think-tag handling tracked in #8, verified against proxy traces and a scan of ~33k logged assistant turns:
</think>leaks into visible content (~0.5–0.6%/turn, all presets, not MTP-specific): the model either double-emits</think>(draft answer between two close tags) or its thinking mentions`</think>`in backticks and llama-server's parser consumes the mention as the real close tag. Either way, thinking text plus a literal</think>spills down the content channel.</tool_call) was unhandled.Changes
SSEChunkLogger(chat_logger.py):THINK_LEAK_WINDOW) of content deltas; on a standalone</think>line outside a code fence, reroute pre-tag text toreasoning_content, drop the tag, stream the rest as content. No trigger → flush and pass through. Inline/fenced tag mentions are untouched.<function=…>…</function>block and a residue that is a prefix of</tool_call>is synthesized as a real tool call (finish_reason rewritten totool_calls).[DONE]is forwarded; empty transform output no longer terminatesreadany()early (itsb""= EOF contract).Tests: new replication harness
tests/test_think_leak_replication.pybuilt from real production seams (sessions 8d62feae, 093cd785), shared SSE fakes extracted totests/_helpers.py, plus control tests pinning passthrough for legit tag mentions, pure stalls, and normal streams.Behavioral note: responses under 4KB now arrive as a single content delta after thinking completes (window flush at the finish event); reasoning still streams live.
Test plan
./.venv/Scripts/python.exe -m unittest discover -s tests— 20/20 green (run twice)finish_reason: stoppreserved, MTP drafting healthy, pi round-trip OKCloses #8
🤖 Generated with Claude Code