fix: accept the common reasoning tag names, not just <think> - #9
Merged
senamakel merged 1 commit intoSep 11, 2026
Merged
Conversation
Inline reasoning-tag extraction matched a single hard-coded name, `think`.
The tag name is not standardized and the runtime does not report it, so a
model using any other convention had its entire chain of thought delivered
as visible answer text.
Captured live from `exaone-deep:2.4b` through Ollama's OpenAI-compatible
`/v1/chat/completions` (2026-09-10): the model wraps its reasoning in
`<thought>…</thought>` and sends no `reasoning` / `reasoning_content` side
channel at all. One captured response was 5011 bytes of `content` — 4979 of
chain of thought and a 9-byte answer (`\boxed{4}`). Every byte was visible.
Streamed, the opening tag arrives split across three deltas (`<`, `thought`,
`>`), so the partial-tag buffering has to hold across a candidate set.
Nothing downstream re-inspects the text — reasoning is separated from the
answer by this one decision — so a name this module does not match is
unrecoverable. `tag_name: String` becomes `tag_names: Vec<String>`, with the
default accepting `think`, `thinking`, `thought` and `reasoning`.
A section is closed by the closing tag of the name that opened it, so a wider
set cannot let one convention terminate another (`<thinking>a</think>b` stays
open until `</thinking>`). `first_tag_match` picks the earliest candidate and
prefers a resolved tag over an undecided partial at the same index; two tags
can never resolve at one index, since the terminating `>` means no tag literal
is a prefix of another.
`ReasoningTagExtraction::new` keeps its single-name signature; `for_tags`
takes an explicit set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ipSkHi47nsBmnxX5dC4dA
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: Comment |
How this change flows1 changed behaviour across 10 relationships. 6 surrounding behaviours are shown (60 graph nodes walked). 48 further behaviours left out to keep the diagram readable. flowchart LR
n0["ReasoningTagExtraction<br/>changed"]:::changed
n1["default"]:::impacted
n2["run_stream"]:::impacted
n3["extract_reasoning"]:::impacted
n4["collect_sse_with"]:::impacted
n5["parse_chat_response"]:::impacted
n6["new"]:::impacted
n2 -->|uses| n0
n2 -->|calls| n6
n2 -->|tests| n6
n3 -->|uses| n0
n3 -->|calls| n6
n4 -->|uses| n0
n5 -->|uses| n0
n5 -->|calls| n3
n6 -->|uses| n0
n6 -->|calls| n1
classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Inline reasoning-tag extraction matched a single hard-coded tag name,
think, so a model using any other convention had its entire chain of thought delivered as visible answer text. The tag name is not standardized and the runtime does not report it, and nothing downstream re-inspects the text — reasoning is separated from the answer by this one decision — so a name this module does not match is unrecoverable.tag_name: Stringbecomestag_names: Vec<String>; the default acceptsthink,thinking,thoughtandreasoning.This was captured live, not reasoned about.
exaone-deep:2.4bthrough Ollama's OpenAI-compatible/v1/chat/completions(2026-09-10) wraps its reasoning in<thought>…</thought>and sends noreasoning/reasoning_contentside channel at all — streamed delta keys are onlycontentandrole. One captured response:contentbytes\boxed{4})Streamed, the opening tag really does arrive split across three deltas (
<,thought,>), so the partial-tag buffering has to hold across a candidate set rather than one literal.Safety of the wider set
The hazard in widening is stripping text that is not a tag, so the matching is deliberately conservative:
<thinking>a</think>bstays open until</thinking>.first_tag_matchtakes the earliest candidate and prefers a resolved tag over an undecided partial at the same index. Two tags can never resolve at one index: every literal is<name>/</name>, and the terminating>means no tag literal is a prefix of another (<think>does not prefix<thinking>).<thin,<thinke,<thought(unclosed),<reasonand a bare<all survive as verbatim prose, including when held across a delta boundary.Residual risk, stated plainly: a model that writes
<thought>or<reasoning>as literal example markup in an answer now has it treated as reasoning. That exposure already existed for<think>; this widens it to three more names. It is the same trade the single-name default already made, and the alternative — leaking chain of thought as the answer — is the worse failure.Related issue
Addresses tinyhumansai/openhuman#6184 (no closing keyword on purpose: merging here does
not deliver the fix to openhuman, which vendors this crate through
tinyagents. Theissue closes when the pin is bumped there, which is a separate deliverable).
API or behavior changes
ReasoningTagExtraction::new(tag_name)keeps its single-name signature and behavior. NewReasoningTagExtraction::for_tags(iter)takes an explicit set. The changed field is private. Not breaking.Validation
Commands actually run, with their outcome:
cargo fmt --all -- --check— exit 0cargo clippy --all-targets --all-features -- -D warnings— exit 0, no warningscargo build --all-targets --all-features— exit 0cargo test --all-features— exit 0, 286 passed / 0 failed (278 + 1 + 7)Tests
Five tests added, four of which fail without the production change (verified by reverting it and re-running — each failure names the leaked chain of thought in the visible answer, e.g.
left: "<thought>\nOkay, the user is asking…</thought>\n\n\\boxed{4}",right: "\\boxed{4}"):default_extracts_thought_tag_from_live_exaone_capture— the captured non-streaming shape.default_extracts_thought_tag_split_across_deltas— the same, streamed with the real</thought/>split.default_covers_common_reasoning_tag_names— all four names, streamed and whole-string.near_miss_prefixes_are_released_not_swallowed— prose near-misses survive verbatim;<thinking>a</think>b</thinking>is not closed early.tag_names_outside_the_default_set_stay_visible— guards against over-widening. This one passes with or without the fix by design; it pins behavior the change must not alter, so it proves nothing on revert.Additionally verified out-of-tree, not kept in the repo (5KB of captured model output is noise): the full real 5011-byte capture fed through
extract_reasoningyields visible\boxed{4}and 4979 bytes of reasoning, and the same content split one delta per character — a harsher split than the wire produces — gives an identical split.Deliberately untested: whether any hosted OpenAI-compatible provider inlines a non-
thinktag. Only local runtimes were reachable from here.Documentation
Module docs updated: the header no longer claims a single tag, and the contract for the tag set, the opener-pinned closer, and why a mismatch is unrecoverable are stated at the top of
reasoning_tags.rs. Field and constructor docs updated to match.Checklist
#[allow(...)],#[ignore], or relaxed lints.envcontents in the diff or the description🤖 Generated with Claude Code
https://claude.ai/code/session_016ipSkHi47nsBmnxX5dC4dA