Skip to content

fix: accept the common reasoning tag names, not just <think> - #9

Merged
senamakel merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:fix/6184-reasoning-tag-set
Sep 11, 2026
Merged

senamakel merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:fix/6184-reasoning-tag-set

Conversation

@M3gA-Mind

@M3gA-Mind M3gA-Mind commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Inline reasoning-tag extraction matched a single hard-coded tag name, think, so a model using any other convention had its entire chain of thought delivered as visible answer text. The tag name is not standardized and the runtime does not report it, and nothing downstream re-inspects the text — reasoning is separated from the answer by this one decision — so a name this module does not match is unrecoverable.

tag_name: String becomes tag_names: Vec<String>; the default accepts think, thinking, thought and reasoning.

This was captured live, not reasoned about. exaone-deep:2.4b through Ollama's OpenAI-compatible /v1/chat/completions (2026-09-10) wraps its reasoning in <thought>…</thought> and sends no reasoning / reasoning_content side channel at all — streamed delta keys are only content and role. One captured response:

before after
content bytes 5011 5011
shown to the user 5011 9 (\boxed{4})
routed to reasoning 0 4979

Streamed, the opening tag really does arrive split across three deltas (<, thought, >), so the partial-tag buffering has to hold across a candidate set rather than one literal.

Safety of the wider set

The hazard in widening is stripping text that is not a tag, so the matching is deliberately conservative:

  • A section is closed by the closing tag of the name that opened it, so a wider set cannot let one convention terminate another — <thinking>a</think>b stays open until </thinking>.
  • first_tag_match takes the earliest candidate and prefers a resolved tag over an undecided partial at the same index. Two tags can never resolve at one index: every literal is <name>/</name>, and the terminating > means no tag literal is a prefix of another (<think> does not prefix <thinking>).
  • A near-miss is released, not swallowed: <thin , <thinke, <thought (unclosed), <reason and a bare < all survive as verbatim prose, including when held across a delta boundary.

Residual risk, stated plainly: a model that writes <thought> or <reasoning> as literal example markup in an answer now has it treated as reasoning. That exposure already existed for <think>; this widens it to three more names. It is the same trade the single-name default already made, and the alternative — leaking chain of thought as the answer — is the worse failure.

Related issue

Addresses tinyhumansai/openhuman#6184 (no closing keyword on purpose: merging here does
not deliver the fix to openhuman, which vendors this crate through tinyagents. The
issue closes when the pin is bumped there, which is a separate deliverable).

API or behavior changes

  • Behavior (intended): the default now extracts three additional tag names. Content using them moves from the visible channel to the reasoning channel.
  • API: ReasoningTagExtraction::new(tag_name) keeps its single-name signature and behavior. New ReasoningTagExtraction::for_tags(iter) takes an explicit set. The changed field is private. Not breaking.

Validation

Commands actually run, with their outcome:

  • cargo fmt --all -- --check — exit 0
  • cargo clippy --all-targets --all-features -- -D warnings — exit 0, no warnings
  • cargo build --all-targets --all-features — exit 0
  • cargo test --all-features — exit 0, 286 passed / 0 failed (278 + 1 + 7)

Tests

Five tests added, four of which fail without the production change (verified by reverting it and re-running — each failure names the leaked chain of thought in the visible answer, e.g. left: "<thought>\nOkay, the user is asking…</thought>\n\n\\boxed{4}", right: "\\boxed{4}"):

  • default_extracts_thought_tag_from_live_exaone_capture — the captured non-streaming shape.
  • default_extracts_thought_tag_split_across_deltas — the same, streamed with the real < / thought / > split.
  • default_covers_common_reasoning_tag_names — all four names, streamed and whole-string.
  • near_miss_prefixes_are_released_not_swallowed — prose near-misses survive verbatim; <thinking>a</think>b</thinking> is not closed early.
  • tag_names_outside_the_default_set_stay_visible — guards against over-widening. This one passes with or without the fix by design; it pins behavior the change must not alter, so it proves nothing on revert.

Additionally verified out-of-tree, not kept in the repo (5KB of captured model output is noise): the full real 5011-byte capture fed through extract_reasoning yields visible \boxed{4} and 4979 bytes of reasoning, and the same content split one delta per character — a harsher split than the wire produces — gives an identical split.

Deliberately untested: whether any hosted OpenAI-compatible provider inlines a non-think tag. Only local runtimes were reachable from here.

Documentation

Module docs updated: the header no longer claims a single tag, and the contract for the tag set, the opener-pinned closer, and why a mismatch is unrecoverable are stated at the top of reasoning_tags.rs. Field and constructor docs updated to match.

Checklist

  • The change is focused on one logical change
  • No new #[allow(...)], #[ignore], or relaxed lints
  • No secrets, tokens, or .env contents in the diff or the description

🤖 Generated with Claude Code

https://claude.ai/code/session_016ipSkHi47nsBmnxX5dC4dA

Inline reasoning-tag extraction matched a single hard-coded name, `think`.
The tag name is not standardized and the runtime does not report it, so a
model using any other convention had its entire chain of thought delivered
as visible answer text.

Captured live from `exaone-deep:2.4b` through Ollama's OpenAI-compatible
`/v1/chat/completions` (2026-09-10): the model wraps its reasoning in
`<thought>…</thought>` and sends no `reasoning` / `reasoning_content` side
channel at all. One captured response was 5011 bytes of `content` — 4979 of
chain of thought and a 9-byte answer (`\boxed{4}`). Every byte was visible.
Streamed, the opening tag arrives split across three deltas (`<`, `thought`,
`>`), so the partial-tag buffering has to hold across a candidate set.

Nothing downstream re-inspects the text — reasoning is separated from the
answer by this one decision — so a name this module does not match is
unrecoverable. `tag_name: String` becomes `tag_names: Vec<String>`, with the
default accepting `think`, `thinking`, `thought` and `reasoning`.

A section is closed by the closing tag of the name that opened it, so a wider
set cannot let one convention terminate another (`<thinking>a</think>b` stays
open until `</thinking>`). `first_tag_match` picks the earliest candidate and
prefers a resolved tag over an undecided partial at the same index; two tags
can never resolve at one index, since the terminating `>` means no tag literal
is a prefix of another.

`ReasoningTagExtraction::new` keeps its single-name signature; `for_tags`
takes an explicit set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ipSkHi47nsBmnxX5dC4dA
@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 8ee15f7e-73d8-4fea-8aae-0efc23524dcd


Comment @coderabbitai help to get the list of available commands.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

$0.0000 · 0 in / 0 out

@tinysweeper

tinysweeper Bot commented Sep 10, 2026

Copy link
Copy Markdown

How this change flows

1 changed behaviour across 10 relationships. 6 surrounding behaviours are shown (60 graph nodes walked). 48 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["ReasoningTagExtraction<br/>changed"]:::changed
  n1["default"]:::impacted
  n2["run_stream"]:::impacted
  n3["extract_reasoning"]:::impacted
  n4["collect_sse_with"]:::impacted
  n5["parse_chat_response"]:::impacted
  n6["new"]:::impacted
  n2 -->|uses| n0
  n2 -->|calls| n6
  n2 -->|tests| n6
  n3 -->|uses| n0
  n3 -->|calls| n6
  n4 -->|uses| n0
  n5 -->|uses| n0
  n5 -->|calls| n3
  n6 -->|uses| n0
  n6 -->|calls| n1
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Loading

Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.

tinysweeper 0.1.0

@senamakel
senamakel merged commit e188996 into tinyhumansai:main Sep 11, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants