feat(vis): add loop trace evaluation metrics - #2897
Conversation
|
2a89721 to
73c6063
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 73c60638a4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (record.type === 'turn.prompt') { | ||
| current.nextPromptLineNo = entry.lineNo; | ||
| mutablePhases.push(current); | ||
| current = createPhase(mutablePhases.length, entry.lineNo); |
There was a problem hiding this comment.
Do not split phases on rejected prompt records
In v1 wires, a prompt submitted while a turn is active is persisted before TurnFlow.launch rejects it with turn.agent_busy (packages/agent-core/src/agent/turn/index.ts:172-177,205-215); the existing regression trace at packages/agent-core/test/agent/turn.test.ts:2793-2803 confirms that the old turn's tool calls can follow that rejected prompt record. Unconditionally starting a phase here therefore resets exact-run and repetition-window metrics even though the agent never received a new prompt, artificially lowering loop measurements. Correlate boundaries with an actually started/new turnId, rather than treating every recorded prompt attempt as accepted.
Useful? React with 👍 / 👎.
Related Issue
Related to #2622
Problem
The visualizer can reconstruct turns, steps, tools, and context fill, but it cannot quantify the two loop shapes reported in the linked issue: long exact runs and rotating reuse of a small call alphabet. This makes candidate breaker policies difficult to calibrate against healthy traces before changing runtime behavior.
What changed
This PR is diagnostics-only. It does not classify a session as looping, choose alert thresholds, or change agent/runtime behavior.
Verification
pnpm --filter @moonshot-ai/vis-web testpnpm --filter @moonshot-ai/vis-web typecheckpnpm --filter @moonshot-ai/vis-web buildpnpm typecheckpnpm exec oxlint --type-aware --quietChecklist
gen-changesetsskill; vis packages are explicitly ignored and need no changeset.gen-docsreview; this self-contained debug UI addition needs no docs update.