Evaluation traces

[normster]

Hi,

This is a really interesting dataset! Would you be able to share the evaluation traces used to compile the results in Table 1? Or the disaggregated scores from the table.

Thanks!

[tingofurro]

Hey Norman Mu (@normster),

Thank you for your message and interest in our work :)
We're planning to release the traces for the main and secondary experiments soon, but it will likely take a bit of time to release publicly.

If you need them sooner for analysis or experiments, contact me privately and we can likely find a way to share what you need.

Have a good day!

[Mar10CURADOR]

Really interesting work.
One thing that caught my attention is that the benchmark measures corruption/drift effects across delegated workflows, but it seems there is still an open question around structural traceability during iterative document transformations.
For example:
explicit session continuity markers
deterministic structural fields
reconstruction consistency checks
trace validation between iterations
It would be interesting to evaluate whether these mechanisms reduce silent semantic drift in long-context delegated workflows.
Especially because many failures are not catastrophic outputs, but subtle integrity degradations that remain “plausible” to the operator.
Great work.