hardening: cache-scope escape hatch, promptSource on large lines, distinct Grok 4.5 rows - #1031
Merged
Conversation
The month-scoped load a ranged query takes is a behaviour change on a warm cache with no way back except deleting it. Drop the scope in loadCache, the one place every caller (including the resident serve) routes through, so a suspect scoped read can be compared against a full one in place. Read policy only: deliberately not in PROVIDER_ENV_VARS, so setting or unsetting it invalidates nothing.
parseLargeJsonl dropped promptSource for exactly the lines SDK-generated prompts live on, so the recurring-context detector regex-scanned the ends of the raw line for it. Add the field to LARGE_ROOT_FIELDS (tiny scalar, add-only, isSidechain already there) and delete the workaround: it read only 2 KB from each end, so a flag further in was missed. No cache change: optimize scans the raw JSONL each run, so promptSource never has to persist on CachedFile. Fixes #1030. With #994 this closes #1023.
Reports bucket rows by model id and label them afterwards, so the two ids collapsing onto one display name printed what looked like the same row twice with different numbers. Give the variant its own SHORT_NAMES entry, which the longest-first match picks over the grok-4.5 prefix. Display only: ids are untouched, so nothing re-parses and no cost moves. Fixed in the shared table rather than the grok provider so the menubar and model-breakdown, which call getShortModelName directly, get it too. Fixes #1029.
…ning # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three small pre-release hardening items, separate commits.
CODEBURN_CACHE_SCOPE=all— escape hatch for perf(cache): shard the session cache by provider and month, load only the months a query needs #1007's month-scoped shard loads: one line at theloadCachechokepoint forces a full read; memo key becomesallso a resident serve reuses it; deliberately not inPROVIDER_ENV_VARS(read policy, must not invalidate caches). Documented indocs/architecture.md+ CHANGELOG. Test: scoped load under the env equals the unscoped load and the memo is shared. Real data: a fixed past range (export --from 2026-08-16 --to 2026-08-17, 8,217 calls) is identical modulogeneratedwith and without the env.promptSourcekept for > 32 KB lines (fixes parser: add promptSource to LARGE_ROOT_FIELDS and remove the >32 KB head/tail workaround #1030, fully closes parser: parseLargeJsonl drops isSidechain/promptSource for lines > 32 KB #1023): added toLARGE_ROOT_FIELDS; optimize: detect recurring context pasted at the start of sessions #1021's head/tail regex workaround inisMachineWrittenPromptdeleted (it only looked at 2 KB from each end). No cache change — optimize re-scans raw JSONL. Test with a > 32 KB line carryingpromptSource:"sdk"+isSidechain:true; the existing > 32 KB exclusion test now passes via the parsed field.grok-4.5-buildcollapsed ontogrok-4.5ingetShortModelName's prefix match — used by the menubar and model-breakdown too — so the fix isSHORT_NAMES['grok-4.5-build'] = 'Grok 4.5 (build)'(longest-first wins). Display only; ids and pricing untouched. Real data: twoGrok 4.5rows →Grok 4.5 (build)$33.44 +Grok 4.5$0.41.tscclean ·npm test2906 ·test:locks26 · app 503.