You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some significant cost reductions for customers could be acheived through caching of LLM requests request/response.
Proposal: the user specifies a folder and the cli will use it to fetch/write cache files for each request. The file names is the sha of the request payload (+ some format version info), the content is the LLM response JSON.
file layout should be github actions cache compatible so that actions benefits from this feature
Some significant cost reductions for customers could be acheived through caching of LLM requests request/response.
Proposal: the user specifies a folder and the cli will use it to fetch/write cache files for each request. The file names is the sha of the request payload (+ some format version info), the content is the LLM response JSON.