Skip to content

kimik3-fp4-mi355x-vllm-agentic-mtp: add LMCache DRAM KV-offload arm #2598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sammshen wants to merge 42 commits into
base: main
Choose a base branch
from
+416 −1,287
Open

kimik3-fp4-mi355x-vllm-agentic-mtp: add LMCache DRAM KV-offload arm #2598

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
055dfc2
kimik3-fp4-mi355x-vllm-agentic-mtp: add LMCache DRAM KV-offload arm
sammshen Aug 12, 2026
328836b
perf-changelog: select kimik3-fp4-mi355x-vllm-agentic-mtp LMCache arm
sammshen Aug 12, 2026
4584d37
Move the LMCache arm to a dedicated config key at conc 4/8/16
sammshen Aug 12, 2026
aba14d1
Lower the lmcache key's dram-utilization to fit the shm-backed L1
sammshen Aug 12, 2026
6b95b7b
Match the LMCache chunk size to this stack's 1536-token block size
sammshen Aug 12, 2026
a8cda1a
Raise the LMCache chunk size to 3072 for the KDA state group
sammshen Aug 12, 2026
afee680
Pin the LMCache MP server to the lmcache_driven transfer path
sammshen Aug 12, 2026
20b4fda
Hold LMCache L1 read locks for the job duration
sammshen Aug 12, 2026
3c1908b
Revert "Hold LMCache L1 read locks for the job duration"
sammshen Aug 12, 2026
f296829
[AMD] [WIP] [AGENTX] KIMI-K3 Perf Tuning
AjithSirra Aug 13, 2026
d28c525
[AMD] [WIP] [AGENTX] KIMI-K3 Perf Tuning
AjithSirra Aug 13, 2026
38faee4
[AMD] [WIP] [AGENTX] KIMI-K3 Perf Tuning
AjithSirra Aug 13, 2026
d671e53
Merge branch 'main' into kimik3-mi355x-lmcache
ApostaC Aug 13, 2026
0811fb0
trigger lmcache 0.5.4rc2
ApostaC Aug 13, 2026
ac6ad7d
update lmcache configus
ApostaC Aug 13, 2026
e890a2b
update conc
ApostaC Aug 13, 2026
9678373
perf-changelog: point the LMCache MI355X entry at #2598
sammshen Aug 14, 2026
834e42c
Merge remote-tracking branch 'upstream/main' into mi355x-relink
sammshen Aug 14, 2026
c0119e6
[AMD] [AGENTX] KIMI-K3 Perf Tuning
AjithSirra Aug 14, 2026
ad8f584
Merge branch 'main' into amd/kimi-k3-agentic-perf-tuning
ajith-sirra-amd Aug 14, 2026
eb81b48
[AMD] [AGENTX] KIMI-K3 Perf Tuning
AjithSirra Aug 14, 2026
ed6c730
[AMD] [AGENTX] KIMI-K3 Perf Tuning
AjithSirra Aug 14, 2026
b7d1f4f
Merge branch 'main' into amd/kimi-k3-agentic-perf-tuning
ajith-sirra-amd Aug 14, 2026
8366815
[AMD] [WIP] [AGENTX] KIMI-K3 Perf with fixes applied.
AjithSirra Aug 14, 2026
9bc673b
[AMD] [WIP] [AGENTX] KIMI-K3 Perf with fixes applied.
AjithSirra Aug 14, 2026
c1215ff
Merge branch 'amd/kimi-k3-agentic-perf-tuning' of https://github.com/…
AjithSirra Aug 14, 2026
f080865
[AMD] [WIP] [AGENTX] KIMI-K3 Perf with fixes applied.
AjithSirra Aug 14, 2026
4901088
[AMD] [WIP] [AGENTX] KIMI-K3 with fixes applied & Perf Boost
AjithSirra Aug 14, 2026
84bc103
[AMD] [WIP] [AGENTX] KIMI-K3 with fixes applied & Perf Boost
AjithSirra Aug 14, 2026
aab67c0
[AMD] [WIP] [AGENTX] KIMI-K3 with fixes applied, Perf Boost & stability.
AjithSirra Aug 14, 2026
57fd591
[AMD] [WIP] [AGENTX] KIMI-K3 with fixes applied, Perf Boost & stability.
AjithSirra Aug 14, 2026
081d1b8
[AMD] [WIP] [AGENTX] KIMI-K3 with fixes applied, Perf Boost & stability.
AjithSirra Aug 14, 2026
8a52651
[AMD] [WIP] [AGENTX] KIMI-K3 with fixes applied, Perf Boost & stability.
AjithSirra Aug 14, 2026
e985b3e
Merge branch 'main' into amd/kimi-k3-agentic-perf-tuning
ajith-sirra-amd Aug 14, 2026
73ecfc9
Merge remote-tracking branch 'upstream/main' into mi355x-relink
sammshen Aug 14, 2026
b2725e0
Run the LMCache arm on top of the #2602 Kimi-K3 perf tuning
sammshen Aug 14, 2026
8ee5941
Drop #2602's changelog entry; keep its code fixes
sammshen Aug 14, 2026
fbc0ae2
perf-changelog: rebuild from main + this PR's entry only
sammshen Aug 14, 2026
a3a1dca
Merge remote-tracking branch 'upstream/main' into mi355x-relink
sammshen Aug 14, 2026
0a44aff
perf-changelog: repair conflict markers pushed in a3a1dcad
sammshen Aug 14, 2026
26c8ee3
retrigger: rerun MI355X LMCache sweep after cluster node failures
sammshen Aug 15, 2026
38cbcbe
update: lmcache 0.5.4rc4
ApostaC Aug 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading