Eval bug: Nemotron 3.5 lightning fails to load

[vehystrix]

Name and Version

llama --version
version: 0.4.6 (build 11794, commit 78af832)
built with Clang 20.1.8 for Windows x86_64
ggml op count: 105

Operating systems

Windows

GGML backends

CUDA

Hardware

Ryzen 7 7800x3D
RTX 4080 SUPER

Models

Myric/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-GGUF:NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-mini

Problem description & steps to reproduce

Just loading the model with no additional options fails.
Loading the model with base llama.cpp v0.4.0 works.

First Bad Commit

No response

Relevant log output

> llama serve -hf Myric/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-GGUF:NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-mini
0.01.236.365 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.01.236.586 W srv  llama_server: -----------------
0.01.236.589 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
0.01.236.591 W srv  llama_server: this can be a security risk (cross-origin attacks)
0.01.236.591 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
0.01.236.592 W srv  llama_server: -----------------
0.01.249.474 I srv    load_model: loading model 'Myric/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-GGUF:NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-mini'
0.01.961.117 W model has unused tensor blk.52.nextn.enorm.weight (size = 10752 bytes) -- ignoring
0.01.961.130 W model has unused tensor blk.52.nextn.hnorm.weight (size = 10752 bytes) -- ignoring
0.01.961.139 W model has unused tensor blk.52.nextn.eh_proj.weight (size = 8128512 bytes) -- ignoring
0.01.961.148 W model has unused tensor blk.52.nextn.shared_head_norm.weight (size = 10752 bytes) -- ignoring
0.01.961.156 W model has unused tensor blk.52.attn_norm.weight (size = 10752 bytes) -- ignoring
0.01.961.179 W model has unused tensor blk.52.attn_q.weight (size = 7569408 bytes) -- ignoring
0.01.961.186 W model has unused tensor blk.52.attn_k.weight (size = 473088 bytes) -- ignoring
0.01.961.194 W model has unused tensor blk.52.attn_v.weight (size = 473088 bytes) -- ignoring
0.01.961.223 W model has unused tensor blk.52.attn_output.weight (size = 7569408 bytes) -- ignoring
0.01.961.239 W model has unused tensor blk.52.post_attention_norm.weight (size = 10752 bytes) -- ignoring
0.01.961.248 W model has unused tensor blk.52.ffn_gate_inp.weight (size = 688128 bytes) -- ignoring
0.01.961.256 W model has unused tensor blk.52.exp_probs_b.bias (size = 512 bytes) -- ignoring
0.01.961.264 W model has unused tensor blk.52.ffn_down_exps.weight (size = 678494208 bytes) -- ignoring
0.01.961.272 W model has unused tensor blk.52.ffn_up_exps.weight (size = 439025664 bytes) -- ignoring
0.01.961.281 W model has unused tensor blk.52.ffn_down_shexp.weight (size = 10601472 bytes) -- ignoring
0.01.961.290 W model has unused tensor blk.52.ffn_up_shexp.weight (size = 6859776 bytes) -- ignoring
0.05.614.341 I cmn          init: llama threadpool init, n_threads = 8
D:/a/beellama.cpp/beellama.cpp/ggml/src\ggml-impl.h:319: fatal error