llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --load-mode none for better performance
The message
llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --load-mode none for better performanceWhat it means
llama-server can stop partway through loading Qwen3.8-Flash-Next on llama.cpp's HIP backend and never finish. It prints no error and ignores SIGTERM. The same file and flags load on Vulkan in 105 seconds.
What to do
Run the same command with -dev Vulkan0, which loaded the file in 105 seconds. To stay on HIP, keep less of the model on the card: -ncmoe 40 loads at 19.11 GB.
Put enough of Qwen3.8-Flash-Next on an AMD card and llama-server can stop dead partway through the load. No error, no exit, no port to connect to. The last line on screen is a warning about something else.
llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --load-mode none for better performance
- Model
- unsloth/Qwen3.8-Flash-Next-GGUF UD-Q4_K_XL (104 GiB, 4 shards)
- Backend that hangs
- HIP, with VRAM frozen at 30.7 GB
- Backend that loads
- Vulkan, in 105 s at 5.48 tok/s
- Loads on HIP at
- -ncmoe 40, 19.11 GB
What it looks like
The report is issue #29149 against llama.cpp, opened on September 19, 2026 on a build of master with ROCm 7.2.4. While it's wedged, 1 of 37 threads is active and 35 workers sit idle, the container burns 99.5% of a single core, the GPU does 0-3%, disk reads stop advancing and VRAM stays frozen. SIGTERM is ignored, and it takes a docker kill to stop it.
It isn't running out of memory
That was my first guess, and the report rules it out. A real allocation failure on the same machine gives up in about 30 seconds and says so.
llama_model_load: error loading model: unable to allocate ROCm1 buffer
The hang prints nothing like that, and it happens at allocations that fit.
What decides whether it loads
How much of the model sits on the GPU. At -ncmoe 42 the 4-bit file loads with 16.18 GB on the card, and at -ncmoe 40 it loads with 19.11 GB. Two -ot rules on the same single GPU reach 30.7 GB and wedge. The 3-bit file with -ncmoe 32 loads at 20.40 GB and serves at 2.49 tok/s.
What to run instead
Vulkan, with the same flags. Byte-identical apart from the device, it loaded in 105 seconds and ran at 5.48 tok/s; across both cards it reached 43.8 GB and 7.62 tok/s. Vulkan put more on the GPU than the HIP run that wedges, so the backend that works is doing the harder job.
What's already ruled out
- Missing hyper-connection ops in the HIP backend: the build is after the commit that added them.
- The integrated GPU and its unified memory: it reproduces on the discrete card alone.
- A fixed VRAM ceiling: 30.7 GB wedged, well above the roughly 20 GB setups that load fine.
- A damaged download: the same files load and serve under Vulkan.
(The CPU figure is a container figure, because the reporter runs llama-server in Docker. On bare metal you'd be looking for one busy core out of however many you have.)
- llama.cpp merged qwen4exp support
- hc ops added to the HIP backend
Nobody is assigned to the issue, and it carries no labels.