Error messages

failed to create context with model

failed to create context with model

The line people search for is the last one llama.cpp prints before it quits, and it doesn't say much. Here's a full run of it from #24790:

E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 15296.00 MiB on device 0: cudaMalloc failed: out of memory
E alloc_tensor_range: failed to allocate CUDA0 buffer of size 16039018496
E llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache
E common_init_result: failed to create context with model '/home/ubuntu/LLM/Qwen3.6-27B-Q3_K_M.gguf'
E common_init_from_params: failed to create context with model '/home/ubuntu/LLM/Qwen3.6-27B-Q3_K_M.gguf'

By this point the weights have loaded. What failed is the context: the working memory for one run, mostly the KV cache (llama.cpp's store of every token it has already read, which grows with the context length). We read the code in release v0.5.0 (September 23, 2026). The llama_init_from_model line catches whatever went wrong while building the context and prints it after failed to initialize the context:, and the lines below it only repeat that it failed. In v0.5.0 those repeats print as cmn common_init_: with the function name cut to 12 characters. As of September 26, 2026, 128 issues in the llama.cpp repo contain "failed to create context".

Failed to allocate: the context doesn't fit

Most reasons after the colon are allocation failures: failed to allocate buffer for kv cache, or failed to allocate compute pp buffers for the scratch space used while reading a prompt. Look at the size in the lines above it. In the log above that's 15296 MiB for the cache alone, because n_ctx_seq (244736) earlier in the same log shows the context set to almost the model's full 262,144 tokens.

That's what you get when you don't choose. -c (--ctx-size) defaults to 0, which means the length the model was trained with, and for recent models that's huge. The cache grows with every token of context you ask for, so a smaller number like -c 16384 shrinks it.

The other half is what's already on the card. In that same issue another user pointed out that the reporter's nvidia-smi output showed 21956 MiB of 24576 MiB already in use on GPU 0 before llama.cpp started. Whatever else is running on the GPU keeps that memory, and llama.cpp can't use it.

Reasons that aren't about memory

A few messages mean a setting clashes with your build or model. quantized V cache was requested, but this requires Flash Attention (#28524) comes from --cache-type-v q4_0 with Flash Attention off. A contributor replied there that on CUDA, by default, only matching q4_0 or q8_0 types for both K and V are supported, so set -fa on and give --cache-type-k the same type. failed to initialize CUDA0 backend (or another device name) means the GPU backend itself wouldn't start, before any memory was asked for.

If the reason ends requires ctx_other to be set (this warning is normal during memory fitting), it's a draft model probed during automatic memory fitting. The reporter of #24343 found it went away with -fit off.

What to do

• Scroll up to failed to initialize the context: and read what follows. That's the line to search for or paste into an issue.
• For allocation failures, pass -c with the length you actually need, check nvidia-smi (or your GPU's equivalent) for other processes, and offload fewer layers with -ngl if the weights and cache won't share the card.
• Leave -fit on unless it's the cause. When you don't set -c, it's allowed to cut the context down (to no less than 4096 tokens) so the run fits.