Error messages

failed to initialize the context: failed to allocate buffer for kv cache

failed to initialize the context: failed to allocate buffer for kv cache

You'll see this in a llama.cpp log, or wrapped in LM Studio's "Failed to load model." banner, right after the weights have finished loading:

failed to initialize the context: failed to allocate buffer for kv cache

Both halves come from llama.cpp, which LM Studio and Ollama both run underneath. We read them in release v0.6.0 (October 5, 2026). The cache code throws "failed to allocate buffer for kv cache" when the GPU (or whichever device it's using) won't hand over a block that size, and the function that builds the context catches it and prefixes "failed to initialize the context". Older builds named the cache function instead, as llama_kv_cache_init: failed to allocate buffer for kv cache.

We have a broader page on “failed to create context with model”, the line llama-server prints after this one. This page is about the KV cache part, since that's the reason that turns up most.

What the KV cache is, and why it's so big

The KV cache is where the model keeps its working notes on every token in the context, so it doesn't redo them for each new word. Its size doesn't depend on how long your actual prompt is. llama.cpp sets the whole thing aside up front, for the full context length you asked for.

The size is simple multiplication: layers, times KV heads, times the size of each head, times two (one copy for keys, one for values), times bytes per number, times the context length. A llama.cpp user hit this in 2024 with Llama 3.1 8B on a 12 GB RTX 3060. That model has 32 layers, 8 KV heads and 128-wide heads, so at 16-bit precision each token costs 128 KiB. Their context was 131,072 tokens, the model's full window, and the log shows llama.cpp asking for exactly 16,384 MiB. That's more than the whole card, before the weights. They fixed it with -c 8192, which brings the cache down to 1 GB.

That's the usual story. A model's default context is often its maximum, and a long maximum is what blows the budget. Asked about it in October 2025, a llama.cpp maintainer put it plainly: "this is not a bug. You are running out of memory for the settings you are trying to use."

Fixes in llama.cpp

• Set a smaller context with -c (or --ctx-size). Halving it halves the cache.
• Quantize the cache with -ctk q8_0 -ctv q8_0, which roughly halves it again. A quantized V cache needs flash attention (-fa); llama.cpp refuses with "quantized V cache was requested, but this requires Flash Attention" if it's off.
• -nkvo (--no-kv-offload) keeps the cache in system RAM instead of VRAM. It costs speed (an LM Studio user measured about 12% with the equivalent setting, below).
• Offload fewer layers with a lower -ngl, which frees VRAM for the cache.
• Recent builds have -fit on, which the help text describes as adjusting "unset arguments to fit in device memory". It only changes what you haven't set yourself, so leave -c off when you try it.

One thing that doesn't help: lowering -np, the number of server slots. In llama.cpp the total from -c is shared among the slots, so more slots means a shorter context each, not a bigger cache. (Ollama works differently; see below.)

Fixes in LM Studio

LM Studio's main thread on this has been open since May 2025, and the settings that got people loading are the same ones under different names. Lowering Context Length in the model's load settings is the first reply. Two users with AMD cards confirmed that switching on Flash Attention fixed it. Another turned off "Offload KV cache to GPU memory" and wrote that it cost about 12% of their speed (75 tokens a second down to 66) but the model loaded.

If none of that works and the same GGUF file loads in plain llama.cpp, it may be LM Studio's loader. A report from August 2026 found two models that failed in the app even at small contexts while loading fine in the llama-server that ships with it; it's still open.

In Ollama

Ollama logs the same llama.cpp line in its server log, and one report from 2025 shows it with runner.parallel=2 runner.num_ctx=8192 just above. Unlike llama.cpp's slots, Ollama's parallel requests do add up: its FAQ says required memory "will scale by OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH". Lowering either one shrinks the cache. If the model won't load at all, our “unable to load model” page covers Ollama's other loading failures.