Error messages

on startup is less than desired GPU memory utilization

on startup is less than desired GPU memory utilization

vLLM checks memory before it loads anything, and stops with this if the numbers don't work. The example is from a DGX Spark run in vLLM's own test setup (pull request #58232, September 23, 2026):

ValueError: Free memory on device cuda:0 (83.35/121.7 GiB) on startup is less than
  desired GPU memory utilization (0.92, 111.96 GiB).

The full message ends with "Decrease GPU memory utilization or reduce GPU memory used by other processes." Versions from mid-2025 leave out the device name ("Free memory on device (31.44/47.38 GiB)"), and Intel GPUs show xpu:0. The phrase "on startup is less than desired GPU memory utilization" is in every version.

Reading the numbers

The first pair is free memory over total memory on that GPU, measured just before vLLM starts. The second pair is the --gpu-memory-utilization setting and what it comes to in GiB. We read the check in vLLM 0.30.0, and it's a plain comparison: the setting is a share of the card's total (0.92 by default), and if the card doesn't have that much free right now, vLLM won't start. In the example, 0.92 of 121.7 GiB is 111.96 GiB, but only 83.35 GiB was free.

The setting is a share of the whole card, which catches people out. At 0.92, vLLM needs the card almost empty, so a desktop session or another model server using a few GiB is enough to fail.

Fixing it

Run nvidia-smi to see what else holds memory. If it's something you can stop (an Ollama server, a notebook with a model still loaded, an older vLLM that didn't exit), stop it.

If it has to stay, lower the setting below the free fraction. With 20 GiB free on a 24 GiB card, that's under 0.83, so --gpu-memory-utilization 0.8. vLLM's own docs for the setting use the same approach for two instances sharing one GPU: 0.5 each. A forum post on discuss.vllm.ai from February 2026 fixed a 3090 this way with 0.5.

Lowering it also shrinks the memory vLLM keeps for the KV cache (the store for each request's context), so you may then need a smaller --max-model-len.

A machine with more than one GPU

vLLM checks the GPUs it's going to use, which by default starts at device 0. If device 0 is the busy one, choose the others with CUDA_VISIBLE_DEVICES, as a vLLM maintainer suggested in issue #28572 (November 2025) for a box with an RTX 2080 Ti next to 3090s. That reporter said vLLM still checked the 2080 Ti with the variable set, and the issue was closed as stale without a fix, so check the device name in the message after you set it.

DGX Spark and other shared-memory machines

On machines where the GPU and CPU share memory (DGX Spark, GH200, Jetson), vLLM 0.30.0 counts free memory as the RAM the operating system reports as available. The pull request above explains why that's never close to the total: "the OS and page cache hold tens of GiB at all times." The default of 0.92 fails there, and the fix in that pull request was to set a lower gpu_memory_utilization for the Spark config. Do the same.