Error messages

RuntimeError: CUDA error: no kernel image is available for execution on the device

RuntimeError: CUDA error: no kernel image is available for execution on the device

vLLM gets as far as loading the model, or even the first request, and then:

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

That's from issue #629, one of 90 in the vLLM repo quoting it as of September 26, 2026. The first line is Nvidia's own wording, passed up through PyTorch. The lines under it are PyTorch's standard advice for any CUDA error and don't tell you anything specific here.

What a kernel image is

A kernel is a piece of code that runs on the graphics card. Nvidia cards come in generations, each with a compute capability number (7.5 for a T4 or RTX 20-series, higher for newer ones), and GPU code has to be compiled for each generation it should run on. A build of vLLM or PyTorch carries a fixed list of them. When your card's generation isn't on that list, the driver finds no matching image and returns this error.

vLLM 0.30.0 ships precompiled with CUDA 12.9 (builds for 12.8 and 13.0 exist too), and its install page says the card needs "compute capability 7.5 or higher".

Three ways to end up here

The card is older than vLLM supports. Anything below 7.5 isn't covered by the prebuilt packages at all. Building from source may get further, but you're outside what the project tests.

The card is newer than the build. When the RTX 5080 and 5090 came out, the prebuilt vLLM couldn't drive them yet. A vLLM team member's guide in #14452 (March 2025, 139 comments) walked people through building it themselves, because "we can't use precompiled vLLM because vllm-project/vllm has not moved to the required torch and CUDA versions yet." The install docs now say Blackwell cards need CUDA 12.8 at least. Any brand-new generation can go through the same gap.

vLLM was built on a different machine. A source build compiles for the GPU it was built next to unless you say otherwise. In #629 one fix was "recompiling and reinstalling the lib when deploying on V100. Previously it was compiled on A100."

How to check and fix it

Ask PyTorch what you have and what it was built for, using the same Python that runs vLLM:

python -c "import torch; print(torch.cuda.get_device_capability(), torch.cuda.get_arch_list())"

The first part is your card, such as (8, 6). The second is the list of sm_ generations PyTorch was compiled for. If your card is missing from PyTorch's list, the fix starts with installing a PyTorch built for a newer CUDA. vLLM's docs recommend uv pip install vllm --torch-backend=auto, which picks a PyTorch build to match your driver.

If PyTorch is fine and vLLM still fails, the compiled part of vLLM is the mismatch. The install page warns that its kernels are tied to specific CUDA and PyTorch versions and says to install into a fresh environment, or build from source when you need a different CUDA or an existing PyTorch. For a source build meant for another card, set the architecture list explicitly (the docs use TORCH_CUDA_ARCH_LIST and a torch_cuda_arch_list build argument for Docker).