Error messages

Bfloat16 is only supported on GPUs with compute capability of at least 8.0

Bfloat16 is only supported on GPUs with compute capability of at least 8.0

vLLM raises this while its GPU worker starts up, before any weights load. The whole line in vLLM 0.30.0 reads like this on a T4:

ValueError: Bfloat16 is only supported on GPUs with compute capability of at least 8.0. Your Tesla T4 GPU has compute capability 7.5. You can use float16 instead by explicitly setting the `dtype` flag in CLI, for example: --dtype=half.

Bfloat16 (bf16) is a 16-bit number format many recent models ship their weights in. Compute capability is NVIDIA's version number for a GPU's feature set, and 8.0 is the Ampere generation. Cards below it, like the Tesla T4 at 7.5 or the V100 at 7.0 (both named in reports on the vLLM tracker), can't do bf16 math, so vLLM refuses up front. As of September 26, 2026, 24 issues and pull requests in vllm-project/vllm quote the message.

Why vLLM picked bfloat16 on a card that can't run it

We read the code at the v0.30.0 tag. The --dtype setting defaults to auto, and on an older card auto doesn't raise this error at all: when the model's config asks for bf16 and the GPU can't do it, vLLM logs a warning ("Your device ... doesn't support ... Falling back to ...") and uses float16 instead.

So on 0.30.0 the error means something asked for bfloat16 by name. That can be --dtype bfloat16 on the command line, dtype="bfloat16" in the Python LLM() call, or a launch script or container command you copied that sets it. An explicit value skips the fallback, and vLLM does what it was told until the worker checks the card. The error text itself comes from vllm/platforms/cuda.py, with an identical copy for AMD cards in rocm.py.

Older releases behaved differently: the 2023 reports on a T4 (#1157) and a V100 (#946) come from a time when bf16 models on those cards hit the wall without anyone asking. If you're on an old version, updating may be enough.

Which dtype to use instead

The message suggests --dtype=half, which is float16, and that works for most models. float16 is accepted as the same thing. In Python it's LLM(model=..., dtype="half").

A few model families won't take it. vLLM 0.30.0 keeps a short list of model types it refuses to run in float16 because of numerical instability: gemma2, gemma3, gemma3_text and glm4. Ask for half on one of those and you trade this error for another one telling you to use bfloat16 or float32. On a pre-Ampere card that leaves --dtype float32 (or float), which is 32-bit and needs roughly twice the GPU memory of the 16-bit formats for the same weights.

Float32 is also what a vLLM maintainer, DarkLight1337, pointed a T4 user to in November 2024 (#9990) when they didn't want half precision: "Then I guess you should run the model with --dtype float." The same reply explains the error in one line: "This means your GPU is too old to support this dtype."

Why vLLM doesn't switch for you when you ask for bf16

Someone asked for exactly that in 2023 (#1144). A commenter objected that "a lot of models work fine with bfloat16 and break completely with float16", so the swap should be the user's call. That's where it stayed: vLLM falls back quietly only when you left the choice to auto. If you named bf16, it stops and tells you.

What doesn't fix it

Nothing in the environment changes the answer, because vLLM asks the card itself. A newer driver, CUDA version or PyTorch build won't move a T4 to 8.0. On a cloud machine, the other route is an instance whose GPU reports compute capability 8.0 or higher, where bf16 runs as asked.