ThinkFacility
  1. Can I run it locally?
  2. DeepSeek

DeepSeek VRAM requirements: can I run DeepSeek V4 Flash locally?

DeepSeek V4 Flash is 155 GB at UD-Q4_K_XL, and its smallest file is UD-IQ1_S (82.5 GB). Here's which machines hold each DeepSeek model, down to the R1 distills that fit a gaming PC.

Check your machine

Won't fit 156 GB

Weights, cache and overhead add up to more than the card and the RAM together.

  • Weights 155 GB
  • Context cache not computed
  • Overhead 0.8 GB
  • Fast memory
  • With system RAM

What we found

DeepSeek's current models are big. DeepSeek V4 Flash 0731 has 284 billion parameters. The files that fit a 128 GB Mac or a DGX Spark are the 1-bit and 2-bit ones, starting at UD-IQ1_S (82.5 GB). At 4-bit you're looking at a 256 GB Mac.

Those V4 totals leave the context cache out, and I'd rather say so than guess. V4 compresses its attention cache in a way its config describes with per-layer compression ratios, and we haven't matched that to how llama.cpp stores it. DeepSeek's own figure points small: its V4.1 Flash announcement puts that model at 890 bytes a token and calls it about a quarter of V4 Flash.

DeepSeek V4 Pro 0813 is past any single desktop. Its smallest file is UD-Q4_K_XL (850 GB), bigger than a 512 GB Mac.

Most of the smaller DeepSeek downloads are the R1 distills from January 2025: Qwen and Llama models fine-tuned on R1's reasoning, which run exactly like the models they're built on. DeepSeek R1 Distill Qwen 14B at 16.2 GB is the one for a 16 GB card.

DeepSeek V4 Flash 0731

284 billion parameters · mixture of experts, 6 of 256 active · up to 1M context · 198 thousand downloads in 30 days

DeepSeek V4 Flash 0731 needs about 156 GB at UD-Q4_K_XL before its context cache: the file is 155 GB and llama.cpp wants roughly 0.8 GB more to run it. No single consumer card holds it. A Mac with 256 GB of unified memory does, and a PC can split it between a GPU and system RAM. It's a mixture-of-experts model (6 of 256 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

We don't compute the context cache for the deepseek_v4 architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S82.5 GB83.3 GB83.3 GB96 GB96 GB*
UD-Q2_K_XL96.8 GB97.6 GB97.6 GB96 GB96 GB*
UD-Q3_K_XL128 GB129 GB129 GBnone128 GB*
UD-Q4_K_XL155 GB156 GB156 GBnone256 GB
UD-Q8_K_XL162 GB163 GB163 GBnone256 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek V4 Pro 0813

1.57 trillion parameters · mixture of experts, 6 of 384 active · up to 1M context · 138 thousand downloads in 30 days

DeepSeek V4 Pro 0813 needs about 850 GB at UD-Q4_K_XL before its context cache: the file is 850 GB and llama.cpp wants roughly 0.8 GB more to run it. That's past a 512 GB Mac. Even the smallest file, UD-Q4_K_XL, is 850 GB, so this one is for servers with several GPUs, or a workstation with a lot of system RAM and patience. It routes each token through 6 of 384 experts, so it runs faster than its size suggests.

We don't compute the context cache for the deepseek_v4 architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q4_K_XL850 GB850 GB850 GBnonenone
UD-Q8_K_XL873 GB874 GB874 GBnonenone

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

DeepSeek R1 Distill Qwen 14B

14.8 billion parameters · up to 128K context · 48 thousand downloads in 30 days

DeepSeek R1 Distill Qwen 14B needs about 16.2 GB at Q4_K_M with a 32K-token context. The file itself is 9.0 GB, and the cache for 32K tokens adds 6.4 GB. That fits a 16 GB card (RTX 4060 Ti 16 GB, 5070 Ti, 5080). On a Mac, 24 GB of unified memory is enough.

Every token of context adds 192 KB of cache, so the full 128K would put 25.8 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
Q2_K5.8 GB8.2 GB13.0 GB16 GB24 GB
Q3_K_M7.3 GB9.8 GB14.6 GB16 GB24 GB
Q4_K_M9.0 GB11.4 GB16.2 GB16 GB24 GB
Q5_K_M10.5 GB12.9 GB17.8 GB24 GB24 GB*
Q6_K12.1 GB14.5 GB19.4 GB24 GB24 GB*
Q8_015.7 GB18.1 GB23.0 GB24 GB32 GB*
F1629.5 GB32.0 GB36.8 GB48 GB48 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek R1

671 billion parameters · mixture of experts, 8 of 256 active · up to 160K context · 29 thousand downloads in 30 days

At Q4_K_M, the DeepSeek R1 file is 404 GB, and with 32K tokens of context and runtime overhead it comes to about 407 GB. No single consumer card holds it. A Mac with 512 GB of unified memory does, and a PC can split it between a GPU and system RAM. It routes each token through 8 of 256 experts, so it runs faster than its size suggests.

It caches a compressed latent per token rather than full keys and values, about 69 KB a token, so the cache is small next to the weights: 11.5 GB at 160K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S140 GB142 GB143 GBnone256 GB
UD-Q2_K_XL227 GB228 GB230 GBnone256 GB*
Q3_K_M319 GB321 GB322 GBnone512 GB
Q4_K_M404 GB406 GB407 GBnone512 GB
Q5_K_M475 GB477 GB478 GBnone512 GB*
Q6_K551 GB552 GB554 GBnonenone
Q8_0713 GB715 GB716 GBnonenone
BF161,342 GB1,344 GB1,345 GBnonenone

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek R1 Distill Llama 70B

70.6 billion parameters · up to 128K context · 138 thousand downloads in 30 days

DeepSeek R1 Distill Llama 70B needs about 54.0 GB at Q4_K_M with a 32K-token context. The file itself is 42.5 GB, and the cache for 32K tokens adds 10.7 GB. That fits a RTX PRO 6000 (96 GB).

Every token of context adds 320 KB of cache, so the full 128K would put 42.9 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S15.9 GB19.3 GB27.4 GB32 GB32 GB*
UD-Q2_K_XL27.0 GB30.4 GB38.5 GB48 GB48 GB
UD-Q3_K_XL34.9 GB38.3 GB46.4 GB48 GB48 GB*
Q4_K_M42.5 GB46.0 GB54.0 GB96 GB64 GB*
Q5_K_M49.9 GB53.4 GB61.4 GB96 GB64 GB*
Q6_K116 GB119 GB127 GBnone128 GB*
Q8_075.0 GB78.4 GB86.5 GB96 GB96 GB*
BF16141 GB145 GB153 GBnone256 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek R1 Distill Qwen 1.5B

1.8 billion parameters · up to 128K context · 58 thousand downloads in 30 days

At Q4_K_M, the DeepSeek R1 Distill Qwen 1.5B file is 1.1 GB, and with 32K tokens of context and runtime overhead it comes to about 2.9 GB. Even a 8 GB card (RTX 4060, RTX 3070) holds that. On a Mac, 16 GB of unified memory is enough.

The cache is the part that grows: 28 KB a token, or 3.8 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S0.7 GB1.7 GB2.5 GB8 GB16 GB
UD-Q2_K_XL0.9 GB1.9 GB2.6 GB8 GB16 GB
UD-Q3_K_XL1.0 GB2.0 GB2.7 GB8 GB16 GB
Q4_K_M1.1 GB2.2 GB2.9 GB8 GB16 GB
Q5_K_M1.3 GB2.3 GB3.0 GB8 GB16 GB
Q6_K1.5 GB2.5 GB3.2 GB8 GB16 GB
Q8_01.9 GB2.9 GB3.6 GB8 GB16 GB
BF163.6 GB4.6 GB5.3 GB8 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

DeepSeek R1 0528 Qwen3 8B

8.2 billion parameters · up to 128K context · 57 thousand downloads in 30 days

DeepSeek R1 0528 Qwen3 8B needs about 10.7 GB at Q4_K_M with a 32K-token context. The file itself is 5.0 GB, and the cache for 32K tokens adds 4.8 GB. That fits a RTX 3060 12 GB. On a Mac, 16 GB of unified memory is enough.

Every token of context adds 144 KB of cache, so the full 128K would put 19.3 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S2.3 GB4.3 GB7.9 GB8 GB16 GB
UD-Q2_K_XL3.5 GB5.5 GB9.1 GB12 GB16 GB
UD-Q3_K_XL4.3 GB6.3 GB10.0 GB12 GB16 GB
Q4_K_M5.0 GB7.0 GB10.7 GB12 GB16 GB
Q5_K_M5.9 GB7.9 GB11.5 GB12 GB16 GB*
Q6_K6.7 GB8.7 GB12.4 GB12 GB16 GB*
Q8_08.7 GB10.7 GB14.4 GB16 GB24 GB
BF1616.4 GB18.4 GB22.0 GB24 GB32 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek R1 Distill Qwen 32B

32.8 billion parameters · up to 128K context · 56 thousand downloads in 30 days

At Q4_K_M, the DeepSeek R1 Distill Qwen 32B file is 19.9 GB, and with 32K tokens of context and runtime overhead it comes to about 29.3 GB. The smallest card that holds it whole is a RTX 5090 (32 GB).

The cache is the part that grows: 256 KB a token, or 34.4 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
Q2_K12.3 GB15.3 GB21.7 GB24 GB32 GB
Q3_K_M15.9 GB18.9 GB25.3 GB24 GB32 GB*
Q4_K_M19.9 GB22.8 GB29.3 GB32 GB32 GB*
Q5_K_M23.3 GB26.2 GB32.7 GB32 GB36 GB*
Q6_K26.9 GB29.8 GB36.3 GB48 GB48 GB
Q8_034.8 GB37.8 GB44.2 GB48 GB48 GB*
F1665.5 GB68.5 GB74.9 GB96 GB96 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek R1 Distill Llama 8B

8.0 billion parameters · up to 128K context · 43 thousand downloads in 30 days

DeepSeek R1 Distill Llama 8B needs about 10.0 GB at Q4_K_M with a 32K-token context. The file itself is 4.9 GB, and the cache for 32K tokens adds 4.3 GB. That fits a RTX 3060 12 GB. On a Mac, 16 GB of unified memory is enough.

Every token of context adds 128 KB of cache, so the full 128K would put 17.2 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S2.2 GB4.0 GB7.2 GB8 GB16 GB
UD-Q2_K_XL3.4 GB5.2 GB8.4 GB8 GB16 GB
UD-Q3_K_XL4.2 GB6.0 GB9.3 GB12 GB16 GB
Q4_K_M4.9 GB6.8 GB10.0 GB12 GB16 GB
Q5_K_M5.7 GB7.6 GB10.8 GB12 GB16 GB
Q6_K6.6 GB8.4 GB11.7 GB12 GB16 GB*
Q8_08.5 GB10.4 GB13.6 GB16 GB24 GB
BF1616.1 GB17.9 GB21.1 GB24 GB24 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

DeepSeek R1 Distill Qwen 7B

7.6 billion parameters · up to 128K context · 29 thousand downloads in 30 days

At Q4_K_M, the DeepSeek R1 Distill Qwen 7B file is 4.7 GB, and with 32K tokens of context and runtime overhead it comes to about 7.4 GB. Even a 8 GB card (RTX 4060, RTX 3070) holds that. On a Mac, 16 GB of unified memory is enough.

The cache is the part that grows: 56 KB a token, or 7.5 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
Q2_K3.0 GB4.3 GB5.7 GB8 GB16 GB
Q3_K_M3.8 GB5.1 GB6.5 GB8 GB16 GB
Q4_K_M4.7 GB6.0 GB7.4 GB8 GB16 GB
Q5_K_M5.4 GB6.7 GB8.1 GB8 GB16 GB
Q6_K6.3 GB7.5 GB8.9 GB12 GB16 GB
Q8_08.1 GB9.4 GB10.8 GB12 GB16 GB
F1615.2 GB16.5 GB17.9 GB24 GB24 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked September 25, 2026.