- Can I run it locally?
- Other open models
Phi, Nemotron, Step and LFM VRAM requirements: can I run them locally?
Models from labs with one or two popular downloads share this page. A new model from a lab without its own page shows up here the day it passes our download line.
Check your machine
Fits on the card 7.7 GB
Everything stays in video memory, which is the fast case.
- Weights 2.5 GB
- Context cache 4.3 GB
- Overhead 0.9 GB
- Fast memory
- With system RAM
What we found
The architectures here vary more than on any family page. Nemotron 3.5 Lightning 30B A3B has attention in only 6 of its 52 layers. Another 23 are Mamba layers, which keep a fixed-size state, so a million-token context costs 6.5 GB of cache.
Phi 4 mini goes the other way. Its 4-bit file is 2.5 GB, and a 32K context adds 4.3 GB of cache, more than the model itself.
Step 3.7 Flash from StepFun is the big one, 125 GB at 4-bit with a 32K context. A DGX Spark holds it, and so does a 128 GB Mac with its GPU limit raised.
Hardware that runs it
The biggest model on this page each machine holds whole with a 32K context, at the best file that fits. Which one to buy is in our GPU guide for local LLMs.
As an Amazon Associate we earn from qualifying purchases.
| Machine | Runs | Check price on Amazon |
|---|---|---|
| 16 GB graphics cardGeForce RTX 5060 Ti 16 GB, GeForce RTX 5070 Ti, GeForce RTX 5080 | LFM2.5 24B A2B UD-Q4_K_XLand 6 more of the 14 here | |
| 24 GB graphics cardRadeon RX 7900 XTX, Intel Arc Pro B60 | North Mini Code 1.0 UD-Q5_K_XLand 9 more of the 14 here | |
| 32 GB graphics cardGeForce RTX 5090, Radeon AI PRO R9700 | flint UD-Q5_K_XLand 9 more of the 14 here | |
| 48 GB workstation cardRTX PRO 5000 Blackwell | Nemotron 3.5 Bolt 74B A9B UD-Q4_K_XLand 11 more of the 14 here | |
| 96 GB workstation cardRTX PRO 6000 Blackwell Workstation Edition | Ornith 1.5 122B UD-Q5_K_XLand 12 more of the 14 here | |
| Mac with 64 GB of unified memoryMac mini (M5 Pro, 64 GB), Mac Studio (M5 Max, 64 GB) | Nemotron 3.5 Bolt 74B A9B UD-Q4_K_XLand 11 more of the 14 here | |
| Mac with 96 GB of unified memoryMac Studio (M5 Ultra, 96 GB) | Nemotron 3.5 Bolt 74B A9B UD-Q6_K_XLand 12 more of the 14 here | |
| Mac with 128 GB of unified memoryMac Studio (M5 Max, 128 GB), MacBook Pro 14 (M5 Max, 128 GB) | Ornith 1.5 122B UD-Q5_K_XLand 12 more of the 14 here | |
| Mac Studio with 256 GBMac Studio (M5 Ultra, 256 GB) | Step 3.7 Flash UD-Q6_K_XLand 13 more of the 14 here | |
| Mac Studio with 512 GBMac Studio (M5 Ultra, 512 GB) | MiniMax M3.1 Flash UD-Q8_K_XLand 13 more of the 14 here | |
| Ryzen AI Max+ 395 mini PC with 128 GBGMKtec EVO-X2, Beelink GTR9 Pro, HP Z2 Mini G1a | Ornith 1.5 122B UD-Q5_K_XLand 12 more of the 14 here | |
| DGX Spark or another GB10 box, 128 GBNVIDIA DGX Spark, ASUS Ascent GX10 | Step 3.7 Flash UD-Q4_K_Mand 13 more of the 14 here |
Phi 4 mini
3.8 billion parameters · up to 128K context · 119 thousand downloads in 30 days
Phi 4 mini needs about 7.7 GB at Q4_K_M with a 32K-token context. The file itself is 2.5 GB, and the cache for 32K tokens adds 4.3 GB. That fits a 8 GB card (RTX 4060, RTX 3070, RX 7600). On a Mac, 16 GB of unified memory is enough.
Every token of context adds 128 KB of cache, so the full 128K would put 17.2 GB on top of the weights.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
Q2_K | 1.7 GB | 3.7 GB | 6.9 GB | 8 GB | 16 GB |
Q3_K_M | 2.1 GB | 4.1 GB | 7.3 GB | 8 GB | 16 GB |
Q4_K_M | 2.5 GB | 4.5 GB | 7.7 GB | 8 GB | 16 GB |
Q5_K_M | 2.8 GB | 4.8 GB | 8.1 GB | 8 GB | 16 GB |
Q6_K | 3.2 GB | 5.1 GB | 8.4 GB | 8 GB | 16 GB |
Q8_0 | 4.1 GB | 6.1 GB | 9.3 GB | 12 GB | 16 GB |
BF16 | 7.7 GB | 9.7 GB | 12.9 GB | 16 GB | 24 GB |
Smallest GPU and Mac are for the 32K total.
Sources: GGUF files · config.json
Nemotron 3.5 Lightning 30B A3B
32.9 billion parameters · mixture of experts, 6 of 128 active · up to 1M context · 56 thousand downloads in 30 days
At UD-Q4_K_M, the Nemotron 3.5 Lightning 30B A3B file is 25.3 GB, and with 32K tokens of context and runtime overhead it comes to about 26.3 GB. The smallest card that holds it whole is a 32 GB card (RTX 5090, Radeon AI PRO R9700). It's a mixture-of-experts model (6 of 128 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.
Most of its layers use linear attention, which keeps a fixed state instead of a cache, so the cache grows by about 6 KB a token and reaches 6.5 GB at 1M.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 19.4 GB | 20.3 GB | 20.4 GB | 20 GB | 24 GB* |
UD-IQ2_M | 19.4 GB | 20.3 GB | 20.5 GB | 20 GB | 24 GB* |
UD-Q3_K_XL | 21.2 GB | 22.1 GB | 22.3 GB | 24 GB | 32 GB |
UD-Q4_K_M | 25.3 GB | 26.1 GB | 26.3 GB | 32 GB | 32 GB* |
UD-Q5_K_XL | 30.4 GB | 31.3 GB | 31.4 GB | 32 GB | 36 GB* |
UD-Q6_K_XL | 35.0 GB | 35.9 GB | 36.0 GB | 48 GB | 48 GB |
Q8_0 | 35.0 GB | 35.9 GB | 36.0 GB | 48 GB | 48 GB |
BF16 | 65.9 GB | 66.7 GB | 66.9 GB | 96 GB | 96 GB |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
LFM2.5 1.2B
1.2 billion parameters · up to 125K context · 36 thousand downloads in 30 days
LFM2.5 1.2B needs about 1.8 GB at Q4_K_M with a 32K-token context. The file itself is 0.7 GB, and the cache for 32K tokens adds 0.4 GB. That fits a 8 GB card (RTX 4060, RTX 3070, RX 7600). On a Mac, 16 GB of unified memory is enough.
Only 6 of its 16 layers keep a growing cache (the rest carry a fixed-size state), so the full 125K context costs 1.6 GB of cache.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-Q2_K_XL | 0.5 GB | 1.2 GB | 1.5 GB | 8 GB | 16 GB |
UD-Q3_K_XL | 0.6 GB | 1.3 GB | 1.6 GB | 8 GB | 16 GB |
Q4_K_M | 0.7 GB | 1.5 GB | 1.8 GB | 8 GB | 16 GB |
Q5_K_M | 0.8 GB | 1.6 GB | 1.9 GB | 8 GB | 16 GB |
Q6_K | 1.0 GB | 1.7 GB | 2.0 GB | 8 GB | 16 GB |
Q8_0 | 1.2 GB | 2.0 GB | 2.3 GB | 8 GB | 16 GB |
BF16 | 2.3 GB | 3.1 GB | 3.4 GB | 8 GB | 16 GB |
Smallest GPU and Mac are for the 32K total.
Sources: GGUF files · config.json
Step 3.7 Flash
197 billion parameters · mixture of experts, 8 of 288 active · up to 256K context · 10 thousand downloads in 30 days
At UD-Q4_K_M, the Step 3.7 Flash file is 122 GB, and with 32K tokens of context and runtime overhead it comes to about 125 GB. No single consumer card holds it. A Mac with 128 GB of unified memory does once you raise the GPU memory limit, and a PC can split it between a GPU and system RAM. It's a mixture-of-experts model (8 of 288 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.
A sliding window of 512 tokens on 33 layers keeps the cache small: 13.0 GB at the full 256K.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 56.8 GB | 58.1 GB | 59.3 GB | 96 GB | 64 GB* |
UD-Q2_K_XL | 65.8 GB | 67.1 GB | 68.3 GB | 96 GB | 96 GB |
UD-Q3_K_XL | 89.4 GB | 90.7 GB | 91.9 GB | 96 GB | 96 GB* |
UD-Q4_K_M | 122 GB | 123 GB | 125 GB | none | 128 GB* |
UD-Q5_K_XL | 146 GB | 148 GB | 149 GB | none | 256 GB |
UD-Q6_K_XL | 179 GB | 180 GB | 181 GB | none | 256 GB |
Q8_0 | 209 GB | 211 GB | 212 GB | none | 256 GB* |
BF16 | 394 GB | 395 GB | 397 GB | none | 512 GB |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
Ornith 1.5 122B
125 billion parameters · mixture of experts, 8 of 256 active · up to 256K context · 593 thousand downloads in 30 days
Ornith 1.5 122B needs about 81.7 GB at UD-Q4_K_XL with a 32K-token context. The file itself is 79.7 GB, and the cache for 32K tokens adds 1.0 GB. That fits a RTX PRO 6000 (96 GB). It's a mixture-of-experts model (8 of 256 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.
Only 12 of its 48 layers keep a growing cache (the rest carry a fixed-size state), so the full 256K context costs 6.6 GB of cache.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 40.4 GB | 41.7 GB | 42.3 GB | 48 GB | 48 GB* |
UD-Q2_K_XL | 44.5 GB | 45.9 GB | 46.5 GB | 48 GB | 48 GB* |
UD-Q3_K_XL | 59.8 GB | 61.1 GB | 61.7 GB | 96 GB | 64 GB* |
UD-Q4_K_XL | 79.7 GB | 81.1 GB | 81.7 GB | 96 GB | 96 GB* |
UD-Q5_K_XL | 94.6 GB | 96.0 GB | 96.6 GB | 96 GB | 96 GB* |
UD-Q6_K_XL | 114 GB | 116 GB | 116 GB | none | 128 GB* |
Q8_0 | 133 GB | 134 GB | 135 GB | none | 256 GB |
BF16 | 249 GB | 251 GB | 251 GB | none | 256 GB* |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
Nemotron 3.5 Bolt E4B
7.5 billion parameters · up to 128K context · 481 thousand downloads in 30 days
At UD-Q4_K_XL, the Nemotron 3.5 Bolt E4B file is 5.1 GB, and with 32K tokens of context and runtime overhead it comes to about 6.7 GB. Even a 8 GB card (RTX 4060, RTX 3070, RX 7600) holds that. On a Mac, 16 GB of unified memory is enough.
A sliding window of 512 tokens on 20 layers keeps the cache small: 2.2 GB at the full 128K.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-Q2_K_XL | 3.7 GB | 5.0 GB | 5.4 GB | 8 GB | 16 GB |
UD-Q3_K_XL | 4.2 GB | 5.4 GB | 5.8 GB | 8 GB | 16 GB |
UD-Q4_K_XL | 5.1 GB | 6.3 GB | 6.7 GB | 8 GB | 16 GB |
UD-Q5_K_XL | 5.8 GB | 7.0 GB | 7.4 GB | 8 GB | 16 GB |
UD-Q6_K_XL | 7.3 GB | 8.5 GB | 8.9 GB | 12 GB | 16 GB |
UD-Q8_K_XL | 8.2 GB | 9.4 GB | 9.8 GB | 12 GB | 16 GB |
BF16 | 15.1 GB | 16.3 GB | 16.7 GB | 16 GB | 24 GB |
Smallest GPU and Mac are for the 32K total.
Sources: GGUF files · config.json
LFM2.5 24B A2B
24.0 billion parameters · mixture of experts, 4 of 64 active · up to 128K context · 430 thousand downloads in 30 days
LFM2.5 24B A2B needs about 16.3 GB at UD-Q4_K_XL with a 32K-token context. The file itself is 14.9 GB, and the cache for 32K tokens adds 0.7 GB. That fits a 16 GB card (RTX 4060 Ti 16 GB, 5070 Ti, 5080, RX 9070 XT). On a Mac, 24 GB of unified memory is enough. It routes each token through 4 of 64 experts, so it runs faster than its size suggests.
Only 10 of its 40 layers keep a growing cache (the rest carry a fixed-size state), so the full 128K context costs 2.7 GB of cache.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-Q2_K_XL | 8.0 GB | 9.0 GB | 9.5 GB | 12 GB | 16 GB |
UD-Q3_K_XL | 10.9 GB | 11.9 GB | 12.4 GB | 12 GB | 16 GB* |
UD-Q4_K_XL | 14.9 GB | 15.8 GB | 16.3 GB | 16 GB | 24 GB |
UD-Q5_K_XL | 17.9 GB | 18.8 GB | 19.4 GB | 20 GB | 24 GB* |
UD-Q6_K_XL | 21.9 GB | 22.8 GB | 23.3 GB | 24 GB | 32 GB* |
Q8_0 | 25.5 GB | 26.4 GB | 26.9 GB | 32 GB | 32 GB* |
BF16 | 48.0 GB | 48.9 GB | 49.4 GB | 48 GB | 64 GB |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
Ornith 1.0 9B
9.0 billion parameters · up to 256K context · 418 thousand downloads in 30 days
At Q4_K_M, the Ornith 1.0 9B file is 5.7 GB, and with 32K tokens of context and runtime overhead it comes to about 7.8 GB. Even a 8 GB card (RTX 4060, RTX 3070, RX 7600) holds that. On a Mac, 16 GB of unified memory is enough.
Most of its layers use linear attention, which keeps a fixed state instead of a cache, so the cache grows by about 32 KB a token and reaches 8.6 GB at 256K.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-Q2_K_XL | 4.3 GB | 5.7 GB | 6.5 GB | 8 GB | 16 GB |
UD-Q3_K_XL | 5.1 GB | 6.4 GB | 7.2 GB | 8 GB | 16 GB |
Q4_K_M | 5.7 GB | 7.0 GB | 7.8 GB | 8 GB | 16 GB |
Q5_K_M | 6.5 GB | 7.9 GB | 8.7 GB | 12 GB | 16 GB |
Q6_K | 7.5 GB | 8.8 GB | 9.6 GB | 12 GB | 16 GB |
Q8_0 | 9.5 GB | 10.9 GB | 11.7 GB | 12 GB | 16 GB* |
BF16 | 17.9 GB | 19.3 GB | 20.1 GB | 20 GB | 24 GB* |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
flint
39.8 billion parameters · mixture of experts, 6 of 256 active · up to 256K context · 308 thousand downloads in 30 days
flint needs about 25.9 GB at UD-Q4_K_XL before its context cache: the file is 25.1 GB and llama.cpp wants roughly 0.8 GB more to run it. That fits a 32 GB card (RTX 5090, Radeon AI PRO R9700). It's a mixture-of-experts model (6 of 256 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.
We don't compute the context cache for the flint architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 11.4 GB | 12.2 GB | 12.2 GB | 12 GB | 16 GB* |
UD-Q2_K_XL | 13.6 GB | 14.4 GB | 14.4 GB | 16 GB | 24 GB |
UD-Q3_K_XL | 18.5 GB | 19.3 GB | 19.3 GB | 20 GB | 24 GB* |
UD-Q4_K_XL | 25.1 GB | 25.9 GB | 25.9 GB | 32 GB | 32 GB* |
UD-Q5_K_XL | 29.9 GB | 30.7 GB | 30.7 GB | 32 GB | 36 GB* |
UD-Q6_K_XL | 36.3 GB | 37.1 GB | 37.1 GB | 48 GB | 48 GB |
Q8_0 | 42.3 GB | 43.1 GB | 43.1 GB | 48 GB | 48 GB* |
BF16 | 79.6 GB | 80.4 GB | 80.4 GB | 96 GB | 96 GB* |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
North Mini Code 1.0
30.5 billion parameters · mixture of experts, 8 of 128 active · up to 488K context · 233 thousand downloads in 30 days
At UD-Q4_K_M, the North Mini Code 1.0 file is 19.2 GB, and with 32K tokens of context and runtime overhead it comes to about 21.5 GB. The smallest card that holds it whole is a 20 GB card (RX 7900 XT). It routes each token through 8 of 128 experts, so it runs faster than its size suggests.
A sliding window of 4,096 tokens on 36 layers keeps the cache small: 13.7 GB at the full 488K.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 9.4 GB | 11.0 GB | 11.6 GB | 12 GB | 16 GB* |
UD-Q2_K_XL | 10.5 GB | 12.1 GB | 12.7 GB | 12 GB | 16 GB* |
UD-Q3_K_XL | 14.3 GB | 15.9 GB | 16.6 GB | 16 GB | 24 GB |
UD-Q4_K_M | 19.2 GB | 20.8 GB | 21.5 GB | 20 GB | 24 GB* |
UD-Q5_K_XL | 23.0 GB | 24.6 GB | 25.2 GB | 24 GB | 32 GB* |
UD-Q6_K_XL | 27.9 GB | 29.4 GB | 30.1 GB | 32 GB | 36 GB* |
Q8_0 | 32.4 GB | 34.0 GB | 34.7 GB | 48 GB | 48 GB |
BF16 | 61.0 GB | 62.6 GB | 63.3 GB | 96 GB | 64 GB* |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
Nemotron 3.5 Bolt 74B A9B
74.0 billion parameters · up to 256K context · 77 thousand downloads in 30 days
Nemotron 3.5 Bolt 74B A9B needs about 50.7 GB at UD-Q4_K_XL before its context cache: the file is 50.2 GB and llama.cpp wants roughly 0.5 GB more to run it. That fits a 48 GB (two 24 GB cards, Radeon PRO W7900). On a Mac, 64 GB of unified memory is enough.
We don't compute the context cache for the nemotron_h_omni architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_S | 18.5 GB | 19.0 GB | 19.0 GB | 20 GB | 24 GB* |
UD-Q2_K_XL | 28.8 GB | 29.3 GB | 29.3 GB | 32 GB | 32 GB* |
UD-Q3_K_XL | 39.0 GB | 39.5 GB | 39.5 GB | 48 GB | 48 GB* |
UD-Q4_K_XL | 50.2 GB | 50.7 GB | 50.7 GB | 48 GB | 64 GB |
UD-Q5_K_XL | 58.9 GB | 59.4 GB | 59.4 GB | 96 GB | 64 GB* |
UD-Q6_K_XL | 72.4 GB | 72.9 GB | 72.9 GB | 96 GB | 96 GB |
Q8_0 | 78.9 GB | 79.4 GB | 79.4 GB | 96 GB | 96 GB* |
BF16 | 148 GB | 149 GB | 149 GB | none | 256 GB |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
Nemotron 3.5 Super VL 120B A12B
1.6 billion parameters · up to 1M context · 48 thousand downloads in 30 days
Nemotron 3.5 Super VL 120B A12B needs about 87.4 GB at UD-Q4_K_XL before its context cache: the file is 86.9 GB and llama.cpp wants roughly 0.5 GB more to run it. The smallest card that holds it whole is a RTX PRO 6000 (96 GB).
We don't compute the context cache for the nemotron_h_omni architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 46.1 GB | 46.6 GB | 46.6 GB | 48 GB | 48 GB* |
UD-Q2_K_XL | 57.8 GB | 58.3 GB | 58.3 GB | 96 GB | 64 GB* |
UD-Q3_K_XL | 65.8 GB | 66.3 GB | 66.3 GB | 96 GB | 96 GB |
UD-Q4_K_XL | 86.9 GB | 87.4 GB | 87.4 GB | 96 GB | 96 GB* |
UD-Q5_K_XL | 111 GB | 111 GB | 111 GB | none | 128 GB* |
UD-Q6_K_XL | 121 GB | 121 GB | 121 GB | none | 128 GB* |
Q8_0 | 132 GB | 132 GB | 132 GB | none | 128 GB* |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
Nemotron 3 Nano 30B A3B
31.6 billion parameters · mixture of experts, 6 of 128 active · up to 1M context · 40 thousand downloads in 30 days
Nemotron 3 Nano 30B A3B needs about 27.1 GB at Q4_K_M with a 32K-token context. The file itself is 24.6 GB, and the cache for 32K tokens adds 1.7 GB. That fits a 32 GB card (RTX 5090, Radeon AI PRO R9700). It's a mixture-of-experts model (6 of 128 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.
Every token of context adds 52 KB of cache, so the full 1M would put 55.8 GB on top of the weights.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-Q2_K_XL | 19.9 GB | 21.1 GB | 22.4 GB | 24 GB | 32 GB |
UD-Q3_K_XL | 19.9 GB | 21.1 GB | 22.5 GB | 24 GB | 32 GB |
Q4_K_M | 24.6 GB | 25.8 GB | 27.1 GB | 32 GB | 32 GB* |
Q5_K_M | 26.1 GB | 27.4 GB | 28.7 GB | 32 GB | 32 GB* |
Q6_K | 33.5 GB | 34.7 GB | 36.0 GB | 48 GB | 48 GB |
Q8_0 | 33.6 GB | 34.8 GB | 36.1 GB | 48 GB | 48 GB |
BF16 | 63.2 GB | 64.4 GB | 65.7 GB | 96 GB | 96 GB |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
MiniMax M3.1 Flash
426 billion parameters · mixture of experts, 4 of 128 active · up to 1M context · 24 thousand downloads in 30 days
MiniMax M3.1 Flash needs about 250 GB at UD-Q4_K_XL before its context cache: the file is 249 GB and llama.cpp wants roughly 0.9 GB more to run it. No single consumer card holds it. A Mac with 256 GB of unified memory does once you raise the GPU memory limit, and a PC can split it between a GPU and system RAM. It routes each token through 4 of 128 experts, so it runs faster than its size suggests.
We don't compute the context cache for the minimax_m3_vl architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.
| Quant | File | At 8K | At 32K | Smallest GPU | Smallest Mac |
|---|---|---|---|---|---|
UD-IQ1_M | 129 GB | 130 GB | 130 GB | none | 128 GB* |
UD-Q2_K_XL | 144 GB | 145 GB | 145 GB | none | 256 GB |
UD-Q3_K_XL | 194 GB | 195 GB | 195 GB | none | 256 GB |
UD-Q4_K_XL | 249 GB | 250 GB | 250 GB | none | 256 GB* |
UD-Q8_K_XL | 259 GB | 260 GB | 260 GB | none | 256 GB* |
Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.
Sources: GGUF files · config.json
How the numbers are worked out
Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:
cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens
Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.
A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.
Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked November 10, 2026.