ThinkFacility Sign in
  1. Can I run it locally?
  2. Other open models

Phi, Nemotron, Step and LFM VRAM requirements: can I run them locally?

Models from labs with one or two popular downloads share this page. A new model from a lab without its own page shows up here the day it passes our download line.

Check your machine

Fits on the card 7.7 GB

Everything stays in video memory, which is the fast case.

  • Weights 2.5 GB
  • Context cache 4.3 GB
  • Overhead 0.9 GB
  • Fast memory
  • With system RAM

What we found

The architectures here vary more than on any family page. Nemotron 3.5 Lightning 30B A3B has attention in only 6 of its 52 layers. Another 23 are Mamba layers, which keep a fixed-size state, so a million-token context costs 6.5 GB of cache.

Phi 4 mini goes the other way. Its 4-bit file is 2.5 GB, and a 32K context adds 4.3 GB of cache, more than the model itself.

Step 3.7 Flash from StepFun is the big one, 125 GB at 4-bit with a 32K context. A DGX Spark holds it, and so does a 128 GB Mac with its GPU limit raised.

Hardware that runs it

The biggest model on this page each machine holds whole with a 32K context, at the best file that fits. Which one to buy is in our GPU guide for local LLMs.

As an Amazon Associate we earn from qualifying purchases.

MachineRuns Check price on Amazon
16 GB graphics cardGeForce RTX 5060 Ti 16 GB, GeForce RTX 5070 Ti, GeForce RTX 5080LFM2.5 24B A2B UD-Q4_K_XLand 6 more of the 14 here
24 GB graphics cardRadeon RX 7900 XTX, Intel Arc Pro B60North Mini Code 1.0 UD-Q5_K_XLand 9 more of the 14 here
32 GB graphics cardGeForce RTX 5090, Radeon AI PRO R9700flint UD-Q5_K_XLand 9 more of the 14 here
48 GB workstation cardRTX PRO 5000 BlackwellNemotron 3.5 Bolt 74B A9B UD-Q4_K_XLand 11 more of the 14 here
96 GB workstation cardRTX PRO 6000 Blackwell Workstation EditionOrnith 1.5 122B UD-Q5_K_XLand 12 more of the 14 here
Mac with 64 GB of unified memoryMac mini (M5 Pro, 64 GB), Mac Studio (M5 Max, 64 GB)Nemotron 3.5 Bolt 74B A9B UD-Q4_K_XLand 11 more of the 14 here
Mac with 96 GB of unified memoryMac Studio (M5 Ultra, 96 GB)Nemotron 3.5 Bolt 74B A9B UD-Q6_K_XLand 12 more of the 14 here
Mac with 128 GB of unified memoryMac Studio (M5 Max, 128 GB), MacBook Pro 14 (M5 Max, 128 GB)Ornith 1.5 122B UD-Q5_K_XLand 12 more of the 14 here
Mac Studio with 256 GBMac Studio (M5 Ultra, 256 GB)Step 3.7 Flash UD-Q6_K_XLand 13 more of the 14 here
Mac Studio with 512 GBMac Studio (M5 Ultra, 512 GB)MiniMax M3.1 Flash UD-Q8_K_XLand 13 more of the 14 here
Ryzen AI Max+ 395 mini PC with 128 GBGMKtec EVO-X2, Beelink GTR9 Pro, HP Z2 Mini G1aOrnith 1.5 122B UD-Q5_K_XLand 12 more of the 14 here
DGX Spark or another GB10 box, 128 GBNVIDIA DGX Spark, ASUS Ascent GX10Step 3.7 Flash UD-Q4_K_Mand 13 more of the 14 here

Phi 4 mini

3.8 billion parameters · up to 128K context · 119 thousand downloads in 30 days

Phi 4 mini needs about 7.7 GB at Q4_K_M with a 32K-token context. The file itself is 2.5 GB, and the cache for 32K tokens adds 4.3 GB. That fits a 8 GB card (RTX 4060, RTX 3070, RX 7600). On a Mac, 16 GB of unified memory is enough.

Every token of context adds 128 KB of cache, so the full 128K would put 17.2 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
Q2_K1.7 GB3.7 GB6.9 GB8 GB16 GB
Q3_K_M2.1 GB4.1 GB7.3 GB8 GB16 GB
Q4_K_M2.5 GB4.5 GB7.7 GB8 GB16 GB
Q5_K_M2.8 GB4.8 GB8.1 GB8 GB16 GB
Q6_K3.2 GB5.1 GB8.4 GB8 GB16 GB
Q8_04.1 GB6.1 GB9.3 GB12 GB16 GB
BF167.7 GB9.7 GB12.9 GB16 GB24 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Nemotron 3.5 Lightning 30B A3B

32.9 billion parameters · mixture of experts, 6 of 128 active · up to 1M context · 56 thousand downloads in 30 days

At UD-Q4_K_M, the Nemotron 3.5 Lightning 30B A3B file is 25.3 GB, and with 32K tokens of context and runtime overhead it comes to about 26.3 GB. The smallest card that holds it whole is a 32 GB card (RTX 5090, Radeon AI PRO R9700). It's a mixture-of-experts model (6 of 128 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

Most of its layers use linear attention, which keeps a fixed state instead of a cache, so the cache grows by about 6 KB a token and reaches 6.5 GB at 1M.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M19.4 GB20.3 GB20.4 GB20 GB24 GB*
UD-IQ2_M19.4 GB20.3 GB20.5 GB20 GB24 GB*
UD-Q3_K_XL21.2 GB22.1 GB22.3 GB24 GB32 GB
UD-Q4_K_M25.3 GB26.1 GB26.3 GB32 GB32 GB*
UD-Q5_K_XL30.4 GB31.3 GB31.4 GB32 GB36 GB*
UD-Q6_K_XL35.0 GB35.9 GB36.0 GB48 GB48 GB
Q8_035.0 GB35.9 GB36.0 GB48 GB48 GB
BF1665.9 GB66.7 GB66.9 GB96 GB96 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

LFM2.5 1.2B

1.2 billion parameters · up to 125K context · 36 thousand downloads in 30 days

LFM2.5 1.2B needs about 1.8 GB at Q4_K_M with a 32K-token context. The file itself is 0.7 GB, and the cache for 32K tokens adds 0.4 GB. That fits a 8 GB card (RTX 4060, RTX 3070, RX 7600). On a Mac, 16 GB of unified memory is enough.

Only 6 of its 16 layers keep a growing cache (the rest carry a fixed-size state), so the full 125K context costs 1.6 GB of cache.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL0.5 GB1.2 GB1.5 GB8 GB16 GB
UD-Q3_K_XL0.6 GB1.3 GB1.6 GB8 GB16 GB
Q4_K_M0.7 GB1.5 GB1.8 GB8 GB16 GB
Q5_K_M0.8 GB1.6 GB1.9 GB8 GB16 GB
Q6_K1.0 GB1.7 GB2.0 GB8 GB16 GB
Q8_01.2 GB2.0 GB2.3 GB8 GB16 GB
BF162.3 GB3.1 GB3.4 GB8 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Step 3.7 Flash

197 billion parameters · mixture of experts, 8 of 288 active · up to 256K context · 10 thousand downloads in 30 days

At UD-Q4_K_M, the Step 3.7 Flash file is 122 GB, and with 32K tokens of context and runtime overhead it comes to about 125 GB. No single consumer card holds it. A Mac with 128 GB of unified memory does once you raise the GPU memory limit, and a PC can split it between a GPU and system RAM. It's a mixture-of-experts model (8 of 288 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

A sliding window of 512 tokens on 33 layers keeps the cache small: 13.0 GB at the full 256K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M56.8 GB58.1 GB59.3 GB96 GB64 GB*
UD-Q2_K_XL65.8 GB67.1 GB68.3 GB96 GB96 GB
UD-Q3_K_XL89.4 GB90.7 GB91.9 GB96 GB96 GB*
UD-Q4_K_M122 GB123 GB125 GBnone128 GB*
UD-Q5_K_XL146 GB148 GB149 GBnone256 GB
UD-Q6_K_XL179 GB180 GB181 GBnone256 GB
Q8_0209 GB211 GB212 GBnone256 GB*
BF16394 GB395 GB397 GBnone512 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Ornith 1.5 122B

125 billion parameters · mixture of experts, 8 of 256 active · up to 256K context · 593 thousand downloads in 30 days

Ornith 1.5 122B needs about 81.7 GB at UD-Q4_K_XL with a 32K-token context. The file itself is 79.7 GB, and the cache for 32K tokens adds 1.0 GB. That fits a RTX PRO 6000 (96 GB). It's a mixture-of-experts model (8 of 256 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

Only 12 of its 48 layers keep a growing cache (the rest carry a fixed-size state), so the full 256K context costs 6.6 GB of cache.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M40.4 GB41.7 GB42.3 GB48 GB48 GB*
UD-Q2_K_XL44.5 GB45.9 GB46.5 GB48 GB48 GB*
UD-Q3_K_XL59.8 GB61.1 GB61.7 GB96 GB64 GB*
UD-Q4_K_XL79.7 GB81.1 GB81.7 GB96 GB96 GB*
UD-Q5_K_XL94.6 GB96.0 GB96.6 GB96 GB96 GB*
UD-Q6_K_XL114 GB116 GB116 GBnone128 GB*
Q8_0133 GB134 GB135 GBnone256 GB
BF16249 GB251 GB251 GBnone256 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Nemotron 3.5 Bolt E4B

7.5 billion parameters · up to 128K context · 481 thousand downloads in 30 days

At UD-Q4_K_XL, the Nemotron 3.5 Bolt E4B file is 5.1 GB, and with 32K tokens of context and runtime overhead it comes to about 6.7 GB. Even a 8 GB card (RTX 4060, RTX 3070, RX 7600) holds that. On a Mac, 16 GB of unified memory is enough.

A sliding window of 512 tokens on 20 layers keeps the cache small: 2.2 GB at the full 128K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL3.7 GB5.0 GB5.4 GB8 GB16 GB
UD-Q3_K_XL4.2 GB5.4 GB5.8 GB8 GB16 GB
UD-Q4_K_XL5.1 GB6.3 GB6.7 GB8 GB16 GB
UD-Q5_K_XL5.8 GB7.0 GB7.4 GB8 GB16 GB
UD-Q6_K_XL7.3 GB8.5 GB8.9 GB12 GB16 GB
UD-Q8_K_XL8.2 GB9.4 GB9.8 GB12 GB16 GB
BF1615.1 GB16.3 GB16.7 GB16 GB24 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

LFM2.5 24B A2B

24.0 billion parameters · mixture of experts, 4 of 64 active · up to 128K context · 430 thousand downloads in 30 days

LFM2.5 24B A2B needs about 16.3 GB at UD-Q4_K_XL with a 32K-token context. The file itself is 14.9 GB, and the cache for 32K tokens adds 0.7 GB. That fits a 16 GB card (RTX 4060 Ti 16 GB, 5070 Ti, 5080, RX 9070 XT). On a Mac, 24 GB of unified memory is enough. It routes each token through 4 of 64 experts, so it runs faster than its size suggests.

Only 10 of its 40 layers keep a growing cache (the rest carry a fixed-size state), so the full 128K context costs 2.7 GB of cache.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL8.0 GB9.0 GB9.5 GB12 GB16 GB
UD-Q3_K_XL10.9 GB11.9 GB12.4 GB12 GB16 GB*
UD-Q4_K_XL14.9 GB15.8 GB16.3 GB16 GB24 GB
UD-Q5_K_XL17.9 GB18.8 GB19.4 GB20 GB24 GB*
UD-Q6_K_XL21.9 GB22.8 GB23.3 GB24 GB32 GB*
Q8_025.5 GB26.4 GB26.9 GB32 GB32 GB*
BF1648.0 GB48.9 GB49.4 GB48 GB64 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Ornith 1.0 9B

9.0 billion parameters · up to 256K context · 418 thousand downloads in 30 days

At Q4_K_M, the Ornith 1.0 9B file is 5.7 GB, and with 32K tokens of context and runtime overhead it comes to about 7.8 GB. Even a 8 GB card (RTX 4060, RTX 3070, RX 7600) holds that. On a Mac, 16 GB of unified memory is enough.

Most of its layers use linear attention, which keeps a fixed state instead of a cache, so the cache grows by about 32 KB a token and reaches 8.6 GB at 256K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL4.3 GB5.7 GB6.5 GB8 GB16 GB
UD-Q3_K_XL5.1 GB6.4 GB7.2 GB8 GB16 GB
Q4_K_M5.7 GB7.0 GB7.8 GB8 GB16 GB
Q5_K_M6.5 GB7.9 GB8.7 GB12 GB16 GB
Q6_K7.5 GB8.8 GB9.6 GB12 GB16 GB
Q8_09.5 GB10.9 GB11.7 GB12 GB16 GB*
BF1617.9 GB19.3 GB20.1 GB20 GB24 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

flint

39.8 billion parameters · mixture of experts, 6 of 256 active · up to 256K context · 308 thousand downloads in 30 days

flint needs about 25.9 GB at UD-Q4_K_XL before its context cache: the file is 25.1 GB and llama.cpp wants roughly 0.8 GB more to run it. That fits a 32 GB card (RTX 5090, Radeon AI PRO R9700). It's a mixture-of-experts model (6 of 256 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

We don't compute the context cache for the flint architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M11.4 GB12.2 GB12.2 GB12 GB16 GB*
UD-Q2_K_XL13.6 GB14.4 GB14.4 GB16 GB24 GB
UD-Q3_K_XL18.5 GB19.3 GB19.3 GB20 GB24 GB*
UD-Q4_K_XL25.1 GB25.9 GB25.9 GB32 GB32 GB*
UD-Q5_K_XL29.9 GB30.7 GB30.7 GB32 GB36 GB*
UD-Q6_K_XL36.3 GB37.1 GB37.1 GB48 GB48 GB
Q8_042.3 GB43.1 GB43.1 GB48 GB48 GB*
BF1679.6 GB80.4 GB80.4 GB96 GB96 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

North Mini Code 1.0

30.5 billion parameters · mixture of experts, 8 of 128 active · up to 488K context · 233 thousand downloads in 30 days

At UD-Q4_K_M, the North Mini Code 1.0 file is 19.2 GB, and with 32K tokens of context and runtime overhead it comes to about 21.5 GB. The smallest card that holds it whole is a 20 GB card (RX 7900 XT). It routes each token through 8 of 128 experts, so it runs faster than its size suggests.

A sliding window of 4,096 tokens on 36 layers keeps the cache small: 13.7 GB at the full 488K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M9.4 GB11.0 GB11.6 GB12 GB16 GB*
UD-Q2_K_XL10.5 GB12.1 GB12.7 GB12 GB16 GB*
UD-Q3_K_XL14.3 GB15.9 GB16.6 GB16 GB24 GB
UD-Q4_K_M19.2 GB20.8 GB21.5 GB20 GB24 GB*
UD-Q5_K_XL23.0 GB24.6 GB25.2 GB24 GB32 GB*
UD-Q6_K_XL27.9 GB29.4 GB30.1 GB32 GB36 GB*
Q8_032.4 GB34.0 GB34.7 GB48 GB48 GB
BF1661.0 GB62.6 GB63.3 GB96 GB64 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Nemotron 3.5 Bolt 74B A9B

74.0 billion parameters · up to 256K context · 77 thousand downloads in 30 days

Nemotron 3.5 Bolt 74B A9B needs about 50.7 GB at UD-Q4_K_XL before its context cache: the file is 50.2 GB and llama.cpp wants roughly 0.5 GB more to run it. That fits a 48 GB (two 24 GB cards, Radeon PRO W7900). On a Mac, 64 GB of unified memory is enough.

We don't compute the context cache for the nemotron_h_omni architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S18.5 GB19.0 GB19.0 GB20 GB24 GB*
UD-Q2_K_XL28.8 GB29.3 GB29.3 GB32 GB32 GB*
UD-Q3_K_XL39.0 GB39.5 GB39.5 GB48 GB48 GB*
UD-Q4_K_XL50.2 GB50.7 GB50.7 GB48 GB64 GB
UD-Q5_K_XL58.9 GB59.4 GB59.4 GB96 GB64 GB*
UD-Q6_K_XL72.4 GB72.9 GB72.9 GB96 GB96 GB
Q8_078.9 GB79.4 GB79.4 GB96 GB96 GB*
BF16148 GB149 GB149 GBnone256 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Nemotron 3.5 Super VL 120B A12B

1.6 billion parameters · up to 1M context · 48 thousand downloads in 30 days

Nemotron 3.5 Super VL 120B A12B needs about 87.4 GB at UD-Q4_K_XL before its context cache: the file is 86.9 GB and llama.cpp wants roughly 0.5 GB more to run it. The smallest card that holds it whole is a RTX PRO 6000 (96 GB).

We don't compute the context cache for the nemotron_h_omni architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M46.1 GB46.6 GB46.6 GB48 GB48 GB*
UD-Q2_K_XL57.8 GB58.3 GB58.3 GB96 GB64 GB*
UD-Q3_K_XL65.8 GB66.3 GB66.3 GB96 GB96 GB
UD-Q4_K_XL86.9 GB87.4 GB87.4 GB96 GB96 GB*
UD-Q5_K_XL111 GB111 GB111 GBnone128 GB*
UD-Q6_K_XL121 GB121 GB121 GBnone128 GB*
Q8_0132 GB132 GB132 GBnone128 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Nemotron 3 Nano 30B A3B

31.6 billion parameters · mixture of experts, 6 of 128 active · up to 1M context · 40 thousand downloads in 30 days

Nemotron 3 Nano 30B A3B needs about 27.1 GB at Q4_K_M with a 32K-token context. The file itself is 24.6 GB, and the cache for 32K tokens adds 1.7 GB. That fits a 32 GB card (RTX 5090, Radeon AI PRO R9700). It's a mixture-of-experts model (6 of 128 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

Every token of context adds 52 KB of cache, so the full 1M would put 55.8 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL19.9 GB21.1 GB22.4 GB24 GB32 GB
UD-Q3_K_XL19.9 GB21.1 GB22.5 GB24 GB32 GB
Q4_K_M24.6 GB25.8 GB27.1 GB32 GB32 GB*
Q5_K_M26.1 GB27.4 GB28.7 GB32 GB32 GB*
Q6_K33.5 GB34.7 GB36.0 GB48 GB48 GB
Q8_033.6 GB34.8 GB36.1 GB48 GB48 GB
BF1663.2 GB64.4 GB65.7 GB96 GB96 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

MiniMax M3.1 Flash

426 billion parameters · mixture of experts, 4 of 128 active · up to 1M context · 24 thousand downloads in 30 days

MiniMax M3.1 Flash needs about 250 GB at UD-Q4_K_XL before its context cache: the file is 249 GB and llama.cpp wants roughly 0.9 GB more to run it. No single consumer card holds it. A Mac with 256 GB of unified memory does once you raise the GPU memory limit, and a PC can split it between a GPU and system RAM. It routes each token through 4 of 128 experts, so it runs faster than its size suggests.

We don't compute the context cache for the minimax_m3_vl architecture yet, because its config describes an attention scheme we haven't matched to llama.cpp's cache, so the totals here are weights and overhead only.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M129 GB130 GB130 GBnone128 GB*
UD-Q2_K_XL144 GB145 GB145 GBnone256 GB
UD-Q3_K_XL194 GB195 GB195 GBnone256 GB
UD-Q4_K_XL249 GB250 GB250 GBnone256 GB*
UD-Q8_K_XL259 GB260 GB260 GBnone256 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked November 10, 2026.