ThinkFacility
  1. Can I run it locally?
  2. Other open models

Phi, Nemotron, Step and LFM VRAM requirements: can I run them locally?

Models from labs with one or two popular downloads share this page. A new model from a lab without its own page shows up here the day it passes our download line.

Check your machine

Won't fit 125 GB

Weights, cache and overhead add up to more than the card and the RAM together.

  • Weights 122 GB
  • Context cache 1.7 GB
  • Overhead 0.8 GB
  • Fast memory
  • With system RAM

What we found

The architectures here vary more than on any family page. Nemotron 3.5 Lightning 30B A3B has attention in only 6 of its 52 layers. Another 23 are Mamba layers, which keep a fixed-size state, so a million-token context costs 6.5 GB of cache.

Phi 4 mini goes the other way. Its 4-bit file is 2.5 GB, and a 32K context adds 4.3 GB of cache, more than the model itself.

Step 3.7 Flash from StepFun is the big one, 125 GB at 4-bit with a 32K context. A DGX Spark holds it, and so does a 128 GB Mac with its GPU limit raised.

Step 3.7 Flash

197 billion parameters · mixture of experts, 8 of 288 active · up to 256K context · 308 thousand downloads in 30 days

Step 3.7 Flash needs about 125 GB at UD-Q4_K_M with a 32K-token context. The file itself is 122 GB, and the cache for 32K tokens adds 1.7 GB. No single consumer card holds it. A Mac with 128 GB of unified memory does once you raise the GPU memory limit, and a PC can split it between a GPU and system RAM. It's a mixture-of-experts model (8 of 288 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

33 of its layers only look back 512 tokens, so they stop growing early, and the whole cache at 256K is 13.0 GB.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M56.8 GB58.1 GB59.3 GB96 GB64 GB*
UD-Q2_K_XL65.8 GB67.1 GB68.3 GB96 GB96 GB
UD-Q3_K_XL89.4 GB90.7 GB91.9 GB96 GB96 GB*
UD-Q4_K_M122 GB123 GB125 GBnone128 GB*
UD-Q5_K_XL146 GB148 GB149 GBnone256 GB
UD-Q6_K_XL179 GB180 GB181 GBnone256 GB
Q8_0209 GB211 GB212 GBnone256 GB*
BF16394 GB395 GB397 GBnone512 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Phi 4 mini

3.8 billion parameters · up to 128K context · 119 thousand downloads in 30 days

At Q4_K_M, the Phi 4 mini file is 2.5 GB, and with 32K tokens of context and runtime overhead it comes to about 7.7 GB. Even a 8 GB card (RTX 4060, RTX 3070) holds that. On a Mac, 16 GB of unified memory is enough.

The cache is the part that grows: 128 KB a token, or 17.2 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
Q2_K1.7 GB3.7 GB6.9 GB8 GB16 GB
Q3_K_M2.1 GB4.1 GB7.3 GB8 GB16 GB
Q4_K_M2.5 GB4.5 GB7.7 GB8 GB16 GB
Q5_K_M2.8 GB4.8 GB8.1 GB8 GB16 GB
Q6_K3.2 GB5.1 GB8.4 GB8 GB16 GB
Q8_04.1 GB6.1 GB9.3 GB12 GB16 GB
BF167.7 GB9.7 GB12.9 GB16 GB24 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Nemotron 3.5 Lightning 30B A3B

32.9 billion parameters · mixture of experts, 6 of 128 active · up to 1M context · 68 thousand downloads in 30 days

Nemotron 3.5 Lightning 30B A3B needs about 26.3 GB at UD-Q4_K_M with a 32K-token context. The file itself is 25.3 GB, and the cache for 32K tokens adds 0.3 GB. That fits a RTX 5090 (32 GB). It's a mixture-of-experts model (6 of 128 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

Only 6 of its 52 layers keep a growing cache (the rest carry a fixed-size state), so the full 1M context costs 6.5 GB of cache.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M19.4 GB20.3 GB20.4 GB24 GB24 GB*
UD-IQ2_M19.4 GB20.3 GB20.5 GB24 GB24 GB*
UD-Q3_K_XL21.2 GB22.1 GB22.3 GB24 GB32 GB
UD-Q4_K_M25.3 GB26.1 GB26.3 GB32 GB32 GB*
UD-Q5_K_XL30.4 GB31.3 GB31.4 GB32 GB36 GB*
UD-Q6_K_XL35.0 GB35.9 GB36.0 GB48 GB48 GB
Q8_035.0 GB35.9 GB36.0 GB48 GB48 GB
BF1665.9 GB66.7 GB66.9 GB96 GB96 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

LFM2.5 1.2B

1.2 billion parameters · up to 125K context · 42 thousand downloads in 30 days

At Q4_K_M, the LFM2.5 1.2B file is 0.7 GB, and with 32K tokens of context and runtime overhead it comes to about 1.8 GB. Even a 8 GB card (RTX 4060, RTX 3070) holds that. On a Mac, 16 GB of unified memory is enough.

Most of its layers use linear attention, which keeps a fixed state instead of a cache, so the cache grows by about 12 KB a token and reaches 1.6 GB at 125K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL0.5 GB1.2 GB1.5 GB8 GB16 GB
UD-Q3_K_XL0.6 GB1.3 GB1.6 GB8 GB16 GB
Q4_K_M0.7 GB1.5 GB1.8 GB8 GB16 GB
Q5_K_M0.8 GB1.6 GB1.9 GB8 GB16 GB
Q6_K1.0 GB1.7 GB2.0 GB8 GB16 GB
Q8_01.2 GB2.0 GB2.3 GB8 GB16 GB
BF162.3 GB3.1 GB3.4 GB8 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Ornith 1.0 9B

9.0 billion parameters · up to 256K context · 483 thousand downloads in 30 days

Ornith 1.0 9B needs about 7.8 GB at Q4_K_M with a 32K-token context. The file itself is 5.7 GB, and the cache for 32K tokens adds 1.1 GB. That fits a 8 GB card (RTX 4060, RTX 3070). On a Mac, 16 GB of unified memory is enough.

Only 8 of its 32 layers keep a growing cache (the rest carry a fixed-size state), so the full 256K context costs 8.6 GB of cache.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL4.3 GB5.7 GB6.5 GB8 GB16 GB
UD-Q3_K_XL5.1 GB6.4 GB7.2 GB8 GB16 GB
Q4_K_M5.7 GB7.0 GB7.8 GB8 GB16 GB
Q5_K_M6.5 GB7.9 GB8.7 GB12 GB16 GB
Q6_K7.5 GB8.8 GB9.6 GB12 GB16 GB
Q8_09.5 GB10.9 GB11.7 GB12 GB16 GB*
BF1617.9 GB19.3 GB20.1 GB24 GB24 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

North Mini Code 1.0

30.5 billion parameters · mixture of experts, 8 of 128 active · up to 488K context · 316 thousand downloads in 30 days

At UD-Q4_K_M, the North Mini Code 1.0 file is 19.2 GB, and with 32K tokens of context and runtime overhead it comes to about 21.5 GB. The smallest card that holds it whole is a 24 GB card (RTX 3090, RTX 4090). It routes each token through 8 of 128 experts, so it runs faster than its size suggests.

A sliding window of 4,096 tokens on 36 layers keeps the cache small: 13.7 GB at the full 488K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M9.4 GB11.0 GB11.6 GB12 GB16 GB*
UD-Q2_K_XL10.5 GB12.1 GB12.7 GB12 GB16 GB*
UD-Q3_K_XL14.3 GB15.9 GB16.6 GB16 GB24 GB
UD-Q4_K_M19.2 GB20.8 GB21.5 GB24 GB24 GB*
UD-Q5_K_XL23.0 GB24.6 GB25.2 GB24 GB32 GB*
UD-Q6_K_XL27.9 GB29.4 GB30.1 GB32 GB36 GB*
Q8_032.4 GB34.0 GB34.7 GB48 GB48 GB
BF1661.0 GB62.6 GB63.3 GB96 GB64 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Ornith 1.0 35B

34.7 billion parameters · mixture of experts, 8 of 256 active · up to 256K context · 24 thousand downloads in 30 days

Ornith 1.0 35B needs about 23.9 GB at UD-Q4_K_M with a 32K-token context. The file itself is 22.1 GB, and the cache for 32K tokens adds 0.7 GB. That fits a 24 GB card (RTX 3090, RTX 4090). It routes each token through 8 of 256 experts, so it runs faster than its size suggests.

Only 10 of its 40 layers keep a growing cache (the rest carry a fixed-size state), so the full 256K context costs 5.4 GB of cache.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S10.5 GB11.8 GB12.3 GB12 GB16 GB*
UD-Q2_K_XL12.3 GB13.5 GB14.0 GB16 GB24 GB
UD-Q3_K_XL16.8 GB18.0 GB18.5 GB24 GB24 GB*
UD-Q4_K_M22.1 GB23.4 GB23.9 GB24 GB32 GB*
UD-Q5_K_XL26.5 GB27.8 GB28.3 GB32 GB32 GB*
UD-Q6_K_XL31.8 GB33.1 GB33.6 GB32 GB36 GB*
Q8_036.9 GB38.1 GB38.6 GB48 GB48 GB
BF1669.4 GB70.6 GB71.1 GB96 GB96 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

LFM2.5 8B A1B

8.5 billion parameters · mixture of experts, 4 of 32 active · up to 125K context · 21 thousand downloads in 30 days

At UD-Q4_K_M, the LFM2.5 8B A1B file is 5.3 GB, and with 32K tokens of context and runtime overhead it comes to about 6.5 GB. Even a 8 GB card (RTX 4060, RTX 3070) holds that. On a Mac, 16 GB of unified memory is enough. It routes each token through 4 of 32 experts, so it runs faster than its size suggests.

Most of its layers use linear attention, which keeps a fixed state instead of a cache, so the cache grows by about 12 KB a token and reaches 1.6 GB at 125K.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_M2.6 GB3.4 GB3.7 GB8 GB16 GB
UD-Q2_K_XL2.9 GB3.8 GB4.1 GB8 GB16 GB
UD-Q3_K_XL4.0 GB4.9 GB5.2 GB8 GB16 GB
UD-Q4_K_M5.3 GB6.2 GB6.5 GB8 GB16 GB
UD-Q5_K_XL6.4 GB7.3 GB7.6 GB8 GB16 GB
UD-Q6_K_XL7.7 GB8.6 GB8.9 GB12 GB16 GB
Q8_09.0 GB9.9 GB10.2 GB12 GB16 GB
BF1616.9 GB17.8 GB18.1 GB24 GB24 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked September 25, 2026.