ThinkFacility
  1. Can I run it locally?
  2. Llama and Muse

Llama and Muse VRAM requirements: can I run Muse Glimmer 30B locally?

Muse Glimmer 30B is 17.3 GB at 4-bit with a 32K context, so a 24 GB card runs it. Here's Meta's newest open model beside the Llamas people still download.

Check your machine

Fits on the card 17.3 GB

Everything stays in video memory, which is the fast case.

  • Weights 15.9 GB
  • Context cache 0.5 GB
  • Overhead 0.9 GB
  • Fast memory
  • With system RAM

What we found

Put Muse Glimmer 30B beside Llama 3.3 70B and the difference in the cache is enormous. At the full 128K context Glimmer's cache is 1.8 GB and Llama 3.3's is 42.9 GB, about as big as the Llama's 4-bit weights. Glimmer keeps three of every four layers on a 2,048-token window and uses just two key-value heads.

Llama 3.3 70B is a two-card model. Two 24 GB cards hold it at an 8K context (46.0 GB), and at 32K it spills over.

Llama 4 Scout 17B 16E lists a 10-million-token context in its config. The cache for all of it would be 517 GB, so that number is a ceiling nobody reaches at home.

The small Llama 3.2 models flip the usual picture. The 3B's 4-bit file is 2.0 GB, and a 32K context adds 3.8 GB of cache on top, more than the model. Meta's own lineup is in our Muse guide.

Muse Glimmer 30B

27.9 billion parameters · up to 128K context · 281 thousand downloads in 30 days

Muse Glimmer 30B needs about 17.3 GB at UD-Q4_K_XL with a 32K-token context. The file itself is 15.9 GB, and the cache for 32K tokens adds 0.5 GB. That fits a 24 GB card (RTX 3090, RTX 4090).

39 of its layers only look back 2,048 tokens, so they stop growing early, and the whole cache at 128K is 1.8 GB.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL12.4 GB13.6 GB13.9 GB16 GB24 GB
UD-Q3_K_XL13.4 GB14.5 GB14.8 GB16 GB24 GB
UD-Q4_K_XL15.9 GB17.0 GB17.3 GB24 GB24 GB*
UD-Q5_K_XL21.8 GB22.9 GB23.2 GB24 GB32 GB*
UD-Q6_K_XL26.3 GB27.4 GB27.7 GB32 GB32 GB*
Q8_029.6 GB30.7 GB31.1 GB32 GB36 GB*
BF1655.7 GB56.9 GB57.2 GB96 GB64 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Llama 3.2 3B

3.2 billion parameters · up to 128K context · 196 thousand downloads in 30 days

At Q4_K_M, the Llama 3.2 3B file is 2.0 GB, and with 32K tokens of context and runtime overhead it comes to about 6.5 GB. Even a 8 GB card (RTX 4060, RTX 3070) holds that. On a Mac, 16 GB of unified memory is enough.

The cache is the part that grows: 112 KB a token, or 15.0 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S0.9 GB2.6 GB5.4 GB8 GB16 GB
UD-Q2_K_XL1.4 GB3.1 GB5.9 GB8 GB16 GB
UD-Q3_K_XL1.7 GB3.4 GB6.3 GB8 GB16 GB
Q4_K_M2.0 GB3.7 GB6.5 GB8 GB16 GB
Q5_K_M2.3 GB4.0 GB6.8 GB8 GB16 GB
Q6_K2.6 GB4.3 GB7.2 GB8 GB16 GB
Q8_03.4 GB5.1 GB7.9 GB8 GB16 GB
BF166.4 GB8.1 GB11.0 GB12 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Llama 3.3 70B

70.6 billion parameters · up to 128K context · 34 thousand downloads in 30 days

Llama 3.3 70B needs about 54.0 GB at Q4_K_M with a 32K-token context. The file itself is 42.5 GB, and the cache for 32K tokens adds 10.7 GB. That fits a RTX PRO 6000 (96 GB).

Every token of context adds 320 KB of cache, so the full 128K would put 42.9 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S15.9 GB19.3 GB27.4 GB32 GB32 GB*
UD-Q2_K_XL27.0 GB30.4 GB38.5 GB48 GB48 GB
UD-Q3_K_XL34.8 GB38.2 GB46.3 GB48 GB48 GB*
Q4_K_M42.5 GB46.0 GB54.0 GB96 GB64 GB*
Q5_K_M49.9 GB53.4 GB61.4 GB96 GB64 GB*
Q6_K116 GB119 GB127 GBnone128 GB*
Q8_075.0 GB78.4 GB86.5 GB96 GB96 GB*
BF16141 GB145 GB153 GBnone256 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Llama 4 Scout 17B 16E

108 billion parameters · mixture of experts, 1 of 16 active · up to 10M context · 31 thousand downloads in 30 days

At Q4_K_M, the Llama 4 Scout 17B 16E file is 65.4 GB, and with 32K tokens of context and runtime overhead it comes to about 69.2 GB. The smallest card that holds it whole is a RTX PRO 6000 (96 GB). On a Mac, 96 GB of unified memory is enough. It's a mixture-of-experts model (1 of 16 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

A sliding window of 8,192 tokens on 36 layers keeps the cache small: 517 GB at the full 10M.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-TQ1_029.3 GB31.8 GB33.1 GB32 GB36 GB*
UD-Q2_K_XL42.4 GB44.9 GB46.2 GB48 GB48 GB*
UD-Q3_K_XL49.0 GB51.5 GB52.8 GB96 GB64 GB*
Q4_K_M65.4 GB67.9 GB69.2 GB96 GB96 GB
Q5_K_M76.5 GB79.1 GB80.4 GB96 GB96 GB*
Q6_K88.4 GB91.0 GB92.2 GB96 GB96 GB*
Q8_0115 GB117 GB118 GBnone128 GB*
BF16216 GB218 GB219 GBnone256 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Llama 3.2 1B

1.2 billion parameters · up to 128K context · 38 thousand downloads in 30 days

Llama 3.2 1B needs about 2.6 GB at Q4_K_M with a 32K-token context. The file itself is 0.8 GB, and the cache for 32K tokens adds 1.1 GB. That fits a 8 GB card (RTX 4060, RTX 3070). On a Mac, 16 GB of unified memory is enough.

Every token of context adds 32 KB of cache, so the full 128K would put 4.3 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S0.4 GB1.5 GB2.3 GB8 GB16 GB
UD-Q2_K_XL0.6 GB1.6 GB2.4 GB8 GB16 GB
UD-Q3_K_XL0.7 GB1.8 GB2.6 GB8 GB16 GB
Q4_K_M0.8 GB1.8 GB2.6 GB8 GB16 GB
Q5_K_M0.9 GB1.9 GB2.7 GB8 GB16 GB
Q6_K1.0 GB2.1 GB2.9 GB8 GB16 GB
Q8_01.3 GB2.4 GB3.2 GB8 GB16 GB
BF162.5 GB3.5 GB4.3 GB8 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked September 25, 2026.