ThinkFacility Sign in
  1. Can I run it locally?
  2. Llama and Muse

Llama and Muse VRAM requirements: can I run Muse Glimmer 30B locally?

Muse Glimmer 30B is 17.3 GB at 4-bit with a 32K context, so a 24 GB card runs it. Here's Meta's newest open model beside the Llamas people still download.

Check your machine

Fits on the card 17.3 GB

Everything stays in video memory, which is the fast case.

  • Weights 15.9 GB
  • Context cache 0.5 GB
  • Overhead 0.9 GB
  • Fast memory
  • With system RAM

What we found

Put Muse Glimmer 30B beside Llama 3.3 70B and the difference in the cache is enormous. At the full 128K context Glimmer's cache is 1.8 GB and Llama 3.3's is 42.9 GB, about as big as the Llama's 4-bit weights. Glimmer keeps three of every four layers on a 2,048-token window and uses just two key-value heads.

Llama 3.3 70B is a two-card model. Two 24 GB cards hold it at an 8K context (46.0 GB), and at 32K it spills over.

Llama 4 Scout 17B 16E lists a 10-million-token context in its config. The cache for all of it would be 517 GB, so that number is a ceiling nobody reaches at home.

The small Llama 3.2 models flip the usual picture. The 3B's 4-bit file is 2.0 GB, and a 32K context adds 3.8 GB of cache on top, more than the model. Meta's own lineup is in our Muse guide.

Hardware that runs it

The biggest model on this page each machine holds whole with a 32K context, at the best file that fits. Which one to buy is in our GPU guide for local LLMs.

As an Amazon Associate we earn from qualifying purchases.

MachineRuns Check price on Amazon
16 GB graphics cardGeForce RTX 5060 Ti 16 GB, GeForce RTX 5070 Ti, GeForce RTX 5080Llama 3.2 3B BF16and 2 more of the 6 here
ASUS Dual RTX 5060 Ti 16GB OCASUS Prime RTX 5070 Ti 16GB OCASUS Prime RTX 5080 EVO 16GB OC
24 GB graphics cardRadeon RX 7900 XTX, Intel Arc Pro B60Muse Glimmer 30B UD-Q5_K_XLand 2 more of the 6 here
Compare in the guide
32 GB graphics cardGeForce RTX 5090, Radeon AI PRO R9700Muse Glimmer 30B Q8_0and 4 more of the 6 here
Compare in the guide
48 GB workstation cardRTX PRO 5000 BlackwellMuse Glimmer 30B Q8_0and 4 more of the 6 here
Compare in the guide
96 GB workstation cardRTX PRO 6000 Blackwell Workstation EditionLlama 4 Scout 17B 16E Q6_Kand 4 more of the 6 here
Compare in the guide
Mac with 64 GB of unified memoryMac mini (M5 Pro, 64 GB), Mac Studio (M5 Max, 64 GB)Muse Glimmer 30B Q8_0and 4 more of the 6 here
Compare in the guide
Mac with 96 GB of unified memoryMac Studio (M5 Ultra, 96 GB)Llama 4 Scout 17B 16E Q4_K_Mand 4 more of the 6 here
Compare in the guide
Mac with 128 GB of unified memoryMac Studio (M5 Max, 128 GB), MacBook Pro 14 (M5 Max, 128 GB)Llama 4 Scout 17B 16E Q6_Kand 4 more of the 6 here
MacBook Pro 14 M5 Max
Mac Studio with 256 GBMac Studio (M5 Ultra, 256 GB)Llama 4 Scout 17B 16E Q8_0and 5 more of the 6 here
Compare in the guide
Mac Studio with 512 GBMac Studio (M5 Ultra, 512 GB)Llama 4 Maverick 17B 128E Q6_Kand 5 more of the 6 here
Compare in the guide
Ryzen AI Max+ 395 mini PC with 128 GBGMKtec EVO-X2, Beelink GTR9 Pro, HP Z2 Mini G1aLlama 4 Scout 17B 16E Q6_Kand 4 more of the 6 here
GMKtec EVO-X2 128GB
DGX Spark or another GB10 box, 128 GBNVIDIA DGX Spark, ASUS Ascent GX10Llama 4 Scout 17B 16E Q8_0and 5 more of the 6 here
Compare in the guide

Muse Glimmer 30B

27.9 billion parameters · up to 128K context · 908 thousand downloads in 30 days

Muse Glimmer 30B needs about 17.3 GB at UD-Q4_K_XL with a 32K-token context. The file itself is 15.9 GB, and the cache for 32K tokens adds 0.5 GB. That fits a 20 GB card (RX 7900 XT).

39 of its layers only look back 2,048 tokens, so they stop growing early, and the whole cache at 128K is 1.8 GB.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-Q2_K_XL12.4 GB13.6 GB13.9 GB16 GB24 GB
UD-Q3_K_XL13.4 GB14.5 GB14.8 GB16 GB24 GB
UD-Q4_K_XL15.9 GB17.0 GB17.3 GB20 GB24 GB*
UD-Q5_K_XL21.8 GB22.9 GB23.2 GB24 GB32 GB*
UD-Q6_K_XL26.3 GB27.4 GB27.7 GB32 GB32 GB*
Q8_029.6 GB30.7 GB31.1 GB32 GB36 GB*
BF1655.7 GB56.9 GB57.2 GB96 GB64 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Llama 3.2 3B

3.2 billion parameters · up to 128K context · 253 thousand downloads in 30 days

At Q4_K_M, the Llama 3.2 3B file is 2.0 GB, and with 32K tokens of context and runtime overhead it comes to about 6.5 GB. Even a 8 GB card (RTX 4060, RTX 3070, RX 7600) holds that. On a Mac, 16 GB of unified memory is enough.

The cache is the part that grows: 112 KB a token, or 15.0 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S0.9 GB2.6 GB5.4 GB8 GB16 GB
UD-Q2_K_XL1.4 GB3.1 GB5.9 GB8 GB16 GB
UD-Q3_K_XL1.7 GB3.4 GB6.3 GB8 GB16 GB
Q4_K_M2.0 GB3.7 GB6.5 GB8 GB16 GB
Q5_K_M2.3 GB4.0 GB6.8 GB8 GB16 GB
Q6_K2.6 GB4.3 GB7.2 GB8 GB16 GB
Q8_03.4 GB5.1 GB7.9 GB8 GB16 GB
BF166.4 GB8.1 GB11.0 GB12 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Llama 4 Scout 17B 16E

108 billion parameters · mixture of experts, 1 of 16 active · up to 10M context · 41 thousand downloads in 30 days

Llama 4 Scout 17B 16E needs about 69.2 GB at Q4_K_M with a 32K-token context. The file itself is 65.4 GB, and the cache for 32K tokens adds 2.9 GB. That fits a RTX PRO 6000 (96 GB). On a Mac, 96 GB of unified memory is enough. It's a mixture-of-experts model (1 of 16 experts per token), which is why people run it with the experts parked in system RAM and the rest on a 24 GB card.

36 of its layers only look back 8,192 tokens, so they stop growing early, and the whole cache at 10M is 517 GB.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-TQ1_029.3 GB31.8 GB33.1 GB32 GB36 GB*
UD-Q2_K_XL42.4 GB44.9 GB46.2 GB48 GB48 GB*
UD-Q3_K_XL49.0 GB51.5 GB52.8 GB96 GB64 GB*
Q4_K_M65.4 GB67.9 GB69.2 GB96 GB96 GB
Q5_K_M76.5 GB79.1 GB80.4 GB96 GB96 GB*
Q6_K88.4 GB91.0 GB92.2 GB96 GB96 GB*
Q8_0115 GB117 GB118 GBnone128 GB*
BF16216 GB218 GB219 GBnone256 GB*

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Llama 3.3 70B

70.6 billion parameters · up to 128K context · 37 thousand downloads in 30 days

At Q4_K_M, the Llama 3.3 70B file is 42.5 GB, and with 32K tokens of context and runtime overhead it comes to about 54.0 GB. The smallest card that holds it whole is a RTX PRO 6000 (96 GB).

The cache is the part that grows: 320 KB a token, or 42.9 GB if you use all 128K tokens it supports.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S15.9 GB19.3 GB27.4 GB32 GB32 GB*
UD-Q2_K_XL27.0 GB30.4 GB38.5 GB48 GB48 GB
UD-Q3_K_XL34.8 GB38.2 GB46.3 GB48 GB48 GB*
Q4_K_M42.5 GB46.0 GB54.0 GB96 GB64 GB*
Q5_K_M49.9 GB53.4 GB61.4 GB96 GB64 GB*
Q6_K116 GB119 GB127 GBnone128 GB*
Q8_075.0 GB78.4 GB86.5 GB96 GB96 GB*
BF16141 GB145 GB153 GBnone256 GB

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

Llama 3.2 1B

1.2 billion parameters · up to 128K context · 52 thousand downloads in 30 days

Llama 3.2 1B needs about 2.6 GB at Q4_K_M with a 32K-token context. The file itself is 0.8 GB, and the cache for 32K tokens adds 1.1 GB. That fits a 8 GB card (RTX 4060, RTX 3070, RX 7600). On a Mac, 16 GB of unified memory is enough.

Every token of context adds 32 KB of cache, so the full 128K would put 4.3 GB on top of the weights.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-IQ1_S0.4 GB1.5 GB2.3 GB8 GB16 GB
UD-Q2_K_XL0.6 GB1.6 GB2.4 GB8 GB16 GB
UD-Q3_K_XL0.7 GB1.8 GB2.6 GB8 GB16 GB
Q4_K_M0.8 GB1.8 GB2.6 GB8 GB16 GB
Q5_K_M0.9 GB1.9 GB2.7 GB8 GB16 GB
Q6_K1.0 GB2.1 GB2.9 GB8 GB16 GB
Q8_01.3 GB2.4 GB3.2 GB8 GB16 GB
BF162.5 GB3.5 GB4.3 GB8 GB16 GB

Smallest GPU and Mac are for the 32K total.

Sources: GGUF files · config.json

Llama 4 Maverick 17B 128E

401 billion parameters · mixture of experts, 1 of 128 active · up to 1M context · 28 thousand downloads in 30 days

At Q4_K_M, the Llama 4 Maverick 17B 128E file is 243 GB, and with 32K tokens of context and runtime overhead it comes to about 247 GB. No single consumer card holds it. A Mac with 256 GB of unified memory does once you raise the GPU memory limit, and a PC can split it between a GPU and system RAM. It routes each token through 1 of 128 experts, so it runs faster than its size suggests.

A sliding window of 8,192 tokens on 36 layers keeps the cache small: 52.8 GB at the full 1M.

QuantFileAt 8KAt 32KSmallest GPUSmallest Mac
UD-TQ1_0106 GB108 GB110 GBnone128 GB*
UD-Q2_K_XL153 GB155 GB156 GBnone256 GB
UD-Q3_K_XL180 GB182 GB183 GBnone256 GB
Q4_K_M243 GB245 GB247 GBnone256 GB*
Q5_K_M284 GB287 GB288 GBnone512 GB
Q6_K329 GB331 GB333 GBnone512 GB
Q8_0426 GB428 GB430 GBnone512 GB*
BF16801 GB804 GB805 GBnonenone

Smallest GPU and Mac are for the 32K total. * with the Mac's GPU memory limit raised.

Sources: GGUF files · config.json

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked November 10, 2026.