ThinkFacility

Open models on your own hardware

Can I run it locally? How much VRAM open models need

Pick your graphics card, Mac or DGX Spark to see which open models fit, or open a family for every size and quant. File sizes come from Hugging Face, the rest is arithmetic on each model's config, and it's redone every day.

What can my machine run?

Each model at its common 4-bit file. Family pages have every quant.

Fits on the card 44

Runs with help from system RAM 5

Model families

66 models across 10 families. Dots are each model at 4-bit with a 32K context, on a scale from 1 GB to 2 TB; the marks are 8, 24, 128 and 512 GB.

Alibaba

Qwen

The most downloaded open models on Hugging Face, from 2B up to Flash Next.

14 models · 2.7 to 113 GB at 4-bitGoogle

Gemma

Google's open models, whose context cache barely grows.

12 models · 1.4 to 22.0 GB at 4-bitZ.ai

GLM

Z.ai's big GLM-5 models and the Flash versions that fit a desk.

5 models · 8.3 to 472 GB at 4-bitThinking Machines

Inkling

Thinking Machines' two models, one of which fits a DGX Spark.

2 models · 165 to 590 GB at 4-bitSeveral labs

Other open models

Phi, Nemotron, Step, LFM and whatever new lab turns up next.

8 models · 1.8 to 125 GB at 4-bitMoonshot AI

Kimi

Kimi K3 and K2, the largest open models people download.

5 models · 52.9 to 1,511 GB at 4-bitDeepSeek

DeepSeek

V4 Flash and Pro at home, and the R1 distills that fit a gaming card.

10 models · 2.9 to 850 GB at 4-bitOpenAI

gpt-oss

OpenAI's open models, where every quant is almost the same size.

2 models · 13.4 to 64.9 GB at 4-bitMeta

Llama and Muse

Muse Glimmer, Llama 3.3 70B and Llama 4 Scout, side by side.

5 models · 2.6 to 69.2 GB at 4-bitMistral AI

Mistral

Devstral and Mistral Small, the 24B models built for a single card.

3 models · 6.4 to 20.5 GB at 4-bit

What we found

The weights are only half the question. The other half is the context cache, the keys and values a model keeps for every token it has read, and it varies wildly. Llama 3.3 70B at its full 128K context carries 42.9 GB of cache, roughly the size of its 4-bit weights. Qwen3.6 35B A3B, a year and a half newer, needs 5.4 GB for twice the context.

Two unit traps catch people out. A "24 GB" graphics card holds 24 GiB, which is about 25.8 GB in the decimal units Hugging Face uses for file sizes, so a 25 GB total does fit. Macs go the other way: macOS lets the GPU use only two thirds of unified memory up to 32 GB and three quarters above, so a 64 GB Mac starts out with 48 GB for the model.

The download counts were a surprise too. The most downloaded GGUF in Unsloth's catalog is Qwen3 Coder 30B A3B, from July 2025, at 11.6 million in the last 30 days. It fits a 24 GB card at 4-bit. Whether any of this saves money is another question, and we priced a home rig against a subscription to find out.

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked September 25, 2026.