ThinkFacility Sign in

Free AI on your own computer

Can I run it locally? How much VRAM open models need

Tell it what's in your computer and you'll see which free AI models run on it, most popular first. Sizes come from the real files on Hugging Face and get rechecked every day.

1What's in your computer?

2How much memory does the card have?

AMD and Intel cards count the same way: it's the memory that matters. LM Studio runs on all three makers' cards.

Not sure? On Windows, open Task Manager, then Performance. The GPU tab shows "Dedicated GPU memory" and the Memory tab shows your RAM.

More settings

46 models run at full speed on a 24 GB card, and 8 more run slowly

The best one from each family

Strongest first. There's no benchmark covering every open model, so we rank by size: a bigger model is usually a smarter one, and a mixture-of-experts model counts for less than its headline number.

GoogleGemma 4 31B22.0 GBMetaMuse Glimmer 30B17.3 GBAlibabaQwen3.8 27B19.8 GBMistral AIDevstral Small 2 24B 2512Coding20.5 GBDeepSeekDeepSeek R1 Distill Qwen 14B16.2 GBZ.aiGLM-4.6V Flash9.4B · 8.3 GBSeveral labsOrnith 1.0 9B7.8 GBOpenAIgpt-oss-20b13.4 GB
All 46 that run at full speed
8 more that run, but slowly

Want bigger models? A 32 GB graphics card runs 4 more of them at full speed.

Compare cards and Macs

As an Amazon Associate we earn from qualifying purchases.

3Get it running

  1. Install a free app

    LM Studio is point and click. Ollama suits you if you like the command line.

  2. Search for the model's name

    Both apps download straight from Hugging Face, the site that hosts these files.

  3. Pick the 4-bit file and start chatting

    Look for Q4_K_M, or the nearest name with a 4 in it. That's the file our sizes are based on.

Browse by maker

77 models from 10 families, each with every size and what it needs.

Alibaba

Qwen

The most downloaded open models on Hugging Face, from 2B up to Flash Next.

14 models · smallest 2.0 GBGoogle

Gemma

Google's open models, from tiny phone-sized ones up to one that fills a 24 GB card.

12 models · smallest 1.4 GBSeveral labs

Other open models

Phi, Nemotron, Step, LFM and whatever new lab turns up next.

14 models · smallest 1.8 GBDeepSeek

DeepSeek

V4 Flash and Pro at home, and the R1 distills that fit a gaming card.

11 models · smallest 2.9 GBZ.ai

GLM

Z.ai's big GLM-5 models and the Flash versions that fit a desk.

7 models · smallest 8.3 GBMeta

Llama and Muse

Muse Glimmer, Llama 3.3 70B and Llama 4 Scout, side by side.

6 models · smallest 2.6 GBOpenAI

gpt-oss

OpenAI's two open models. The small one fits a 16 GB card.

2 models · smallest 13.4 GBThinking Machines

Inkling

Thinking Machines' two models, one of which fits a DGX Spark.

2 models · smallest 165 GBMistral AI

Mistral

Devstral and Mistral Small, the 24B models built for a single card.

6 models · smallest 6.4 GBMoonshot AI

Kimi

Kimi K3 and K2, the largest open models people download.

3 models · smallest 52.9 GB

Shopping for the computer instead? The best GPU for local LLMs lines up each graphics card, Mac and mini PC against these models.

The words on this page

What's VRAM?

It's the graphics card's own memory. A model that fits in it runs fast; one that spills over into the computer's normal RAM has to wait on it for every word.

What does 4-bit mean?

Models get squeezed for home use by storing each number in fewer bits. 4-bit files are the usual trade between size and quality, and every size on this page assumes one.

What's context?

It's how much text the model keeps in mind at once: your messages, its replies, any document you paste in. More context needs more memory on top of the model itself.

Why does a Mac show less than its full memory?

macOS only lets the graphics side use part of the shared memory by default, so a 64 GB Mac starts with 48 GB for a model.

How we work out the numbers

What we found

The weights are only half the question. The other half is the context cache, the keys and values a model keeps for every token it has read, and it varies wildly. Llama 3.3 70B at its full 128K context carries 42.9 GB of cache, roughly the size of its 4-bit weights. Qwen3.6 35B A3B, a year and a half newer, needs 5.4 GB for twice the context.

Two unit traps catch people out. A "24 GB" graphics card holds 24 GiB, which is about 25.8 GB in the decimal units Hugging Face uses for file sizes, so a 25 GB total does fit. Macs go the other way: macOS lets the GPU use only two thirds of unified memory up to 32 GB and three quarters above, so a 64 GB Mac starts out with 48 GB for the model.

The most downloaded GGUF in Unsloth's catalog is Qwen3 Coder 30B A3B, from July 2025, at 486 thousand in the last 30 days. It fits a 24 GB card at 4-bit. Whether any of this saves money is another question, and we priced a home rig against a subscription to find out.

How the numbers are worked out

Total memory is three parts added together. Weights are the exact size of the GGUF file on Hugging Face, all parts summed when a quant is split. The context cache (the KV cache) comes from each model's own config.json:

cache = layers × kv_heads × head_dim × 2 (K and V) × 2 bytes × tokens

Sliding-window layers stop at their window plus llama.cpp's 512-token batch. Linear-attention and Mamba layers keep a fixed state instead, which we add once. Latent-attention models (DeepSeek, Kimi, GLM-5) cache kv_lora_rank + rope dimensions per layer, the way llama.cpp does. Overhead is our allowance: 0.5 GB for the GPU context plus the output buffer, 512 × vocabulary × 4 bytes.

A "24 GB" card holds 24 GiB, which is 25.8 GB in the decimal units file sizes use, and we count it that way. Macs get the share Metal allows by default, two thirds of unified memory up to 32 GB and three quarters above (llama.cpp discussion 2182); CPU-only machines and the DGX Spark keep 4 GB back for the operating system. All of it assumes llama.cpp's defaults: an f16 cache and one conversation at a time. Quantizing the cache to q8_0 halves it.

Files are Unsloth's GGUF uploads, the most downloaded quantizations on Hugging Face, and the list refreshes daily from Hugging Face's API. A model joins when its repo passes 20,000 downloads in 30 days. Last checked November 10, 2026.