Start with RAM. A 32 GB dual-channel mini PC runs a 7B or 8B chat model; a 27B Q4 model wants 32 to 64 GB of shared memory. A 70B model needs 128 GB of unified memory and still types slowly. Spec-sheet NPU TOPS do not determine the model size.
An N100 cube arrives on a Tuesday with “AI Mini PC” on its listing. Ollama installs in two minutes, but llama3.1:8b then spends its time swapping. Its file occupies about 4.9 GB on disk, and Windows needs several gigabytes for itself. Check the available memory pool before trusting the AI label.
How much RAM does a mini PC need for a local LLM?
Thirty-two gigabytes is a practical starting point with two RAM sticks or a fast soldered pool. Sixteen gigabytes can load a 7B Q4 file but leaves little room for Chrome. A dense 70B Q4 file is about 40 GB before the cache, so allow room for both the weights and the OS.
Framework’s September 2026 local-AI note recommends a model for its 32 GB machine: “For the 32GB Framework Desktop, my current recommendation is Qwen3.8-27B with the Unsloth Dynamic four-bit quantization.” Framework puts that GGUF at 17.6 GB, leaving space for the operating system, runtime, and a usable context window. The recommendation applies to that chip family; another 32 GB mini PC may behave differently.
Q4 sizing keeps landing in the same ballpark: about 5 GB for 7B–8B, 8 to 10 GB for 13B–14B, 18 to 22 GB for 27B–32B, and about 40 GB for a dense 70B. Ollama’s GPU page does not publish a RAM table. It does list which GPUs the app will try to use, including AMD’s Ryzen AI Max+ 395 on Linux ROCm.
| Model class (Q4-style) | Weight file, typical | RAM I would buy on a mini PC | What it feels like |
|---|---|---|---|
| 7B–8B chat | –5 GB | 32 GB, two SODIMMs or soldered LPDDR | Usable replies |
| 13B–14B | –8–10 GB | 32 GB if you close the browser; 64 GB if you will not | Fine until context grows |
| 27B–32B dense | –18–22 GB | 32 GB is tight; 64 GB is calmer | Slower than a website |
| 70B dense | –40 GB | 128 GB unified memory | Fits. Does not sprint. |
On an upgradeable mini PC, check the number of RAM sticks. Two 16 GB modules use the full 128-bit DDR5 bus; one 32 GB module has the same capacity at about half the bandwidth. A LocalAIMaster Ollama hardware write-up puts the split around 90 GB/s versus 45 GB/s. That can separate an 8B model that keeps up with typing from one that pauses every clause. Given those choices, prefer 32 GB in two sticks over one, even if the product photos look identical.
Soldered LPDDR avoids the single-stick problem, but you cannot add more later. Choose 32, 64, or 128 GB at checkout. AMD’s Ryzen AI Max+ 395 page lists a 128 GB ceiling on 256-bit LPDDR5x-8000. Many HX 370 mini PCs stop at 32 GB of soldered LPDDR5X. ComputerBase’s Beelink SER9 review recorded 32 GB LPDDR5X-7500 with no upgrade path. If you may want a 27B next year, the $80 cheaper 32 GB SKU cannot grow with it.
Do I need a discrete GPU in a mini PC to run local models?
No. A 7B or 8B chat box can live on the iGPU. Buy a discrete GPU when the whole model must sit in fast VRAM and you care about tokens per second. A 12 GB used card beats 64 GB of system RAM on an 8B, then loses when the model no longer fits.
Ars Technica’s Framework Desktop review put the trade in one sentence: “For people running certain AI or machine learning workloads, the 8060S’s unified memory setup means you can get a GPU with 64GB or 128GB of VRAM for less than the price of a single RTX 5090 (Framework says the GPU can use up to 112GB of RAM on the 128GB Desktop).” That is capacity, not peak shader speed.
Notebookcheck found that Windows shipped on the same machine with GPU memory set to 8 GB. Reviewers raised it to 16 GB. Leave the setting at 8 GB when loading a 17 GB GGUF, and layers fall back to the CPU even though the box carries an “AI” label. Check the memory setting before judging model speed.
Ollama can use NVIDIA CUDA, AMD ROCm, Apple Metal, or Vulkan. Its support list includes Ryzen AI Max+ 395, Ryzen AI 9 HX 370, and RTX 30- and 40-series cards. A 16 GB N100 remains “supported” for CPU-only use, though CPU-only 7B chat may be too slow for daily work.
USB4 eGPU docks are another option, and some Minisforum boards add OCulink for a separate card. You then have a second chassis on the desk. If you already want CUDA and 24 GB of GDDR, compare a used tower with a 3090 against the combined cost of a cube and eGPU setup.
Which mini PC processor actually matters for a local LLM?
For Ollama or LM Studio on llama.cpp, examine the iGPU and memory bus before the “50 TOPS NPU” badge. Those runtimes depend on GPU memory bandwidth. The NPU serves Windows Studio Effects and some ONNX paths, but it does not supply the memory needed to fit a 70B model.
AMD publishes the Max+ 395 as 16 Zen 5 cores, 32 threads, Radeon 8060S with 40 graphics cores, up to 50 NPU TOPS, and up to 126 TOPS overall. Default TDP is 55 W, configurable 45 to 120 W. That is the Strix Halo class. Below it, the Ryzen AI 9 HX 370 is 12 cores with a Radeon 890M (16 compute units) and the same 50 TOPS NPU number, per AMD’s own Strix Point tables as reported in ComputerBase’s SER9 review. Older HS chips such as the Ryzen 9 8945HS ship with a Radeon 780M and no NPU. For chat, I would take 780M plus 64 GB of dual-channel DDR5 over HX 370 plus 32 GB of soldered RAM I cannot grow.
| Buying class | Memory you can get | iGPU | NPU | Honest local-LLM job |
|---|---|---|---|---|
| HS mini PC (8945HS-style) | SODIMM, often 32–96 GB | Radeon 780M | None | 7B–14B if two sticks |
| HX 370 mini PC | Often 32–64 GB LPDDR5X, soldered | Radeon 890M | 50 TOPS | 7B–27B if RAM is 64 GB |
| Max+ 395 mini workstation | Up to 128 GB unified | Radeon 8060S, 40 CU | 50 TOPS | 27B daily; 70B if you accept slow decode |
| Mac mini M-series | Unified, config-locked | Apple GPU (Metal) | Apple Neural Engine | Quiet 8B–30B depending on GB |
Framework’s Donato Capitella explained why the giant SKU exists: “Ryzen AI Max brought 128GB of memory shared between the CPU and integrated GPU to a desktop system, making it possible to run local models that would not fit into the memory on most consumer graphics cards.” A 24 GB RTX 4090 is still faster on a 27B that fits. The 128 GB box is for models the 4090 cannot hold.
An “AI PC” title says less than the memory configuration. Check soldered versus SODIMM RAM, capacity, channel count, iGPU, and whether the vendor lets you raise GPU-visible memory.
How fast will chat feel on a mini PC versus a gaming GPU?
An 8B Q4 model on a dual-channel HS mini PC can keep up with reading. A dense 70B on Strix Halo often lands around 4 to 6 tokens per second in published roundups. Buy the box so the weights stay in the house, not because a clip called it a ChatGPT killer.
Framework’s guide explains the speed limit: “Strix Halo can hold large models in unified memory, but its 256GB/s memory bandwidth limits how quickly the GPU can read the weights.” GDDR on a midrange NVIDIA card is several times that bandwidth. The larger memory pool does not imply faster output.
A June 2026 comparison of Max+ 395 boxes collected Q4 generation rates: about 28 to 38 tok/s on an 8B, 8 to 12 on a dense 32B, and 4 to 6 on a dense 70B. Mixture-of-experts models look faster because fewer parameters fire per token. That same roundup cited gpt-oss-120b at 31 to 55 tok/s depending on the engine, with a –64 GB footprint. I did not rerun those benches. Treat them as someone else’s snapshot.
The weeknight version is simpler. You paste a 2,000-token function and wait for prefill. Then the answer drips. That drip is fine for a private PDF box on the LAN. It gets old if you iterate on code.
What storage and power specs should I check on a mini PC for local LLMs?
One terabyte of NVMe is the floor once you keep two models and Windows. A 70B Q4 file is already around 40 GB, and a 512 GB drive fills by Friday. Prefer two M.2 slots: OS on one, models on the other.
Power is the part people skip because mini PCs look like streaming sticks. AMD lists 45 to 120 W configurable TDP on the Max+ 395. Load sits well above idle. That is still a fraction of a 4090 tower. It is not zero. Check that the included brick matches the TDP you intend to set. A 100 W adapter on a 120 W BIOS profile is how you get random throttling and a forum thread titled “my LLM is slow.”
Ports matter more than the light bar. You want a 2.5 GbE jack if this box is a household server, USB4 if you might add a display, and a BIOS that exposes GPU memory. Wi-Fi 6 is enough for a desk. I would not make Wi-Fi 7 the reason I picked a 32 GB SKU over a 64 GB one.
Cooling shows up at 11 p.m. An HX 370 cube can run an 8B all day. A 70B decode for an hour is a different heat soak. Notebookcheck and Ars both treated the Framework Desktop as quiet enough for a desk. If the only review is a five-star “super fast AI PC” with no tok/s, skip it.
SODIMM HS boxes can grow. LPDDR Strix Point and Strix Halo boxes cannot. Ars filed soldered RAM under “The ugly” for a reason. You are buying a memory size, not a future-proof workstation.
What else do people ask before they buy a mini PC for a local LLM?
Before ordering the smallest “AI” mini PC, check RAM size, whether the RAM is soldered, and how long generation takes. Those specifications may change which model belongs in the cart.
Can an Intel N100 mini PC run a local LLM?
It can load a 1B to 3B model and crawl through a 7B if you have 16 GB and nowhere else to be. I would use that box for a home server, not for a daily assistant. The CPU will run Ollama. You will hate the wait. Put the next $200 into RAM and a 780M-class chip instead of into a second N100.
Is 32 GB enough in 2026?
Yes for 7B to 8B chat, and on some unified boxes a 27B Q4 if you close other apps. Framework’s own 32 GB recommendation is that 17.6 GB Qwen GGUF. It is not enough for a comfortable 70B. If the RAM is soldered, 32 GB is a commitment. If it is two SODIMMs, start with two sticks now so the bus is full.
Should I buy a Mac mini instead of a Windows mini PC?
If you already live in macOS and want quiet 8B to 30B inference, Metal on an M-series mini is a sane path. Ollama documents Apple GPU acceleration. You will not get CUDA, and you will not get 128 GB unless you pay Apple’s unified-memory tax. I would not switch ecosystems only to run llama.cpp.
Does a higher TOPS number mean a better local LLM box?
No. Two chips can both say 50 NPU TOPS and disagree by 96 GB of GPU-visible memory. llama.cpp cares about the pool the iGPU can read. TOPS is a Copilot subtitle. Ask for GB, bus width, and whether the RAM is soldered.
