What is Strix Halo? Six months of local AI on AMD hardware

You've probably seen LinkedIn posts about a Mac Studio with 128 gigabytes of memory, or about companies running Nvidia DGX Spark boxes. They let you run larger open language models locally, and both rest on the same idea: unified memory, which gives a model plenty of room without a data-center-class graphics card. But there's a third option many people don't know about: AMD's Strix Halo. We bought one six months ago to complement our local GPU resources. Here we've gathered what we learned about what it can do, where it differs from its peers, and who it's for.

Three ways to run large models locally

Broadly, there are three options:

Discrete GPU (dGPU). Fastest, but memory is scarce: 16-32 GB on a consumer card, which caps model size. Large models need several cards, and graphics cards aren't cheap.
Apple Silicon (unified memory). Plenty of memory, and very high memory bandwidth on the top models, but the ecosystem is macOS/Metal.
Compact unified-memory mini PCs. A large shared memory pool in a small box, with the processor and GPU on the same chip. For example Nvidia's DGX Spark (GB10, ARM) and AMD's Strix Halo (x86).

In practice the choice comes down to three factors: the size of the model you want, speed, and your hardware budget. No option is best on all three; what matters is which factor you put first.

What Strix Halo is

Our machine is a Minisforum MS-S1 MAX (around €2,700 + VAT): an AMD Ryzen AI Max+ 395 ("Strix Halo"), an integrated Radeon 8060S GPU, 128 GB of LPDDR5x unified memory (around 256 GB/s bandwidth) and a 2 TB NVMe. We replaced the bundled Windows install with Ubuntu Server, and models are served with Ollama. You can give the GPU around 100 gigabytes of that memory, so the box can hold a model that would otherwise need a data-center card, while the machine draws about as much power as a laptop: we measured the chip pulling around 10 watts at idle and at most around 130 watts under heavy inference.

Differences from its peers, and the limits that follow:

vs. Nvidia DGX Spark: Spark is ARM-based and brings the CUDA ecosystem, which is the most mature in terms of software support. Halo is x86, so it runs ordinary Linux and ordinary tools with no special adaptation, and it's clearly cheaper (Spark is currently in the €4,500 + VAT class). Memory bandwidth is in the same ballpark on both, so the speed profile is similar.
vs. Mac (Apple Silicon): A top-end Mac can offer several times the memory bandwidth, and thus the speed, but at a comparable memory capacity the price is considerably higher. Halo is cheaper and runs x86 Linux, but is slower in bandwidth.
Shared limitation: all unified-memory boxes are bandwidth-limited compared to a GPU. There's plenty of memory, but its speed (not its capacity) decides real-world performance.
Halo's own limitation: the software stack (ROCm and Ollama on this chip) is the youngest and roughest of these, and versions differ in what loads and how much memory it takes. Staying on an old version is not the safe option either, since new model architectures need newer runtime support, so test each update yourself.

Speed: what you actually get

The bandwidth limit shows clearly in performance: generating text (decode) is fast, but reading the input (prefill) is slow, because the compute is at integrated-GPU level.

A concrete example with the best-known open-weights model, OpenAI's GPT-OSS 120B, which we've used the most as a general model: text generation runs at around 36 tokens per second, and lighter models reach close to 60, which is plenty for interactive use. Processing long context (large files, long conversations), on the other hand, means prefill takes time, and agentic work, which reads a growing context on every turn, suffers the most from this. In practice a coding task that a frontier cloud model handles in seconds may take a minute or two locally, mostly in processing the input.

Architecture is decisive. GPT-OSS is a Mixture-of-Experts model: it activates only a fraction of its parameters per token, so it's fast even though it's large overall. A dense model of the same size would read every parameter for each token and drop to single-digit tokens per second, in practice too slow. That's why it's worth choosing MoE models for this machine.

What other models fit

Thanks to the unified memory, the selection is wide. Among others, we run:

GPT-OSS 120B (about 65 GB) as a general model and for heavier reasoning.
Qwen3-Coder-Next 80B (about 52 GB) for agentic coding.
Qwen 3.6 35B (about 24 GB), Gemma 4 26B (about 18 GB) and other lighter models for vision and general tasks.

Quantization is worth choosing deliberately. We tested both the four-bit (about 52 GB) and eight-bit (about 85 GB) versions of the Qwen3 coding model. The eight-bit was about 1.5-2 times slower with no noticeable quality gain in coding, so we stayed with four-bit. A well-chosen mid-size model often beats a larger one in both speed and practicality.

An always-on local endpoint, and why usage is worth optimizing

The biggest practical value for a company is that the machine is always on and offers one persistent endpoint (the Ollama API) that every tool and service in the house can connect to. The same local model is continuously available for different purposes without a cloud call.

An important reality: in practice only one large model is loaded at a time. That's why usage is worth optimizing: choose the model and configuration to fit the use you prioritize, and don't try to keep several large models loaded in rotation (loading GPT-OSS 120B into memory took about 40 seconds in our measurement). By default we keep one general model warm, and specialized models (such as the one best suited to agentic coding) are brought up when they're needed.

At the moment the machine hosts, among other things, always-on agents (such as OpenClaw and Hermes) that handle scheduled and orchestrated tasks, a set of self-hosted internal services, and the processing of client project data when that data shouldn't leave the house. All of this runs in securely scoped and isolated environments.

Coding with local models: the tool has to fit the model

We wanted to know whether local models can do real software work. They can, with the right setup and honest expectations. Local open models are weaker than frontier cloud models at following instructions precisely, so it's better not to give them free rein but to constrain them: the agent runs on an isolated copy of the code, and automated tests decide whether the result is fit to merge into the software.

We also tested several terminal-based coding-agent harnesses, including Codex CLI, Qwen Code, Aider and OpenCode. The clearest finding was that what matters is the harness's editing method, not whether the model is "the same family" as the tool. For example, Qwen Code and OpenAI's GPT-OSS work well together. In the end we settled on using OpenCode the most, because it fit our way of working best and works with every model. Someone else might prefer Aider, whose functions differ slightly, but that's largely a matter of taste, as long as the tool and the model fit together.

The most important thing to remember is to set up the agentic stack carefully. Most "it doesn't work" situations were configuration problems, not model limits. One typical pitfall is the context-window default. Ollama gives models only 4096 tokens by default, which is too small for agentic tool use and makes tool calls fail. Once the context is raised to fit the model, the problem disappears.

Honest limits, and where the benefit actually comes from

Local models aren't as good as the best frontier models, and the choice of harness doesn't change that. Tasks that demand the heaviest reasoning are best handed to frontier models in the cloud where possible. The biggest benefit from a local machine comes when local models are placed inside workflows where you want to keep the data in-house, the runs are often repetitive or long-running, and you run a lot of them. Then the economics shift decisively: when a query doesn't go to the cloud, the running cost is essentially the price of electricity. Often the best pattern is a hybrid, where an external frontier model orchestrates the high-level task and a local model carries out its repetitive, data-sensitive parts.

So who is Strix Halo for? If you want to run fairly large open models privately, affordably and at reasonable speed on ordinary x86 Linux, it's the cheapest way to get about 100 GB of model memory. If instead you need the throughput of a discrete GPU, very long contexts, top-tier memory bandwidth or a fully mature software ecosystem, look instead at a Mac, multiple GPUs or the cloud, or complement Halo with them. For us, Strix Halo has found its place: part of the everyday workload runs on the local machine, and the cloud is used judiciously for what it's unmatched at.

Looking for expert solutions? Discover Crestia's professional services today.