DGX Spark Explained: What Runs on 1, 2, 3 or 4 Linked Units

"Will it run this model?" is the first question anyone asks about a DGX Spark, and the honest answer is "it depends on three things". Those are how many parameters the model has, how many bytes you store per parameter, and how much context you want to keep in memory. How fast it runs depends on a fourth thing that's easy to miss: memory bandwidth.

This guide covers what one Spark is, what changes as you link two, three or four, and how to think about speed, quantization and serving. Specs and capacity figures come from NVIDIA's own published numbers, and we say clearly where a figure is our estimate.

One DGX Spark, in numbers

Each unit has:

• Superchip: NVIDIA GB10 Grace Blackwell
• CPU: 20-core Arm (10× Cortex-X925 + 10× Cortex-A725)
• Memory: 128 GB LPDDR5x unified memory, 273 GB/s
• AI compute: up to 1 PFLOP FP4 (with sparsity)
• Storage: 4 TB NVMe, self-encrypting
• Networking: ConnectX-7, 2× QSFP, up to 200 Gb/s each
• OS: DGX OS (Ubuntu-based), ARM64

Sources: the NVIDIA DGX Spark User Guide and the NVIDIA DGX Spark product page.

"Unified" is the key word. The CPU and GPU share the same 128 GB, so a model isn't limited by a separate, smaller pool of VRAM. That's why a desk-sized machine can hold models that would otherwise need more than one datacenter GPU.

How much model fits: the bytes-per-parameter rule

A model's weights take roughly parameters × bytes per parameter:

• FP16/BF16: ~2 bytes per parameter. A 70B model is ~140 GB, which doesn't fit on one Spark.
• FP8: ~1 byte per parameter. 70B is ~70 GB, which fits with room to spare.
• FP4: ~0.5 bytes per parameter. 200B is ~100 GB, which fits, but the space left over is small.

The weights aren't the whole bill. Inference also needs a KV cache for the context it's holding, which grows with context length and the number of concurrent requests, plus memory for the runtime and the OS. That's why every figure below is approximate: a model that "fits" at a short context may not fit at a long one.

Quantization is the trade you make to go down that list. Lower precision means smaller weights and more parameters per gigabyte, at some cost in output quality that varies by model and method. Blackwell's tensor cores support FP4 natively, which is why FP4 is the headline format for the largest configurations.

What you can run, by configuration

All figures are approximate and depend on quantization and context length.

What NVIDIA publishes: the product page lists up to 200B parameters on one 128 GB unit, up to 400B on two, and up to 700B on four, plus fine-tuning up to 70B on one unit (product page). The User Guide says "up to 200 billion parameters (or 405B for dual-Spark configuration)" (User Guide). NVIDIA doesn't publish a three-unit figure, so the ×3 row is our own estimate, scaled from the same bytes-per-parameter arithmetic.

Why 273 GB/s sets your speed

Capacity tells you whether a model loads. Bandwidth tells you how fast it talks.

When a model generates text, each new token needs (roughly) the whole set of active weights read from memory. For a single request, that makes generation memory-bound: the compute units spend much of their time waiting for data, not doing arithmetic. That gives a useful rough ceiling:

tokens per second (one stream) ≲ memory bandwidth ÷ bytes of active weights read per token

Some illustrative arithmetic, for dense models on one unit:

• 70B at FP8 ≈ 70 GB of weights: 273 ÷ 70 ≈ 3.9 tokens/s ceiling
• 70B at FP4 ≈ 35 GB: 273 ÷ 35 ≈ 7.8 tokens/s ceiling
• 8B at FP8 ≈ 8 GB: 273 ÷ 8 ≈ 34 tokens/s ceiling

These are theoretical upper bounds, not benchmarks. Real numbers come in lower because of KV-cache reads, kernel efficiency and software overheads. They show the shape of the trade-off, though:

• Quantization speeds you up as well as shrinking the model. Halving the bytes per parameter roughly halves what each token has to read.
• Mixture-of-experts (MoE) models suit the Spark well. An MoE model stores many parameters but uses only a fraction of them per token, so it needs the capacity of a big model and the bandwidth of a smaller one. That's why the very large classes in the table are often MoE models.
• Batching raises total throughput. Serving several requests at once lets each read of the weights serve many tokens, so total tokens per second across users rises even though each user's stream doesn't get faster.

The fair comparison: a datacenter GPU with HBM has several times this bandwidth, so it generates faster on any model that fits in its memory. The Spark's advantage is fitting large models in one quiet box you own, not beating an H100 on speed.

Linking units: the four topologies

Each Spark has two ConnectX-7 QSFP ports at up to 200 Gb/s each. How we cable them depends on the number of units:

×1: single unit

One unit, with both ConnectX-7 ports free for later expansion. This is the simplest and fastest setup per token, because nothing crosses a cable.

×2: direct link

Two units joined by one direct ConnectX-7 cable (200 GbE), with no switch. This is the configuration NVIDIA documents for dual-Spark use, and it takes you from 128 GB to 256 GB.

×3: triangle

Three units in a ConnectX-7 triangle: each unit is cabled to the other two, using both of its ports, with no switch. Every pair is one hop apart. Note that NVIDIA's guidance recommends connecting Sparks in a power-of-2 number for the best load balance and parallelism (product page). Some parallelism schemes split work more evenly across two or four units than across three, and the capacity figure for ×3 is our estimate, not NVIDIA's. Three units make sense when you need the memory and your serving stack handles an uneven split well.

×4: switchless ring

Four units in a switchless ConnectX-7 ring: each unit is cabled to its two neighbours, using both of its ports, with no switch. Traffic between opposite units goes through a neighbour, so those pairs are two hops apart, not one. NVIDIA's own four-unit reference uses an external 200 GbE switch (product page). The ring avoids the switch, at the cost of that extra hop for opposite pairs. If your workload needs every unit one hop from every other, switched configurations are available on request.

Every multi-unit configuration is cabled and validated by us before handover.

Splitting a model is a software decision

Once a model is bigger than one unit, it has to be split across units: pipeline parallelism (different layers on different units), tensor parallelism (each layer divided across units), or expert placement for MoE models. Each sends different amounts of traffic over the ConnectX-7 links, and the network is far slower than local memory.

So capacity grows almost linearly as you add units, but speed doesn't. How well a split model performs depends on the interconnect, the topology, and the inference engine you use and how you configure it. Expect a model split across four units to be slower per token than one that fits on a single unit, and test your actual model before committing to a layout.

Beyond inference

• Fine-tuning. NVIDIA rates one unit for fine-tuning models up to 70B with memory-saving methods such as LoRA and QLoRA. Smaller models can be fully fine-tuned.
• Multi-agent development. Run several models and agents side by side on hardware you control, with no per-token bill and no data leaving your machine.
• Local inference serving. Serve models to your team over a private endpoint with tools such as vLLM, TensorRT-LLM or Ollama.

Serving it like an API

Most of these tools can expose an OpenAI-compatible endpoint, so existing SDKs and agent frameworks can call your Spark just by changing a base URL. A useful pattern is to put a gateway in front of that endpoint. LocalRouter is a local OpenAI-compatible API gateway that routes requests across AI providers. You can send traffic to your own Spark first and fall back to a hosted provider when the model isn't loaded locally or the box is busy. Your application code doesn't change, and the hosted bill covers only the overflow.

Choosing a size

• 1 unit, $6,500: models up to ~70B at FP8 comfortably, and up to ~200B at FP4. Fine-tuning up to 70B with LoRA or QLoRA. The best tokens per second per dollar, because nothing is split.
• 2 units, $13,000: the ~405B class at FP4, or two independent 128 GB machines when you aren't splitting a model.
• 3 units, $19,500: ~600B class at FP4 by our estimate. Best when you need the memory and your software handles an uneven three-way split.
• 4 units, $26,000: up to ~700B at FP4, including 671B MoE models, in a switchless ring.

Management and rack is $99 per unit per month, so a four-unit ring costs $396 a month to manage. Prices exclude taxes and import duties. Need more than four? See larger orders.

Next steps

• Compare configurations: ×1, ×2, ×3, ×4, or all hardware.
• See pricing and what management includes.
• Weighing it against cloud GPUs? Read rent vs own over 36 months.
• Tell us the model you want to run and we'll help you size it.
• Already have one? First steps on DGX Spark covers logging in, a first model with Ollama or vLLM, and working with linked units.