DGX Spark: Budget memory, bandwidth and workload together

DGX Spark’s 128 GB configuration offers model capacity without guaranteeing interactive throughput. Memory capacity, memory bandwidth, model architecture, precision, and concurrency all belong in the same decision.

NVIDIA’s DGX Spark product page advertises up to 1 PFLOP of FP4 tensor throughput and inference on models up to 200 billion parameters. These are compute and model-capacity claims, rather than a measured tokens-per-second result. The current page also labels a 64 GB version as coming soon; the 128 GB hardware guide and the older benchmarks discussed here should not be silently applied to that configuration.

DGX Spark: Budget memory, bandwidth and workload together

The 128 GB hardware guide describes the GB10 Grace Blackwell system, a 20-core Arm CPU, a 256-bit memory interface, and unified LPDDR5X memory. NVIDIA specifies 273 GB/s of memory bandwidth. The CPU and GPU share the memory pool, so the total is also a system resource; the full capacity is not an unconditional weight budget. (DGX Spark hardware guide)

The hardware guide distinguishes inference capacity on one Spark from the larger dual-Spark configuration. Its two-system 405B example is a distinct setup with networking and partitioning costs. A network connection does not turn two machines into one local-memory allocator or prove that their throughput doubles. (DGX Spark hardware guide)

The porting guide adds another practical constraint: Spark is an Arm-based system-on-chip platform, rather than an x86 workstation with a discrete GPU. The intended container, native package, runtime kernel, and model artifact need to support that platform. A CUDA feature name alone does not establish that an existing x86 deployment will run unchanged. (DGX Spark porting guide)

Use bandwidth arithmetic as an estimate

For a simple dense, batch-one decode estimate, dividing memory bandwidth by weight bytes read for a generated token gives a useful rough ceiling. If a hypothetical dense model reads 35 GB of weights per token, 273 GB/s divided by 35 GB is about 7.8 tokens per second before cache traffic and other overhead. This is arithmetic under stated assumptions, not a benchmark of a particular 70B artifact.

The assumptions are the important part. A dense model and a mixture-of-experts model need different accounting. An MoE checkpoint may keep many weights resident while selecting only some experts for a token. Quantization changes storage and kernel work. Batching can reuse weight reads across sequences, and speculative decoding can change how many accepted output tokens emerge from a target-model evaluation. A single full-checkpoint-size formula should not be treated as the measured speed of every architecture.

NVIDIA’s performance article separates fine-tuning, image generation, and inference workloads and reports configurations for them. Its fine-tuning tokens-per-second figures, for example, describe training settings and are not streaming decode rates. Check which operation a number measures before comparing it with a local chat experience. (NVIDIA DGX Spark performance configurations)

What the dated measurements establish

LMSYS’s October 2025 review measured several devices, model artifacts, and runtimes. It lists SGLang tests with batch sizes from 1 to 32, Ollama tests at batch one, and different precisions by model. The authors explicitly warn that software changes can make those early results outdated. These are useful dated observations, not a performance guarantee for the current software stack. (LMSYS DGX Spark methodology and results)

The review reports Llama 3.1 70B in FP8 at 803 tokens per second for prefill and 2.7 for decode. It also reports a DeepSeek-R1 14B FP8 test at batch eight with 2,074 and 83.5 tokens per second. Different models and batch settings prevent these rows from becoming a single universal Spark rate. Prefill, aggregate output, and one user’s token latency answer different questions. (LMSYS DGX Spark methodology and results)

The useful inference is modest: fitting a large model and serving it with the desired latency are separate achievements. The old review can help select a test case, while an intended deployment needs its own current runtime, artifact, and workload measurement. For a related model-specific capacity discussion, see DeepSeek R1 deployment on local hardware.

Evaluate the job you intend to run

For one interactive user, measure time to first token, inter-token latency, and total completion time at the intended prompt and output lengths. For a shared endpoint, add concurrency, queueing, and the fraction of requests meeting the service target. Keep peak memory and any offloading behavior beside those results. These are proposed evaluation checks, rather than measurements made for this article.

Treat these as conditional decision axes. They do not establish that one memory technology, operating system, or vendor is always the best purchase. The same care applies to the related MLX vs llama.cpp on Apple Silicon discussion: a runtime result is tied to its model and configuration.

Price can change the decision, but a historic price increase is not today’s quote or a total-cost comparison. The related Apple hardware pricing discussion concerns another purchasing question. Get a current quote for the exact Spark configuration, support arrangement, and software workload instead of deriving value from a gigabyte count alone.

Spark’s unified-memory capacity and NVIDIA tooling can be useful for a supported local workload. The evidence for buying it is that the exact configuration fits the artifact and meets the required job’s limits. A capacity advertisement, an ideal bandwidth calculation, and a dated benchmark are three different inputs to that decision.