Can Idle Macs Become a GPU Cloud? What an Apple Silicon Cluster Delivers

Yes, idle Macs can be pooled into something that behaves like a small GPU cloud, and a new preprint describes a working version of exactly that. But the honest answer to “should my team do this” is narrower than the headline: pooling is viable today for a specific kind of work, namely batch embeddings and Ollama-style LLM completions that can tolerate modest throughput and idle-machine availability, and it complements rented GPUs rather than replacing them.

One caveat belongs up front, before any number in this article. Every cluster-scale claim here comes from a single, non-replicated arXiv preprint, Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices, observed 2026-10-12. The paper reports no cluster-level throughput or latency benchmarks, no measured recovery performance under device churn, and no cost-per-token comparison; its evaluation is operational, centered on queue-time relief. Treat the system’s reported behavior as one team’s self-report from their own testbed, and check the paper’s deployment details against your own fleet before building anything on it.

What the Minions preprint actually claims

The paper describes Minions as “a dynamic and distributed virtual GPU cloud that aggregates idle Apple Silicon GPUs across macOS devices to execute large-scale machine learning inference workloads,” with the explicit goal that teams use existing hardware “without requiring additional GPU procurement” (arXiv:2610.12970). The motivation is concrete and, for anyone who has watched a job sit in a queue, familiar: “Prior to its adoption, some inference jobs experienced queue times exceeding ten hours due to limited centralized GPU availability.” That ten-hour figure is self-reported, so read it as the authors’ description of their environment rather than a measured industry baseline.

Two scope facts matter more than the architecture. First, the pool serves exactly two workload classes today, per the paper: “Minions currently supports two categories of GPU inference workloads on macOS devices,” embedding generation through the HuggingFace transformers Python API and LLM completion through Ollama, “enabling local inference without remote API calls” (arXiv:2610.12970). There is no fine-tuning claim here, and the paper says nothing about prefill-heavy batch generation. If your workload is training or high-throughput prompt processing, this system, as described, does not cover it.

Second, tenancy is a designed feature, not an afterthought. The paper states that “Minions supports team- or organization-scoped deployments backed by dedicated PostgreSQL databases and isolated executor pools” (arXiv:2610.12970). For teams whose real reason to avoid rented GPUs is data confidentiality rather than price, that isolation model is the actual product.

The real menu

A small ML team sitting on idle Macs has four practical options, and the preprint only speaks to one of them directly. The rest of this article maps the evidence onto all four, flagging where the mapping is inference rather than measurement.

The last row deserves emphasis. The natural comparison points for a Mac pool, llama.cpp’s RPC mode and exo-style device clustering, are not evaluated in any of the papers cited here. I would not write them off, but this evidence base cannot rank them, and neither should you without testing them against your own workload.

The per-node ceiling is the pool’s atomic unit

Whatever a pool of Macs delivers, it delivers in units of single-Mac performance, and that unit is now reasonably well measured. The vllm-mlx paper reports “throughput of up to 525 tokens per second on text models” for Qwen3-0.6B on an M4 Max (arXiv:2601.19139); Qwen3-0.6B is the smallest model in its benchmark suite, so expect larger models to land well below that ceiling. The same paper reports a 28x speedup on repeated image queries that cut multimodal latency “from 21.7 seconds to under 1 second” (arXiv:2601.19139). A comparative study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS finds these frameworks “still trail NVIDIA GPU-based systems such as vLLM in absolute performance,” while judging them “rapidly maturing into viable, production-grade solutions for private, on-device LLM inference” (arXiv:2511.05502).

Batching helps, but sublinearly. On an M2 Max, BERT-base inference under MLX rose from 8.02 ms at batch size 1 to 70.48 ms at batch size 32, roughly a 9x latency increase for a 32x batch increase, which the authors attribute to “efficient parallel processing capabilities of the M2 Max” (arXiv:2510.18921). That sublinear scaling is good news for a pool: a Mac that is idle except for your embedding batch can absorb real concurrency before it saturates.

Two arithmetic traps follow. First, do not multiply 525 tokens/s (arXiv:2601.19139) by the number of Macs and call it cluster throughput. Decode is memory-bound per device, so pooling parallelizes across requests rather than accelerating a single one; N Macs serve roughly N independent streams at per-device rates, and nothing in the evidence suggests more. Second, the Apple Neural Engine does not add to the pool. The ANE is “the fixed-function matrix accelerator” in Apple silicon “exposed to applications only through the Core ML model framework” (arXiv:2606.22283). Stacks like MLX and Ollama draw on the GPU, so when you count your fleet’s usable capacity, count GPU capacity only and ignore the ANE headline on the spec sheet.

What distribution actually buys: load per device, not free speed

The strongest quantitative evidence on what sharding a model across devices accomplishes comes from an edge-inference study, not from Apple hardware. In it, “monolithic GPT-2 Large inference requires approximately 22 s of CPU time per generated token on a single device,” while “distributed execution reduces the per-device CPU time to 1.8 - 4.4 s per token, depending on shard size” (arXiv:2603.28622). Read that carefully: the win is per-device load relief, a 5x to 12x reduction in what each machine carries, which lets a device sustain inference alongside other work. It is not a promise that total latency drops by the same factor, and the numbers come from GPT-2 Large on non-Apple edge devices, so the magnitude transfers to a Mac fleet only qualitatively.

The uneven payoff across workloads traces to a phase asymmetry that applies to every heterogeneous cluster, Mac-based or not. “The prefill phase is typically compute-bound, and the decode phase is memory-bound,” so “in a heterogeneous cluster, this behavior leads to decoupled bottlenecks, where the bottleneck stage differs between phases” (arXiv:2606.18600). Practically: embedding generation and short-completion workloads lean on prefill-like compute that a pool of idle GPUs parallelizes well. Long-form generation is decode-dominated, bottlenecked by memory bandwidth on whichever device holds the model, and no added Mac makes a single such request faster. That limit is about per-request speed, not about pooling itself: under request-level parallelism, a pool adds aggregate throughput for decode-bound completions too, by running more streams at once. Each stream still runs at the speed of the device that accepted it.

Where the bottleneck moves

The clearest sign that Mac pooling is early: the leading Apple-native serving stack cannot yet do it. The vllm-mlx paper states the framework “currently supports only Apple Silicon, limiting deployment to macOS environments” and lists “distributed inference across multiple Apple Silicon devices via network-connected Mac clusters” under future directions (arXiv:2601.19139). Minions works around this by parallelizing at the request level (each device runs Ollama or a transformers embedding model locally, end to end) rather than splitting one model across machines. That sidesteps the interconnect problem but inherits a different one: your pool’s throughput is the sum of per-device rates, and its latency floor is the slowest device that accepts a job.

Device churn is handled by design, though not yet by measurement. A pool built on idle machines is a pool whose members leave when an employee opens a laptop or a lab machine gets reimaged, and the paper treats executor failure as expected in this environment: executors running subtasks send heartbeats at five-minute intervals, and when one goes unresponsive, whether from network loss, device sleep, power-off, or the device’s owner reclaiming the machine, “the Minion Watchdog reclaims unfinished subtasks and makes them available for re-execution.” The paper claims this design lets Minions “tolerate ad hoc interruptions of individual executor machines without manual intervention” (arXiv:2610.12970). What it does not provide is measured recovery performance: no recovery times, no success rates under churn, and a five-minute heartbeat interval that bounds how quickly a vanished executor can even be detected. If your jobs cannot tolerate a restart, watch the requeue path under your own fleet’s real churn before trusting it.

The tenancy argument is stronger than the speed argument

The most durable reason to pool Macs is not performance; it is that inference never leaves. Minions runs Ollama and embedding models “locally on Apple Silicon GPUs” with “no remote API calls,” and it isolates tenants through “dedicated PostgreSQL databases and isolated executor pools” (arXiv:2610.12970). Read the privacy property narrowly, though: the paper states that “Minions uses AWS S3 as the primary storage layer for GPU inference inputs and outputs,” so task inputs and results pass through an external object store even though the model call stays on-device. For a team whose constraint is keeping prompts and documents away from third-party model APIs, the design fits. For a team under contractual data-residency requirements, the S3 path is part of the deployment and deserves the same scrutiny as any cloud storage. The comparative study’s conclusion aligns: Apple Silicon stacks are maturing specifically “for private, on-device LLM inference” (arXiv:2511.05502), not for beating datacenter serving.

Build-vs-rent decision rules

Putting the evidence together, here is how I would sort workloads, with the caveat that the cluster-side of this rests on one unreplicated preprint:

Pool the Macs when the work is embeddings or Ollama-compatible completions, jobs would otherwise queue (the authors’ ten-hour queue is extreme, but any multi-hour wait changes the math), confidentiality favors local inference with org-scoped isolation, and the workload tolerates per-device throughput on the order of the single-Mac ceiling, up to 525 tokens/s on an M4 Max for the smallest tested text model, Qwen3-0.6B (arXiv:2601.19139), with sublinear batch scaling helping concurrency (arXiv:2510.18921).

Keep renting when the workload is latency-critical interactive serving, throughput-critical production traffic, fine-tuning, or prefill-heavy batch generation. On absolute performance the evidence is unambiguous that Apple Silicon frameworks trail NVIDIA-backed vLLM serving (arXiv:2511.05502), and the Minions paper reports nothing that closes that gap at cluster scale.

Treat as an open question anything involving llama.cpp RPC, exo, or splitting a single large model across Macs. The sharding evidence says distribution cuts per-device cost rather than adding speed (arXiv:2603.28622), the phase analysis says decode-bound work gains no per-request speed (arXiv:2606.18600), and the leading MLX stack has not shipped multi-Mac inference (arXiv:2601.19139).

What would change this picture

This article’s verdict is falsifiable, and the checklist is short. Replication of Minions by a second team, with cluster-level throughput and measured recovery under realistic device churn, would either confirm or deflate the self-reported design. Published cost-per-token numbers for a Mac pool against a comparable rented-GPU setup would replace the qualitative build-vs-rent rules here with arithmetic. And if vllm-mlx or another MLX stack ships the network-connected Mac clustering it currently lists as future work, the request-level parallelization constraint relaxes and the per-node ceiling section needs a rewrite. Until at least the first of those lands, the practical posture is: pilot a pool on throughput-tolerant, privacy-sensitive embedding and completion work you already have, measure your own fleet’s real availability, and keep the cloud GPU budget for everything that has a latency number attached to it.

Frequently Asked Questions

Does the Apple Neural Engine contribute to the GPU pool capacity?

The ANE is “the fixed-function matrix accelerator” in Apple silicon “exposed to applications only through the Core ML model framework” (arXiv:2606.22283). Stacks like MLX and Ollama draw on the GPU, so when you count your fleet’s usable capacity, count GPU capacity only and ignore the ANE headline on the spec sheet.