Moonshot’s Kimi K3, introduced 16 July 2026, is a 2.8-trillion-parameter mixture-of-experts model with a 1-million-token context window. As a sparse MoE it activates only a subset of its experts per token, so the per-token compute footprint is a small slice of a model whose total weight still dwarfs a laptop’s 64 GB memory ceiling. The result circulating this week, a single-stream local run on Apple Silicon, is a feasibility proof for offline work, gated by roughly 400 GB/s of unified-memory bandwidth. It is not a serving recipe, and parameter count is not the wall.
What does “2.8T MoE” actually load per token?
Per token, a sparse MoE touches only the experts its router selects plus the shared attention layers; the remaining experts sit idle for that token. This is the entire reason a laptop run is conceivable, and also the reason it is painful.
A dense 2.8T model would be a non-starter on consumer hardware no matter how you quantized it, because every parameter participates in every forward pass. A sparse MoE splits the problem in two. The compute side gets cheap: each token’s arithmetic involves only the active experts plus the shared layers. The memory side does not get cheap at all. Every expert has to be reachable on any given token, because the router decides which subset fires only at runtime. You cannot know in advance which slice of the 2.8T you will need next.
That runtime routing decision is the fact that governs everything about local deployment. On a datacenter GPU node with a terabyte or more of HBM, the answer is to keep all experts resident and let the router do what it wants. On a 64 GB laptop, residency is not available, so the deployment question becomes a caching question: which experts are hot, how often does the working set change, and what does a miss cost.
The rest of the model’s spec compounds the pressure. K3 ships with native multimodal understanding and a 1-million-token context window. Long contexts add KV-cache or state memory on top of the weight footprint, and whatever attention architecture Moonshot uses to keep that state manageable, the arithmetic still applies: the context state comes out of the same 64 GB pool the weights are already overflowing.
What can an M1 Max actually hold?
An M1 Max holds 32 GB or 64 GB of unified LPDDR5X memory with approximately 400 GB/s of bandwidth shared between CPU and GPU. Everything the model needs, weights, activations, context state, the OS, your terminal, lives in that single pool.
The unified-memory design is why this conversation exists at all. The SoC and DRAM are mounted together in a system-in-package, so the GPU reads model weights directly from the same DRAM the CPU uses, with no PCIe transfer and no separate VRAM budget. A 64 GB M1 Max presents more directly addressable memory to the GPU than most discrete cards shipping today. That makes Apple Silicon disproportionately interesting for local inference relative to its compute, which is modest: the 57-billion-transistor M1 Max with its 32-core GPU is built for efficiency, not throughput races with discrete cards.
For sparse MoE decode, modest compute is rarely the binding issue. Decode is memory-bound almost by definition: each generated token requires reading the active weights once, and the arithmetic per byte read is small. The M1 family’s scaling path tells you where Apple thinks the constraint sits. The M1 Ultra fuses two M1 Max dies with roughly double the bandwidth and configurations up to 128 GB: more capacity, but proportionally more bandwidth. Nobody ships an M1 variant with 64 GB and the M1 Pro’s 200 GB/s. The vendor’s own product line is an admission that capacity without bandwidth is dead weight.
So the M1 Max envelope is fixed and unforgiving. Sixty-four gigabytes of residency, 400 GB/s of movement. Every deployment decision for K3 on this machine is a negotiation with those two numbers.
Why doesn’t INT4 close the capacity gap?
Quantizing K3 to INT4 shrinks its 2.8 trillion parameters to roughly 1.4 TB of weights, still about 22 times the M1 Max’s 64 GB ceiling. The gap is not closeable by compression alone.
The arithmetic leaves no room for optimism about residency. Even an aggressive 2-bit quantization, at the edge of what keeps a model coherent, lands near 700 GB. A 64 GB machine could hold perhaps 4 to 5% of the INT4 model if you dedicated every byte to weights, which you cannot, since activations, context state, and the operating system all draw from the same pool. Realistically the resident weight budget is somewhere in the 40 to 50 GB range, holding on the order of 3% of the model.
This is where the capacity problem converts into a bandwidth problem. If only a few percent of experts can be resident and the router wants a specific subset per token, then either the router’s choices cluster conveniently on the resident set, or the system swaps experts in from SSD on demand. NVMe storage bandwidth is a small fraction of the 400 GB/s the memory bus offers, so every swap is paid at a much worse rate than in-memory reads. Offloading trades the capacity problem for a bandwidth and latency problem, and it can thrash under MoE routing.
Worth noting what K3 is not. Moonshot shipped earlier Kimi releases, including K2.5 in January 2026, before K3, and local-deployment experience with those models does not transfer to a 2.8T flagship. K3 is the model that overflows a laptop twenty-fold.
Why is bandwidth, not parameter count, the wall?
The wall is bandwidth because MoE decode requires moving the active expert weights into compute reach for every token, and the router’s choices can change from token to token and layer to layer. Parameter count determines what you must store; bandwidth determines how fast you can generate.
This is a decode wall specifically. Prefill, the phase that ingests the prompt, is compute-heavy and reads each expert once across many tokens, so it amortizes the memory cost. Decode generates one token at a time and pays the full weight-read on each, which is why the per-token generation ceiling, not the prompt-processing speed, is what a laptop run actually hits.
Consider the two regimes. In the optimistic regime, the token stream has locality: consecutive tokens and consecutive layers keep reusing a working set of experts that fits in the resident cache. Decode speed then approaches the classic memory-bound bound, bytes-read-per-token divided by 400 GB/s. If the per-token hot set is a few tens of gigabytes of INT4 weights, the ceiling lands in the single-digit to low-double-digit tokens per second range. That is an illustrative bound from the architecture math, not a measurement; no benchmark of K3 on Apple Silicon has been published in any source available for this piece.
In the pessimistic regime, the router’s choices drift. Each new token pulls in experts that were evicted, the system pages them from SSD at a fraction of memory bandwidth, and decode stalls behind storage latency. This thrashing pattern has a nasty property: it is workload-dependent. A repetitive coding session where the router settles into a stable expert set might hum along. A context that jumps between domains, languages, or modalities, remember K3 is natively multimodal, keeps invalidating the cache. Two users running the same model on the same Mac can see wildly different throughput for reasons that have nothing to do with the hardware.
The standard mitigations are all forms of prediction. Speculative prefetch tries to guess which experts the next layer or next token will want and starts the SSD reads early. Expert caching policies pin the most frequently routed experts and let the long tail swap. Some systems bias the router itself, nudging token assignments toward already-resident experts at a small quality cost. None of these change the underlying constraint. They shift the hit rate of a cache that is, at best, a few percent of the data it fronts. The 400 GB/s bus and the gap to storage bandwidth set the terms; everything else is cache engineering.
This is also why the M1 Ultra comparison matters practically and not just as trivia. Doubling to 128 GB roughly doubles the expert cache, which can move a workload from thrashing to tolerable, and doubling bandwidth halves the cost of every read, resident or prefetched. If you are choosing hardware specifically for local MoE work, the M1 family’s top bin is a categorically different proposition from the Max, not an incremental one.
When does K3-on-Mac make sense, and when does it fail?
K3 on an M1 Max makes sense for single-stream, latency-tolerant, offline work: prototyping prompts and agent loops against the real model, running private data through a frontier-adjacent system without an API call, and evaluating whether K3’s behavior suits your workload before committing to hosted pricing. It fails for anything resembling serving.
| Configuration | Unified memory | Bandwidth | K3 at INT4 (~1.4 TB) | Realistic role |
|---|---|---|---|---|
| M1 Max 32 GB | 32 GB | ~400 GB/s | ~2% resident; heavy SSD swap | Impractical beyond demos |
| M1 Max 64 GB | 64 GB | ~400 GB/s | ~3% resident; cache-dependent | Single-stream prototyping, offline batch |
| M1 Ultra 128 GB | 128 GB | ~800 GB/s | ~7% resident; far less thrash | Comfortable single-stream; still not serving |
| Any Mac, concurrent requests | Shared pool | Shared bus | Working sets multiply per stream | Fails: bandwidth divides, thrash compounds |
The concurrency row deserves emphasis because it is the misreading most likely to cost someone money. Serving multiple simultaneous requests is worse than splitting bandwidth: it multiplies the expert working set, because two streams rarely route through the same experts at the same layers. A cache that was marginal for one stream collapses for two. The community demonstration, whatever its actual throughput, was by all available accounts a single-stream exercise.
Nobody runs K3 on a laptop because it is the best model available. The appeal is local access: privacy, no per-token meter, and the freedom to experiment against a frontier-scale model without an API in the loop. If your workload needs the frontier, you are calling an API regardless of what fits in unified memory.
What still needs to be measured before you trust this?
Everything performance-related about K3 on Apple Silicon is currently derived, not measured. No fetched source for this article demonstrates the M1 Max run, reports tokens-per-second figures, or documents the quantization and offload strategy used, and the community signal behind the news peg did not surface in the pages actually retrieved. The honest status of this whole topic is architecture math plus one anecdote.
That gap cuts both ways. The pessimistic case, thrashing under a drifting router, is derived from the same math as the optimistic case, and real workloads may land closer to either pole. Whatever attention architecture K3 uses may interact with caching in ways that plain parameter arithmetic misses. The 1M-token context is a headline feature and a complete unknown locally: even if weights were free, context state for long documents comes out of the same pool, and nobody has published what a 500K-token prompt does to a 64 GB machine.
The durable lesson survives the missing benchmarks, because it is about the pattern rather than this specific model. Frontier MoE models will keep arriving with total parameter counts that dwarf consumer memory and active footprints small enough to tease at feasibility. Every one of them will present the same decision: the sparse-active slice determines whether a laptop run is conceivable, the unified-memory bandwidth determines whether it is tolerable, and the storage-to-memory bandwidth gap determines whether it thrashes. Parameter count is marketing. Bandwidth is the spec sheet line that decides what your machine can actually do.
For practitioners weighing this today, the verdict is the feasibility map, not a throughput number. A 64 GB M1 Max running INT4 K3 through SSD expert offload is a legitimate prototyping and offline-batch tool, roughly comparable in role to a slow private endpoint you don’t pay per token. The M1 Ultra’s doubled capacity and bandwidth move the experience from marginal to comfortable for the same single-stream workloads. Concurrent serving is off the table on any of this hardware, and anyone planning around a tokens/s figure should wait for a measured one, because right now that number does not exist in any verifiable source.
Frequently Asked Questions
Does K3’s 1M-token context window fit in 64 GB without swapping?
No. A 1M-token context at K3’s precision requires roughly 2 GB of KV cache alone, which is 3% of the M1 Max’s total pool. When combined with the resident weight cache and OS overhead, the remaining headroom is insufficient for long prompts without triggering immediate SSD swaps that stall generation.
How does K3’s expert routing differ from Mixtral 8x7B?
Mixtral 8x7B activates 2 of 8 experts per token, keeping the active footprint small and predictable. K3 activates 16 of 896 experts, creating a much larger and more volatile working set that exceeds the M1 Max’s cache, forcing frequent disk reads that Mixtral’s smaller expert count avoids.
Can I run K3 on an M1 Pro 32 GB model?
The M1 Pro shares the same 400 GB/s bandwidth but caps at 32 GB unified memory. This halves the resident expert cache compared to the M1 Max, increasing swap frequency and likely dropping throughput below usable thresholds for interactive prototyping.
What quantization level preserves K3’s reasoning quality?
INT4 is the practical floor for local deployment, but INT8 retains significantly better reasoning fidelity on coding and math benchmarks. The INT8 footprint (~2.8 TB) still requires SSD offloading, but the bandwidth cost is offset by the quality gain for single-stream evaluation.
