Running a 145B LLM on a Laptop: How to Check Edge0's Claims

Edge0 says its newly open-sourced, 145-billion-parameter model runs on a 48GB laptop, peaks at 29GB of memory, decodes above 80 tokens per second, and posts coding scores that rival named frontier cloud models. Every one of those numbers is vendor-stated: as of 2026-11-13, they all come from Edge0’s own launch post and repository. What makes the launch worth your attention is that the claims are specific enough to test, and the vendor’s own repository both documents how to run the model and quietly complicates the laptop framing in two places: the Q3_K weights total 40.81 GiB, comfortably more than the claimed memory footprint, and every measured row in the repo’s speed table is a desktop machine, with the headline 121.2 tok/s agent decode recorded on a desktop RTX 5090 D 24 GB rather than a laptop. If you are deciding whether an on-device model can replace an API coding agent on hardware you actually own, the right move is not to believe or dismiss the launch post. It is to convert each claim into a check you can run in an afternoon, which is what the rest of this article does.

What Edge0 shipped, and which claims are vendor-stated

On 2026-11-12, Edge0’s launch post announced it is “open-sourcing Edge0-145B and Edge0-120B, together with our unified inference framework and deployment tools,” describing the larger model as “the world’s first open-source on-device AI model with 145 billion parameters.” Treat the “world’s first” phrasing as marketing; the checkable substance is elsewhere in the post.

The performance claims, all vendor-stated from internal testing, are unusually concrete:

• 29GB peak memory, “enabling local use on a 48GB laptop,” which the launch post describes as about 40% of an estimated 70GB footprint for standard 4-bit deployment. The post elsewhere says the laptop comparison “reports 80+ tokens/s and 28GB peak memory,” so even the headline number wobbles by a gigabyte depending on which paragraph you read.
• 1,100+ tokens/s prefill and 80+ tokens/s decode, per the launch post, on a “compatible personal laptop, such as a MacBook Pro.”
• Roughly 120W typical power draw and a 128K, 256K context window.
• Benchmark comparisons against named cloud models (launch post): 64.6 on DeepSWE 1.1 (versus 58.7 for Qwen3.8-Flash-Next and 54.4 for DeepSeek-V4-Flash-0731), 90.4 on GPQA-Diamond (versus 90.8 and 91.3 for DeepSeek-V4-Flash-0731 and Claude-Opus-4.6 (Max); comparison scores from the post), 80.6 on IFBench, 86.52 on Terminal-Bench 2.1, and 79.3 on AA-LCR.

None of these scores has been independently replicated. They are a vendor running its own model through public harnesses, which is normal at launch and also exactly why the spot-check procedure below matters.

The hardware bill of requirements

Before the performance claims, there is a more basic question: can your machine run this at all? The vendor’s GitHub repository answers it narrowly:

“145B / 120B are the current latent-MoE series, derived from Qwen3.8-Flash-Next and trained natively at 4 bits. They run on edge0-strata through edge0-infer serve and edge0-infer chat, on Linux or Windows with an NVIDIA GPU and CUDA 13.”

That sentence does a lot of work. The launch post names a MacBook Pro as an example compatible laptop, and the repo tagline advertises Mac, iPhone, and Android, but the documented path for the 145B series is Linux or Windows with an NVIDIA GPU and CUDA 13. The launch post sharpens the contradiction rather than resolving it: it claims “For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips,” while the repo’s build instructions target NVIDIA GPUs with CUDA 13 on Linux or Windows and its one Apple row, an M2 Ultra speed measurement, carries the caveat that it “is a Metal measurement, not an announcement of macOS release availability.” If you are a MacBook owner drawn in by the launch framing, the honest status is “unverified on your machine”: ask the vendor or wait for a documented macOS path before budgeting setup time.

Second, storage. The repo lists Q3_K weights for the 145B model at 40.81 GiB, which exceeds the blog’s 29GB memory claim. That gap is not necessarily a contradiction; it tells you the weights are not fully resident in memory and are streamed from storage during inference. The practical consequence: budget roughly 41 GiB of fast SSD space, and treat storage speed as part of the performance story, not an afterthought. The launch post’s own description of the loading pipeline supports this reading, describing frequently used “hot experts” held in GPU memory, an expert library of approximately 40GB in system memory, and approximately 48GB more read from SSD on demand. Note that this description sits awkwardly next to the 29GB peak-memory headline; the post never defines what “peak memory” counts, which is one of the questions worth putting to the vendor.

The repo does document one constrained-memory run: the same RTX 5090 D held to a 16 GiB VRAM and 20 GiB system-RAM budget via cgroup, with residency enabled and the 40.81 GiB of Q3_K weights in play. The figures are worth citing in full: with residency on, that card “holds Q3_K (40.81 GiB of weights) as 23.14 GiB VRAM + 31.28 GiB RAM,” a combined 54.42 GiB across the two pools and again above the 29GB headline. Under that budget the model still decodes at 36.8 tok/s chat and 44.9 tok/s agent, and the repo notes the budget “is a test limit, not a minimum system requirement.”

Decode speed is memory-bandwidth arithmetic

The single most useful tool for vetting any on-device throughput claim is a lower bound you can compute on an envelope. The reason comes from the decode literature, and both findings here are preprint-reported and unreplicated in this context, though they describe well-established mechanics. As one recent cross-GPU study (arXiv:2605.30571) puts it:

“The standard account of batch-1 decode is that it is HBM-bandwidth-bound: each decode step streams the model weights and the per-layer KV cache through global memory once, arithmetic intensity is too low to keep the SMs busy”

A second preprint on prefill and decode bottlenecks (arXiv:2512.22066) states the same constraint from the other side: “even though only one token is processed, the full weight matrices must be streamed from memory… memory bandwidth dominates.”

The calculation for Edge0-145B, using the vendor’s own architecture description: the post says only approximately 10B of the model’s 145B parameters participate in computation on each inference step. At 4 bits per parameter, that is roughly 5GB of weight data read per generated token, as a floor. Multiply by the 80 tok/s decode rate in the launch post and you get about 400 GB/s of sustained memory bandwidth as the minimum the laptop must deliver just for weights, before the KV cache, the shared expert, scheduling overhead, and imperfect overlap push real traffic higher. That is my arithmetic from vendor numbers, not a measurement, and it is a lower bound by construction.

Two things follow. First, the laptop claim is at least arithmetically plausible only for machines with several hundred GB/s of real, sustained memory bandwidth; many laptops are nowhere near that, so “48GB laptop” alone tells you nothing without a bandwidth figure. Second, do not assume your machine’s peak bandwidth converts linearly into tokens per second. The cross-GPU preprint reports that the achieved fraction of peak HBM bandwidth falls as peak bandwidth rises, and the vendor’s own desktop number is consistent with that: 121.2 tok/s implies roughly 600 GB/s of effective weight streaming on a card whose peak is far higher. Expect your achieved rate to land well under the naive division, and measure rather than extrapolate.

Architecture and quantization: what to look for on the model card

The repo documents the architecture in enough detail to ask sharp questions. The 145B model uses latent width 1792, 48 layers, and routes each token to 10 of 512 experts alongside one shared expert; per the launch post, each group of 64 experts shares two large matrices, which is how the parameter count grows without proportional memory growth. The models are trained natively at 4 bits rather than quantized after the fact, and ship in Q3_K and NVFP4 formats.

Native low-bit training is a real answer to the usual quality objection to aggressive quantization, but it does not close the file. Research on MoE quantization, again preprint-reported, QuantMoE-Bench (arXiv:2406.08155), flags a structural asymmetry worth checking here:

“MoE’s sparse routing mechanism leads to different activation patterns, where shared experts are accessed by all tokens while token-conditioned experts are selectively activated. This activation disparity suggests different quantization requirements”

Edge0-145B has exactly this structure: one shared expert touched by every token, plus routed experts touched rarely. So the model-card questions write themselves. Was the shared expert quantized to the same bitwidth as the routed experts, or protected at higher precision? Do the published Q3_K and NVFP4 builds show different scores on the coding benchmarks, and were the headline numbers produced with the same format you would actually download? A model that posts 64.6 on DeepSWE in one format and something lower in the format that fits your disk is a different product than the launch post describes.

Spot-checking the vendor benchmark yourself

The reproduction path the vendor documents is short enough to run in an afternoon:

1. Confirm the platform. Linux or Windows, NVIDIA GPU, CUDA 13. If that is not your machine, stop here and treat the launch as not-yet-applicable to you.
2. Verify the footprint. Download the Q3_K weights and confirm the 40.81 GiB on disk. During a run, watch actual resident memory and compare it against the 29GB claim, keeping in mind the vendor never defined “peak memory.”
3. Measure decode on your hardware. Serve the model through edge0-infer and time token generation on representative prompts. Compare your number against the vendor reference points: the repo’s headline 121.2 tok/s on the RTX 5090 D desktop and the 80+ tok/s laptop claim. If your machine lands far below both, the bandwidth arithmetic above tells you why.
4. Rerun one public benchmark. The vendor cites Terminal-Bench 2.1, DeepSWE 1.1, GPQA-Diamond, and IFBench. Pick the one closest to your workload and run it locally rather than trusting the launch table.
5. Run your own tasks last. A half-dozen real prompts from your codebase, with pass/fail judged by you, is worth more than a reproduced benchmark for the actual decision, because the decision is whether this replaces your API agent, not whether it tops a chart.

Local versus API: the decision table

The cost baseline for the API side comes with a provenance caveat: it is Anthropic’s own figure, cited secondhand through the super-token-meter repository, which reports that “Claude Code averages ~$13/developer/active‑day and stays under $30 for 90% of users.” Treat it as a vendor-reported average, useful for order-of-magnitude budgeting, not a quote of your future bill.

Two cells deserve emphasis because they are easy to read past. “Offline scope” rests on a single boundary sentence; if your workflow depends on tool use, retrieval, or anything that phones home, verify which pieces keep running with the network cable pulled. And “capability ceiling” is where the evidence is thinnest: the coding comparisons come from the vendor, the launch post itself says the Edge0, Qwen, DeepSeek, and Claude results “come from Edge0’s internal evaluations,” and nothing independent confirms them.

What the evidence cannot tell you yet

The gaps are worth listing plainly, because they define what to ask Edge0 before or alongside your own testing. “Compatible laptop” is undefined, and none of the measured rows in the repo’s speed table is a laptop; they cover desktop GPUs plus one Apple M2 Ultra measurement that the repo says is not a macOS release announcement. The MacBook Pro mention in the launch post has no documented macOS path in the repo for the 145B series, even though the post claims day-one macOS support. The 29GB versus 40.81 GiB tension is explicable by SSD streaming, and the post’s own loading description supports that reading, but “peak memory” is never defined precisely enough to audit. And which capabilities keep working offline is unspecified beyond that one boundary sentence, which matters most to the air-gapped readers this product is aimed at.

The verdict

Treat Edge0-145B as promising but unverified. The launch claims are specific, internally checkable, and at least arithmetically coherent with a bandwidth-bound MoE design, which puts them well ahead of most on-device announcements. But every performance number is vendor-stated, the repo’s own weight sizes and desktop-only benchmark rows complicate the laptop story, and no independent replication exists yet. So: confirm the Linux/Windows + NVIDIA + CUDA 13 requirement fits your machine, budget roughly 41 GiB of fast storage alongside whatever memory the model actually uses, measure decode throughput yourself against the 121.2 tok/s desktop reference, and rerun one public coding benchmark plus a handful of your own tasks. Keep paying the API bill, at the reported ~$13 per developer per active day (super-token-meter), until your own eval clears your bar. If the claims do hold up under that testing, the constraint on offline coding agents stops being API pricing and becomes memory bandwidth and eval trust, which is a trade worth making on your own terms rather than the launch post’s.

Frequently Asked Questions

What hardware is required to run Edge0-145B according to the repository?

They run on edge0-strata through edge0-infer serve and edge0-infer chat, on Linux or Windows with an NVIDIA GPU and CUDA 13.

How much storage space is needed for the Edge0-145B model weights?

The repo lists Q3_K weights for the 145B model at 40.81 GiB, which exceeds the blog’s 29GB memory claim. That gap is not necessarily a contradiction; it tells you the weights are not fully resident in memory and are streamed from storage during inference. The practical consequence: budget roughly 41 GiB of fast SSD space, and treat storage speed as part of the performance story, not an afterthought.

What is the estimated cost of using an API coding agent like Claude Code?

Claude Code averages ~$13/developer/active‑day and stays under $30 for 90% of users.