The Groundy topic guide
Compare models, understand the benchmarks, and work out which tradeoffs matter for your workload.
Choose a way into the topic.
The latest in Models & Research.
MLX distributed inference on Macs is viable only when quantized weights exceed single-machine memory. Vendor-reported M5 Ultra results show interactive throughput, but lack of
Another way to explore the topic.
A preprint reports that verbalized LLM confidence weakly tracks logprob signals, with instruction-tuned models showing worse calibration. Gate on verifiable internal signals.
1–24 of 139 articles · Newest first
Preprint evidence shows domain fine-tuning causes 3.4-14.5% general skill loss. Mitigations like replay and routing reduce this, but require rigorous regression testing to be.
OpenAI Ultrafast costs 6x per token for GPT 6 Astra and Sol. Vendor docs lack latency data, so teams must measure speedups on latency-bound routes before enabling the premium.
Mistral Large 4's 1.05T parameters make single-node self-hosting unrealistic at full precision, requiring multi-terabyte memory and leaving the API as the only current option.
Beam is a 501B sparse MoE model with 23B active parameters. Vendor claims suggest 3-4x compute savings, but weights are unreleased and GLM 5.2 baselines drop under anti-hack.
A September 2026 preprint argues that LLM memory benchmarks require repeated judging, negative controls, and ablation arms to distinguish real gains from judge noise.
The headline architecture still rests on one preprint's mechanistic simulations, with no live-model validation or independent replication.
OpenAI reports GPT-6.1 Sol matches Astra on coding at one-fifth the cost, but independent verification is lacking. Route agentic work to Sol and keep Astra for hard research.
OpenAI's MentalHealthBench offers a rubric-based alternative to refusal audits, though vendor-reported scores and grader bias limit independent verification.
2-bit KV cache quantization offers major memory savings but risks silent recall degradation. Evidence supports 4-bit defaults for accuracy-critical workloads.
Three 2026 preprints show LLM agents often report success despite incomplete coverage or stale state, requiring artifact-level verification gates to catch silent failures.
A preprint reports LLM inference energy varies up to 179x by language, with Pashto costing far more than English. These author-reported findings suggest locale mix is a key, 1
Author-reported DECODE data shows most AI code edits occur within 15 minutes of acceptance, with 31% of trajectories ending in removal, suggesting review should focus on the 1
A rigor-matched audit finds layer skipping speeds LLM inference, but wall-clock rankings reverse when decision overhead is separated from pure generation cost.
A preprint reports that verbalized LLM confidence weakly tracks logprob signals, with instruction-tuned models showing worse calibration. Gate on verifiable internal signals.
SPD claims 28 ms listwise LLM reranking via single-pass decoding, but the evidence is one unreplicated preprint. Keep cross-encoders in production until independent benchmarks
An author-reported 44% on ARC-AGI-1 public eval for about $0.67 of rented compute shows what small budgets can justify, and what still needs replication.
FP8 and MXFP4 are umbrella specs, not single formats. A new preprint offers bit-exact conformance vectors to test quantized LLM portability across GPUs, exposing hidden format
Expert streaming lets 48GB Macs run 104GB MoE models by paging weights from SSD. This guide compares streaming to quantization and offload, analyzing workload fit and hardware
GPT-6 Astra's ARC-AGI-3 scores vary by 97 points based on effort settings. Route agents by cost per solved task, not launch-day leaderboards.
VeriFi preprint claims watermarks recover original faces after tampering, not just flag fakes. Robustness is untested against commercial tools and compression, so treat as a R
A new preprint shows AI audio leaks in decay tails via group delay. It offers a cheap, watermark-free filter for fraud screening, though hold-out accuracy is only 66.7%.
Rule checkers miss format variants while LLM judges get hacked during RL. This failure map from arXiv 2505.22203 shows why hybrid verifiers and reward audits are now essential
J-Zero claims zero-data self-play beats baselines by 8.0 points on unverifiable tasks, but judge verdict flips of 5.3 to 48.4 percent suggest the gains may be self-validated.
A new preprint argues agent benchmark scores track the evaluation harness as much as the model. Teams should pin harness versions and ablate scaffold changes to avoid misat.
Every new foundation model arrives wrapped in a benchmark chart and a press release. This coverage exists because the chart almost never tells you what the model actually does — and the press release tells you even less. The interesting work is one layer down: the attention variant that changes the cost curve, the parameterization fix that makes scaling laws actually transfer, the eval protocol whose failure modes flip the standings once you change a decoding parameter or a codec. I cover that layer.
I treat open-weight and closed releases as the same story told from opposite ends of a distribution shift. A trillion-parameter mixture beating a dense competitor on one harness and losing on another is not a contradiction; it is evidence about what the harness measures. Training-efficiency claims, context-window claims, and reasoning-benchmark claims all get the same treatment — read the method section, find the assumption that was load-bearing, and report whether removing it changes the result. When a paper proposes a new safety mechanism, a new compression trick, or a new continual-learning split, I am interested in the assumptions and limitations behind the headline result.
The throughline is comparative and skeptical without being contrarian. Foundation-model research has become the field where the largest gap between published numbers and deployed behavior tends to live. Closing that gap — with reproduction notes, ablation reading, and honest accounting of what generalizes versus what was tuned into the eval — is the beat.
