Can AI Write CAD Programs? CADBench Measures the Gap
CADBench evaluates eleven systems on 18,000 CAD samples and finds vision-language models unreliable, a gap SWE-bench and LiveCodeBench were never built to measure.
The archive · Page 6 of 7
Models & Research
Where architecture, training tricks, and eval methodology meet the marketing layer — separating durable progress in foundation models from leaderboard theater that quietly falls apart under load.
The archive
121–144 of 159 articles · Newest first
ByteDance's Doubao 2.1 Pro vs GPT-5.5: Reading Self-Reported Benchmarks
ByteDance's Doubao-Seed-2.1 Pro launch deck claims parity with GPT-5.5 on benchmarks ByteDance graded itself, and no independent source has reproduced the scores yet.
Can RoboSSM's State-Space Backbone Replace Transformer Imitation Policies?
The RoboSSM preprint swaps the transformer backbone of in-context robot imitation for a Longhorn state-space model and claims LIBERO gains. The full paper is still pending.
Pruning Experts to Shrink MoE Models: Does Attribution-Guided Compression Beat Magnitude?
Routing architecture decides whether MoE experts can be pruned safely: hard routers keep calibration per expert, soft routers need an aggregate guard.
GLM-5.2 vs Kimi K2.7 Code: Two Open-Weight Bets on Agentic Coding
GLM-5.2 and Kimi K2.7 Code shipped a day apart as open-weight coding models. One leads the open-weights index; the other wins on price and a claimed token-efficiency gain.
How Linear Is a Transformer Feed-Forward Block? A New Test Says It's Learned, Not Built In
Per-block linear recoverability in transformer FFNs swings from near-linear to strongly nonlinear between layers, so compression tools must probe each block per checkpoint.
GLM-5.2 Benchmarks: What 62.1% SWE-bench Pro and 99.2% AIME Actually Mean
Every GLM-5.2 benchmark figure, from 62.1% SWE-bench Pro to 99.2% AIME, traces to Zhipu's own evaluations. This guide separates vendor-run scores from external evals.
GLM-5.2 on Terminal-Bench 2.1: Strengths, Gaps, and How to Route Real Coding Tasks
GLM-5.2 scores 81.0 on Terminal-Bench 2.1, trails Claude Opus 4.8 (85.0) on shell tasks, but wins on 1M-token monorepo context. Here is how to route tasks.
GLM-5.2 vs Claude Opus 4.8: Open-Weight Coding at Frontier Pricing
GLM-5.2 posts 62.1% on SWE-bench Pro and 81.0 on Terminal-Bench 2.1, four points behind Opus 4.8, with a 1M-token context and MIT-licensed self-hostable weights.
GLM-5.2's 744B MoE Costs More to Self-Host Than the MIT License Suggests
GLM-5.2 ships 744B parameters under MIT with strong coding benchmarks, but the full MoE weight load makes self-hosting far heavier than the license terms imply.
STAR Replaces Scalar Reward in Text-to-Image RL with Attention-Derived Spatial Maps
STAR uses attention-derived spatial maps to replace uniform scalar reward in diffusion RL, shifting the bottleneck from preference pair volume to reward localization.
Can Editing One Neuron Fix LLM Repetition Loops?
A June 2026 preprint localizes Gemma 4 repetition loops to a few MLP neurons and removes them with a one-time weight edit while benchmark scores hold.
Claude Fable 5 Benchmarks: What FrontierCode, CursorBench, and ViBench Show
Claude Fable 5 claims top benchmark scores. Verified data shows every model below 14% on FrontierCode Diamond, and partner scores lack public methodology.
Does Attribution Patching Lie? A Fix for a Common Interpretability Shortcut
A June 2026 paper traces attribution patching's errors to downstream non-linearities and proposes a Hessian-vector-product correction that costs one extra backward pass.
Why Pruning a Model Can Raise Its Out-of-Distribution Accuracy
Task-aware layer pruning removes distortion-amplifying layers and improves out-of-distribution accuracy, which means standard in-distribution benchmarks miss the real effect.
Do Unified Multimodal Models Actually Interleave Understanding and Generation?
IMUG-Bench tests whether unified multimodal models can alternate between understanding and generation in one context, exposing gaps that separate benchmarks conceal.
Claude Mythos 5 Access Rules: Who Gets Project Glasswing and Why
Claude Mythos 5 is Fable 5 with cyber safeguards lifted for Project Glasswing partners; the June 9, 2026 release lists biology access as forthcoming.
How LLMs Track Who Did What: The Entity Rebinding Circuit
New research isolates a compact attention-head circuit for entity rebinding in Gemma and Llama, showing tracking failures stem from a binding step, not context length.
Claude Fable 5 vs Opus 4.8: When 2x Pricing Is Worth It
Its biology restrictions matter as much as its benchmark placements: the drug-design results attributed to Mythos 5 do not establish a benefit a general Fable buyer will receive.
Fable 5 Distillation Protection: How Anthropic Blocks Model Copying
Claude Fable 5 ships with distillation protection to prevent capability extraction. A first-principles look at what it is, how it works, and why API consumers should care.
Can LLMs Write Better Research Paper Titles Than Authors?
A new study claims LLMs write 'appropriate' research titles, but the evidence rests on similarity metrics that measure pattern matching, not whether titles actually serve.
Do Concept Bottleneck Model Benchmarks Measure Interpretability or Dataset Bias?
Standard concept bottleneck model benchmarks confound genuine concept learning with dataset shortcuts. Synthetic benchmarks from Skirzynski et al. expose the gap.
Persona Prompts Change Who an LLM Recommends as an Expert
A 43-model audit finds that geographic and role framing in LLM prompts systematically shifts which scholars get recommended as experts, with no neutral default.
Opus 4.8 Batch API: 1M Context, 300k Output, and Team Cost Controls
Opus 4.8 has a 1M token context window (200k on Foundry), 128k standard output, and 300k output via Batch API beta. January 2026 cutoff. Batch design and quota allocation.
