GLM-5.2 Benchmarks: What 62.1% SWE-bench Pro and 99.2% AIME Actually Mean

The two headline numbers deserve opposite amounts of trust. Zhipu’s model card reports 62.1 on SWE-bench Pro, run by Zhipu through OpenHands scaffolding with a prompt it tuned for the suite, and 99.2 on AIME 2026, where the frontier column sits at or near ceiling. The same table also carries the comparison the release summary frames as a closing gap: Claude Opus 4.8 at 69.2 on SWE-bench Pro, 7.1 points ahead. And on SWE-Marathon, one of the few rows an outside team ran, Opus 4.8 doubles GLM-5.2. This walkthrough weighs each score by who measured it and what it can actually tell you.

GLM-5.2 is Zhipu’s flagship for long-horizon work: a 744B-A40B open-weight model with a 1M-token context window and MIT-licensed weights (model card). The repository’s download table lists both BF16 and FP8 at that size, and the API documentation caps output at 128K tokens. One thing has changed since this suite was published: Zhipu now leads with GLM-5.3, which it says uses the same base model as GLM-5.2 with gains coming from post-training (GitHub). The numbers below still describe the base capability underneath it.

Who ran each evaluation

The footnote block on the model card sorts the suite into three evidentiary classes, and reading it that way is the fastest way to weight the table.

• Zhipu-run, vendor-configured harnesses: SWE-bench Pro through OpenHands with what the footnote calls “a tailored instruction prompt”; Terminal-Bench 2.1 under both the Terminus-2 framework and Claude Code; the math suite graded by GPT-5.5 (medium) as judge; MCP-Atlas judged by Gemini-3.0-Pro.
• Evaluations the card attributes to outside teams: FrontierSWE by Proximal (dominance score as of June 16, 2026), PostTrainBench by the PostTrainBench project, and SWE-Marathon by Abundant AI, each at 1M context with maximum reasoning effort.
• Competitor columns (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro, Qwen3.7-Max, MiniMax M3, DeepSeek-V4-Pro), measured or transcribed by Zhipu for its own comparison.

Nothing in the vendor’s materials is presented as an independent reproduction; the three externally run rows are the partial exception, and Zhipu still selected them and relays their settings. Treat the whole suite as vendor-reported, with differing degrees of harness control.

The table, dated

All figures below are Zhipu-reported, from the GLM-5.2 benchmark table, retrieved October 5, 2026.

* Full-set result per the card’s footnote; unmarked figures are the text-only subset, so starred and unmarked numbers are not directly comparable.

SWE-bench Pro and the tailored harness

SWE-bench Pro asks a model to resolve a real repository issue by producing a patch that passes that repository’s tests. GLM-5.2 scores 62.1 against 58.4 for GLM-5.1, both from Zhipu’s table (model card). The footnote deserves equal attention: the suite ran through OpenHands with a prompt “tailored” to it, at temperature 1, top_p 1, 32K maximum new tokens and a 400K context window. SWE-bench results move several points with scaffolding, retry policy and prompt wording, so a vendor-run figure is best read as an upper bound on what a stock harness would report.

Two adjacent facts keep the number honest. Passing tests is the grading criterion, so a patch that passes while breaking untested behavior still counts as resolved. And the same row puts Claude Opus 4.8 at 69.2 (card), with Qwen3.7-Max at 60.6, GPT-5.5 at 58.6 and Gemini 3.1 Pro at 54.2: second place in a strong field, clearly behind one closed model. The pattern repeats on the card’s other coding rows, where Opus 4.8 leads on each: NL2Repo 69.7 to 48.9, DeepSWE 58 to 46.2, ProgramBench 71.9 to 63.7 (card).

Terminal-Bench: the head-to-head flips with the harness

Two rows, one table. Under the Terminus-2 harness, GLM-5.2 scores 81.0, Opus 4.8 85.0 and GPT-5.5 84 (card). Under the row labeled “Best Reported Harness,” GLM-5.2 scores 82.7, GPT-5.5 83.4 and Opus 4.8 78.9 (card). That second row is a Claude Code run, and the footnote is unusually candid about the engineering involved: Claude Code 2.1.167, the 64K output-token CLI cap bypassed through a transparent proxy to restore a 128K setting, wall-clock limits removed while per-task CPU and memory caps stayed in place, results averaged over five runs. Disclosed, and also a reminder that a best-harness number is best-case by construction.

The baseline is softer than the release prose implies. The card’s table lists GLM-5.1 at 63.5 on Terminus-2; the GitHub summary says 62.0. A 1.5-point disagreement inside one vendor’s own materials is small in absolute terms and useful as a calibration point for how much precision self-reported comparisons carry.

AIME 2026 near the ceiling

GLM-5.2 reports 99.2 on AIME 2026 (card). The rest of the column: GLM-5.1 at 95.3, Opus 4.8 at 95.7, GPT-5.5 at 98.3, Gemini 3.1 Pro at 98.2, Qwen3.7-Max at 97 (card). The whole frontier sits at or near ceiling, which is what you would expect when a contest’s problems and solutions have circulated on the open web for months before a training cutoff. A 99.2 confirms the model is not weak on structured competition math; it cannot distinguish recall from generalization, and it says little about novel problem-solving.

The harder rows refine this. On HMMT November 2025, GLM-5.2’s 94.4 trails Opus 4.8 and GPT-5.5 at 96.5; on HMMT February 2026, its 92.5 trails both at 96.7 (card). IMOAnswerBench cuts the other way, 91.0 for GLM-5.2 against 83.5 for Opus 4.8. So the ceiling concern is specific to widely circulated AIME-style material rather than all competition math. Answers across these suites were extracted under a fixed response format and judged by GPT-5.5 (medium), per the footnote, which adds a layer of judge-dependent noise on top.

GPQA-Diamond and HLE, where an asterisk does real work

GPQA-Diamond: 91.2 for GLM-5.2 (card), against 93.6 for Opus 4.8, 93.6 for GPT-5.5, 94.3 for Gemini 3.1 Pro and 90 for Qwen3.7-Max. Competitive, mid-pack, not saturated.

HLE needs careful reading. The card’s footnote says text-only results are the default and starred results come from the full set. GLM-5.2’s 40.5 is unmarked, so text-only; Opus 4.8’s 49.8 and GPT-5.5’s 41.4 carry the star. Comparing 40.5 with 49.8 mixes subsets. The like-for-like statement this table supports is narrower: on text-only HLE, GLM-5.2’s 40.5 sits below Gemini 3.1 Pro’s 45 and well above GLM-5.1’s 31, while the two strongest closed competitors appear only in full-set terms. With tools, GLM-5.2 reports 54.7 against Opus 4.8’s full-set 57.9 (card). HLE remains the least gameable row in the suite, unsaturated and built to resist pattern-matching; it is also the row where a subset choice most changes the comparison.

The long-horizon rows, run by outside teams

Three rows carry the most evidentiary weight because Zhipu did not run them. On FrontierSWE (Dominance), conducted by Proximal as of June 16, 2026, GLM-5.2 scores 74.4 against 75.1 for Opus 4.8 and 72.6 for GPT-5.5 (card). PostTrainBench gives GLM-5.2 34.3 against Opus 4.8’s 37.2 (card). SWE-Marathon, run by Abundant AI, is the sobering one: 13.0 against 26.0 for Opus 4.8 (card). Zhipu’s documentation frames the same picture, placing GLM-5.2 between Opus 4.7 and 4.8 on long-horizon benchmarks overall and about 13 points behind 4.8 on SWE-Marathon.

These rows measure sustained, multi-hour agentic work at 1M context, which is what GLM-5.2 was built for, and they are where the gap to the closed frontier is widest. On agentic tool use the vendor’s own runs show a split: MCP-Atlas 76.8 to Opus 4.8’s 77.8 on the 500-task public set, but Tool-Decathlon 48.2 to 59.9 (card).

What the architecture buys

IndexShare reuses one indexer across every four sparse attention layers, cutting per-token FLOPs by 2.9× at 1M context (model card). That is the mechanism that makes a 1M window affordable on a model this size at all, and the long-horizon rows above were run at 1M context with maximum effort. The improved MTP layer adds up to 20% speculative-decoding acceptance length, per the same card, which speeds serving without changing what the model computes. One lineage note: the repository says the earlier GLM-5 integrated DeepSeek Sparse Attention by name; for GLM-5.2, Zhipu names only IndexShare and describes no relationship to DSA (GitHub).

Self-hosting, serving and access

Weights are on HuggingFace in BF16 and FP8, both MIT-licensed. As of the October 5, 2026 retrieval, the BF16 repo shows about 694K downloads in the prior month against about 422K for FP8, so the quantized variant is not the default pick among self-hosters here. The memory math binds either way: even at FP8, the 744B total listed in the download table is roughly 744 GB of weights before KV cache and runtime overhead, which means a multi-node cluster, and BF16 roughly doubles it.

Minimum serving versions, per the card: SGLang v0.5.13.post1+, vLLM v0.23.0+, Transformers v0.5.12+, KTransformers v0.5.12+, Unsloth v0.1.47-beta+, with vLLM-Ascend, xLLM and SGLang supported on Ascend NPUs. One operational wrinkle from the repository: GLM-5.2 accepts only high and max for reasoning_effort, unlike GLM-5.3, which adds a low setting. Prompts that assume a cheap mode have nowhere to land on 5.2.

The documented API is an OpenAI-style chat/completions endpoint on open.bigmodel.cn, with official Python and Java SDKs (documentation). Z.ai also sells managed access through its GLM Coding Plan, which Zhipu says received GLM-5.2 before general release; check the plan page for current tiers and pricing.

GLM-5.3 changes the buying question

Both model-card pages now carry a banner pointing to GLM-5.3, and the repository leads with it. Three verifiable facts matter. GLM-5.3 uses the same base model as GLM-5.2, with gains Zhipu attributes entirely to post-training. Zhipu claims a 50% improvement over GLM-5.2 on its in-house Z.ai Code Bench, plus open-source state of the art on Terminal Bench 3.0 and Agents’ Last Exam; those are vendor claims on newer benchmarks, on the same evidentiary footing as everything above (GitHub). And GLM-5.3-Flash is a separately trained 320B-A18B model with a hybrid sparse-plus-linear attention design aimed at cheaper long-context serving.

I would evaluate GLM-5.3 first for any new open-weight coding deployment today. The numbers here describe the base it was built on, and the caveats travel unchanged: vendor-run harnesses, near-ceiling math scores, and a measured gap to Opus 4.8 on exactly the long tasks GLM-5.2 was designed for.