Running GLM-5.2 at Home: SGLang, vLLM, Transformers, and KTransformers Setup Guide

GLM-5.2 weights landed on HuggingFace on June 13, 2026,¹ and as of June 19, 2026, the FP8 variant has roughly 93,900 downloads against around 11,900 for the BF16 original.³ That ratio tells you something: the community already voted for the quantized path. This article covers what you need to run either variant on your own hardware, how SGLang, vLLM, Transformers, and KTransformers differ for this workload, and how to think about hardware cost against the subscription alternative. An August re-verification of all four runtime paths added a troubleshooting section for the precision-level failure modes people actually hit: MTP dropped from int4 builds, w4afp8 flag confusion on SGLang, and the model.layers.63 indexer load warning.

What is GLM-5.2 and what makes self-hosting it unusual?

GLM-5.2 is a 753B-parameter² mixture-of-experts (MoE) model released by Zhipu, a 2019 Tsinghua University KEG lab spin-off now publicly listed in Hong Kong as 02513.HK.⁶ The model ships under an MIT license² (not Apache 2.0, not a custom community license), which is the cleanest permissive term you can get on a frontier-scale weight set.

Two architectural details shape the inference setup:²

• IndexShare sparse attention reuses the same indexer across every four sparse attention layers, reducing per-token FLOPs by 2.9x at 1M context length compared to dense attention at the same context size.
• MTP (speculative decoding) layer adds a draft head that predicts multiple tokens ahead, reducing wall-clock latency per output token on capable hardware. Zhipu reports the revised MTP layer raises acceptance length by up to 20%.²

On parameters: Zhipu’s README states GLM-5 as 744B total with 40B active in plain text.¹ GLM-5.2’s model card confirms 753B total² but publishes no active-parameter figure of its own. Treat 40B as inherited architecture from GLM-5, not a confirmed 5.2 number, and use 753B when citing 5.2’s size. The 744B figure circulating for 5.2 is GLM-5’s count.

Context window is 1,000,000 input tokens with a 128K (131,072) output ceiling.⁴ That is a 5x jump over GLM-5.1’s 200K window.¹ At that scale, MoE memory handling matters more than it does for dense models of the same active-parameter count, because all 753B parameters must reside in memory even though only a fraction fire per token.

What are the verified benchmark scores?

Zhipu posted the following numbers in the GitHub README:¹

For context: Claude Opus 4.8 scores 85.0 on Terminal-Bench 2.1,¹ leading GLM-5.2 by 4 points on that benchmark. One disagreement inside Zhipu’s own materials is worth knowing: the GitHub README reports GLM-5.1 at 62.0 on Terminal-Bench 2.1,¹ while the HuggingFace model card reports 63.5 on the Terminus-2 harness and 69.0 on “Best Reported Harness”.² The GLM-5.2 side shifts too: 81.0 on Terminus-2, 82.7 on the best-reported harness.² Name the harness when you quote either number, or you are comparing different evaluations. These are vendor-reported figures; the README and model cards cite no independent replications.

BF16 or FP8: which weight variant to download?

Both variants are publicly downloadable from the zai-org HuggingFace organization (not THUDM, which now has zero public models).²³

BF16 (zai-org/GLM-5.2) is the reference weight format. It preserves full precision and is the right choice if you have the memory and want the highest-fidelity baseline.²

FP8 (zai-org/GLM-5.2-FP8) cuts memory footprint by roughly half compared to BF16 on a per-parameter basis, which is why it leads on downloads by approximately 8:1.³ The tradeoff is that FP8 quantization can affect accuracy on tasks at the edge of the model’s capability, and the magnitude of that effect on GLM-5.2 specifically has not been published by Zhipu.

For most teams doing an initial self-hosting evaluation, FP8 is the practical starting point: lower GPU memory, faster to iterate on, and the download community has clearly converged on it. Regression testing on your specific task distribution against BF16 is the right follow-up, not a prerequisite.

How do the four deployment frameworks compare?

The model card now pins minimum versions per framework: SGLang v0.5.13.post1+, vLLM v0.23.0+, Transformers v0.5.12+, and KTransformers v0.5.12+.² It also lists a fifth path, Unsloth (v0.1.47-beta+), and Ascend NPU support via vLLM-Ascend, xLLM, and SGLang.² When this guide was published in June 2026, no minimum versions were documented anywhere public. Here is how the four main options differ for a 753B MoE:

SGLang is a structured-generation serving layer designed around batched, concurrent requests. It has native support for constrained decoding (JSON schemas, regex) and implements RadixAttention for KV-cache sharing across requests. For workloads with many concurrent clients or structured output requirements (agent loops calling the model as a service), SGLang is the natural starting point. It is also the framework most likely to expose the MTP speculative decoding gain under concurrent load.

vLLM is the more established open-source serving engine, with wider hardware support and a larger operator community. Its paged-attention memory management handles variable-length sequences efficiently and is well-tested on large MoE models from other families. If your team already runs vLLM for other models, adding GLM-5.2 behind the same stack is the lowest operational delta. Speculative decoding support is present but the configuration path differs from SGLang.

Transformers (HuggingFace’s library) is the reference path for single-machine, lower-concurrency evaluation. It is slower than either serving framework under load but requires the least custom setup and is the canonical way to verify that weight loading works before investing in a serving configuration. Use it to validate the download and run a few thousand-token samples; do not use it as your production serving layer for a 753B model.

KTransformers is a kernel-optimized inference library targeting high-throughput on consumer and prosumer hardware. It is the least established of the four by community size, but it is the framework most likely to make sense if you are running on a non-datacenter GPU cluster where raw throughput per dollar matters more than operational familiarity.

For a first deployment, the decision tree is: if you need serving concurrency and structured output, start with SGLang; if you already run vLLM and want operational consistency, add GLM-5.2 there; if you are evaluating on a single node with no concurrency requirement, use Transformers to validate before migrating to a serving layer. Whichever branch you take, honor the version pins. The failure modes in the next section cluster around builds below the pin and checkpoints outside the official two.

What breaks in practice, by precision path?

The model card pins versions and links per-framework recipes and cookbooks.² That covers the official BF16 and FP8 checkpoints. The questions now arriving at this guide come from outside that envelope, and they sort cleanly by precision path:

MTP disappears on int4 builds. The MTP draft head is extra weights on top of the transformer proper, and quantization pipelines that repack the routed experts do not always carry it. vLLM’s contract here is unforgiving: no draft weights in the checkpoint, no speculative decoding, and the failure surfaces as an ignored or erroring speculative config rather than a labeled message. Reports on GLM-5.2 int4 checkpoints match that pattern.[unverified] Before debugging vLLM, list the quant’s weight index and check whether the draft-head tensors exist. If they do not, the fix is a different checkpoint, not a different flag. The supported MTP path is the official FP8 or BF16 release on vLLM v0.23.0+, per the recipes the model card links.²

w4afp8 flag confusion on SGLang. The name is community-quantization convention: w4 for 4-bit routed experts, afp8 for FP8 everywhere else. It is a mixed-precision checkpoint, and SGLang has to be told exactly which scheme it is loading. The reported GLM-5.2 failure mode is a load error or garbage output from a wrong or missing quantization flag,[unverified] which is easy to misread as a broken download. Two things follow. The version pin (SGLang v0.5.13.post1+) covers the official checkpoints, so a third-party mixed-precision build sits outside the supported envelope and its own documentation has to supply the flag. And when a w4afp8 load fails, suspect the flag first and the download second.

The model.layers.63 indexer.k_norm warning. Loading GLM-5.2 in Transformers can emit a “weights were not initialized” warning naming model.layers.63’s indexer.k_norm. The reflex is to assume a corrupt download and re-pull three-quarters of a terabyte. The architecture offers two better explanations with very different consequences. IndexShare shares one indexer across every four sparse-attention layers,² so a loader that expects per-layer indexer tensors will report the unmaterialized copies as uninitialized while the forward pass reads the shared module: a warning with zero capability loss.[unverified] Or the loader instantiated a fresh, random indexer for the layers without tensors, in which case sparse top-k selection runs on random projections: no crash, no short-prompt symptom, just quietly degraded long-context retrieval, which is exactly where GLM-5.2’s value sits. The warning text does not distinguish between them. Load the same checkpoint under Transformers v0.5.12 or newer,² run a long-context retrieval probe, and compare against the build that warned; the checkpoint’s state-dict key list settles which layers own indexer weights and which share. A missing-weight warning on a sparse-attention indexer is not automatically fatal, and it is not automatically cosmetic.

What hardware do you actually need?

GLM-5.2 is a 753B-parameter MoE model. Every parameter must fit in addressable GPU memory, not just the active subset, because the full expert set is needed to route correctly. FP8 packs each weight into one byte; BF16 uses two bytes.

Rough memory floor estimates (weights only, before activations and KV cache):

These are lower bounds. A 1M-token context window means KV cache at full context can add hundreds of additional gigabytes depending on batch size and implementation. Plan for headroom beyond the weight floor.

IndexShare’s 2.9x FLOP reduction at 1M context² does not change the memory requirement for weights; it reduces compute cost per token at long context, which matters for throughput rather than minimum hardware.

How does self-hosting cost compare to the subscription?

Z.ai prices hosted GLM-5.2 access through flat subscription tiers:⁵

The August re-fetch of the plan page shows movement since June: Pro’s yearly-effective rate rose from roughly $50.4 to $56 per month, Max from roughly $112 to $117.6, and Max’s usage multiple dropped from 20x Lite to 14x.⁵ Lite’s quota is now denominated in credits, 10,000 per week, with a 5-hour rolling usage window layered on top of the weekly cap.⁵ The page also now leads with GLM-5.3 in its marketing alongside GLM-5.2 and GLM-5-Turbo;⁵ this refresh’s source set contains no published GLM-5.3 weights, so the hosted plan currently fronts a model the self-hosting path cannot match.

Under the MIT license,² self-hosting incurs no per-token fee and no subscription. The cost is hardware acquisition or rental plus electricity and operational overhead.

The crossover point depends on utilization. At low usage, the Lite tier at $12.6/month on yearly billing⁵ is almost certainly cheaper than renting the GPU cluster you need for a 753B model. At high, sustained utilization, where you are running the model continuously for production traffic, the hardware cost amortizes against the subscription fast.

A rough framing: ten H100s running 24/7 is about 7,300 GPU-hours a month. At any hourly rate a cloud provider will actually quote for H100s, full-duty compute alone lands in the tens of thousands of dollars monthly, before storage, networking, and the engineering time to keep a 10- or 20-way tensor-parallel stack healthy. That math only works if the model is serving enough traffic that a flat per-query subscription would cost more.

For most teams evaluating GLM-5.2 for the first time, the subscription path is cheaper until you have confirmed the model performs on your workload and have a throughput level that makes the economics of self-hosting close. The MIT license means that calculation stays open: you can switch to self-hosting later without renegotiating a license.

How does the Anthropic-compatible endpoint affect migration?

GLM-5.2 exposes an Anthropic Messages API-compatible endpoint.⁴ For teams running Claude Code, Cline, OpenCode, Roo Code, Goose, Crush, OpenClaw, or Kilo Code,⁵ the migration path is a base-URL and model-name swap rather than a full SDK integration.

This matters for self-hosted deployments too: frameworks like vLLM and SGLang can expose an OpenAI-compatible REST interface, but if your agent tooling is wired to the Anthropic Messages API shape, you will need a thin translation layer or a framework that natively supports the Anthropic API format. Verify this at the framework level before committing to a serving stack, because it affects both the agent integration path and whether you can reuse existing prompt code that relies on Anthropic-specific message fields.

Thinking effort presets and when to use them

GLM-5.2’s API exposes High and Max thinking-effort presets for long multi-step coding tasks.⁴ These presets direct the model to extend its chain-of-thought before producing an output, at the cost of higher latency and token consumption.

For self-hosted deployments, thinking presets interact directly with your hardware budget. A Max-effort request on a 1M-context coding task can produce a substantially longer output than a default request, which fills KV cache faster and increases per-request memory pressure. If you are running near the memory ceiling, High is the safer default until you have profiled your actual request distribution.

Frequently Asked Questions

Does GLM-5.2 require special framework versions to run?

Yes, and the model card now states them: SGLang v0.5.13.post1+, vLLM v0.23.0+, Transformers v0.5.12+, KTransformers v0.5.12+, and Unsloth v0.1.47-beta+.² When this guide was published in June 2026 the pins were undocumented; the July re-check of the model cards shows them published. Treat the pins as hard floors, particularly for the indexer and MTP code paths: older Transformers builds predate GLM-5.2’s module layout, and loading a new architecture on an old loader is the standard route to uninitialized-weight warnings.

Is the model.layers.63 indexer.k_norm “weights were not initialized” warning fatal?

Not necessarily, and treating it as a corrupt checkpoint is premature. GLM-5.2 shares one indexer across every four sparse-attention layers,² and a loader expecting per-layer weights can report the shared copies as uninitialized while the model reads the shared module, which costs nothing. The damaging variant is a loader that instantiates a random indexer instead, which quietly degrades long-context retrieval without crashing. Load under Transformers v0.5.12+ and compare long-context output against the build that warned; that test settles it, not the warning text.[unverified]

Can I run the FP8 weights on consumer GPUs?

The FP8 variant still requires roughly 753 GB of GPU memory for weights alone, which puts it beyond any consumer GPU. The FP8 advantage is that it halves the memory requirement relative to BF16, making the model approachable on a larger but still datacenter-class multi-GPU rig rather than an extreme enterprise configuration.

Is the MIT license on the weights the same as the code license?

No. The GitHub repository at zai-org/GLM-5 is licensed under Apache 2.0 for the code.¹ The model weight files hosted on HuggingFace carry a separate MIT license.² The permissive self-hosting right applies to the weights, not to the inference code in the repo.

What is the model identifier to use in API calls?

The model identifier is glm-5.2, with glm-5.2[1m] used in some documentation to explicitly reference the 1M-context variant.⁴ Confirm the exact identifier string against Zhipu’s current API documentation before wiring it into a production agent loop, as model identifier strings can change between documentation revisions.

Can I use GLM-5.2 in Claude Code directly?

Yes. The plan page now advertises support for more than 20 agent tools, including Claude Code and ZCode.⁵ The Anthropic Messages API-compatible endpoint means a base-URL change in your Claude Code configuration is sufficient; no SDK swap is required.⁴