Running FLUX 3 Image Klein Locally: VRAM Tiers and License Limits

If you own an NVIDIA RTX card with 8 GB of VRAM or more, Black Forest Labs’ new FLUX 3 Image Klein will almost certainly fit on it. That is the easy part of the decision. The harder questions are whether the FLUX Kommunity License actually permits what you plan to do with the model, and which of four precision builds matches your GPU generation, because on some cards the default choice is not the one the naming suggests. Both questions have concrete answers, and the second one has independent evidence behind it that cuts against the vendor’s own build lineup.

On October 15, Black Forest Labs released FLUX 3 Image Klein, a 5.6B-parameter open-weights model covering text-to-image generation, image editing, and multi-reference generation with up to 10 reference images. It shares the full FLUX 3 Image model’s native 4K generation, pixel-level editing, and bounding-box control, in a package roughly 38% smaller than the 9B FLUX.2 [klein] it succeeds, according to the same post. Day-one support covers ComfyUI, Hugging Face Diffusers, TensorRT-LLM, SGLang Diffusion, and vLLM-Omni, so the model slots into the local workflows most small teams already run.

Everything specific in BFL’s announcement is vendor-reported, and the rankings in particular are internal to the vendor: the release post says “In internal evaluations, FLUX 3 Image Klein ranks first in text-to-image, multi-reference generation and image editing, ahead of larger open weights models.” No independent reproduction is cited in this article. That framing matters throughout this article, so here it is once: the VRAM figures, the latency numbers, and every quality ranking below come from BFL or from NVIDIA measurements published by BFL.

Which build fits which VRAM tier

BFL ships a base checkpoint for fine-tuning, a step-distilled checkpoint for low-latency generation, and three quantized builds (FP8, INT8, NVFP4) developed with NVIDIA support. The vendor-reported generator-weight sizes are the anchor for planning:

Per BFL’s release post: “The generator weights require 11.2 GB in BF16 and 5.6 GB in FP8. With the INT8 build, FLUX 3 Image Klein runs on NVIDIA GeForce RTX GPUs with as little as 8GB of VRAM, through ComfyUI or Diffusers, extending what FLUX.2 [klein] started to even smaller hardware.” One table note: BFL’s post names no hardware target for the FP8 build, so that row’s best-fit reading is this article’s inference, argued below.

One number deserves caution before you plan around it. These are generator-weight sizes, not total runtime VRAM. BFL’s post does not state how much additional memory the text encoder, activations, multi-reference conditioning (up to 10 images), or 4K output resolution consume. An 8 GB RTX card that loads the INT8 build comfortably at 1MP with no reference images may run out of headroom on a five-reference edit at high resolution. Treat the 8 GB figure as a floor for basic generation, verified by your own workload before you commit a pipeline to it.

Latency: distilled versus base

BFL’s timing charts, measured by NVIDIA on vendor configurations, put the distilled model at 16 sampling steps against 50 for the base. With the distilled INT8 build, the vendor reports “it generates a 1MP image in about 4 seconds on an NVIDIA GeForce RTX 5090 and under 9 seconds on an RTX 4070 through ComfyUI.” On NVIDIA’s GB300 datacenter hardware, BFL reports that the distilled NVFP4 build produces a 1MP image in under 0.6 seconds, with a served LoRA adding around 5% to generation time (release post).

Two caveats apply. First, these are vendor-side measurements on vendor-chosen configurations; real ComfyUI graphs with custom nodes will differ. Second, BFL’s API latency comparisons against rival services were timed asymmetrically (FLUX on BFL’s own servers, competitors measured from the client including polling and network time), so the local numbers are more trustworthy than the cross-service ones. For interactive editing loops, the distilled checkpoint is the practical default; the base checkpoint is for fine-tuning and for cases where you want the full 50-step quality budget.

Does your GPU generation even have FP8?

This is where independent evidence changes the obvious reading of the build list. FP8 sounds like the natural middle option: half the BF16 footprint, modern format. But whether FP8 buys you anything depends on your card’s tensor cores, and the answer is not flattering for a large installed base.

An independent post-training quantization study of Ideogram 4.0, a comparable 9.3B flow-matching diffusion transformer run on an RTX 3090 cluster, states the hardware constraint directly (arXiv:2606.12280): “Ampere has no FP8 tensor cores, so the FP8 checkpoint executes by dequantizing each linear to bf16 every forward; in practice this overhead is modest (FP8 runs at 172.9 s/image, ≈5% over NF4’s 164.5), so the published FP8 variant is usable on Ampere.” On RTX 30-series cards, in other words, an FP8 build pays the storage savings but none of the compute benefit, since every matmul silently runs in bf16 anyway. FLUX 3 Image Klein is a different model, so this is an inference from adjacent evidence, not a measurement of Klein itself; but the tensor-core fact is a property of the silicon, not the model.

Even where FP8 runs natively, the format may not earn its complexity. Qualcomm AI Research’s FP8-versus-INT8 analysis concludes: “In direct comparison, there are no significant performance gains from FP8 over the standard INT8 format.” The same analysis prices FP8-E4 at roughly 53% higher compute cost than INT8 in dedicated hardware, and 183% less efficient when an FP32 accumulator is required. Combined with BFL’s stated support for the INT8 path down to 8 GB cards, the practical conclusion is that INT8 is the default consumer build. FP8 is a specialization for cards that actually execute it in tensor cores, and the independent evidence settles only the negative side of that question: Ampere does not qualify. Confirm your card against NVIDIA’s architecture documentation before assuming the FP8 build helps.

NVFP4, the fourth build, is the one BFL positions at the GB300 for serving, not local use. It is an S1E2M1 element format with 16-element blocks and FP8 E4M3 block scales, structurally distinct from MXFP4 per an independent numeric-format catalog. If you are buying an RTX card for this model, NVFP4 is not part of your decision.

The license in plain language

For most readers, this section decides more than the VRAM table. BFL’s terms as published set the commercial threshold at $5M in annual revenue: “The weights are available under the FLUX Kommunity License, which allows free commercial use for individuals and companies with annual revenue under $5M, alongside research and non-commercial use. Larger organizations can deploy commercially through our self-hosted licensing options or API.”

Three consequences follow:

• Under $5M annual revenue (BFL’s terms), an individual or company can use the weights commercially, locally, at no license cost. That covers most freelancers, studios, and startups, and it is genuinely permissive for a frontier-adjacent image model.
• At or above $5M, per the same terms, commercial use requires a paid self-hosted license or the API.
• The API fallback is priced by BFL from $0.009 per 1K images for the distilled model (vendor-reported, release post), including prompt upsampling and pixel restoration that leaves unedited regions of an image exactly unchanged. For teams over the threshold with modest volume, the API may be cheaper than a license negotiation, at the cost of data leaving your machines.

What the evidence does not cover matters as much as what it does. The release post establishes the revenue threshold and nothing finer: no visibility into how annual revenue is measured for a team embedded in a larger company, fine-tuning restrictions, LoRA-redistribution rights, training-data clauses, or what happens to a fine-tune’s licensing status. If your plan involves training on top of Klein and distributing the result, read the actual Kommunity License text before committing. This is a real gap, not a formality; revenue thresholds and training restrictions are exactly the clauses that bite small teams a year into a project.

Fine-tuning locally

BFL supports LoRA training from day one through Diffusers and Ostris’s AI Toolkit, with recipes in NVIDIA NeMo AutoModel that run directly from the released checkpoints. The base checkpoint is the one built for fine-tuning (release post), which makes precision choice a live question: should you fine-tune in FP8 to fit more on the card?

The evidence argues for BF16 as the reference precision. A study of FP8 versus BF16 training on Llama-3-70B found FP8 raised throughput from 415 to 570 TFLOPS but “led to unstable training loss and frequent loss spikes,” with uneven downstream degradation. That is an LLM training result, not a diffusion fine-tune, so apply it as a prior rather than a rule: FP8’s inference virtues do not transfer cleanly to optimization. On the memory side, parameter-efficient fine-tuning work is the reason this conversation is even possible on consumer hardware: adapters cut the memory and runtime cost of fine-tuning enough that BF16 base weights plus a small adapter fit where full fine-tuning never would.

Practically: BF16 weights (11.2 GB, vendor-reported) plus a LoRA make a 24 GB card the comfortable tier. That 11.2 GB is generator weights only, so on a 16 GB card the text encoder, activations, and training state will compete for what little is left. If that does not fit, the answer is a smaller adapter or gradient checkpointing, not a lower-precision base you will not be able to trust.

How to read the +91 Elo claim

BFL’s internal evaluations rank FLUX 3 Image Klein first among open models in text-to-image, multi-reference generation, and image editing, and above the closed Ideogram V4.5 and FLUX.2 [pro]. The headline comparison: “Compared with FLUX.2 [klein] 9B, it gains 91 Elo in text-to-image and 70 in image editing, with 38% fewer parameters” (BFL’s post). The comparison set includes Qwen-Image 2.1 (7B) and Krea 2 (12.9B).

Every one of those rankings is a BFL-internal evaluation, and this article cites no independent reproduction of them. Until independent comparisons appear, the honest status of “first among open weights” is: the vendor says so.

There is also a subtler reason to wait for independent numbers, and it comes from the Ideogram 4.0 quantization study. That study’s central finding was that “the variants separate on fidelity and text rendering, not on aggregate image-quality scores”: quantized builds matched the reference on CLIP and PickScore while diverging in pixel-level fidelity and on rendered text. Its INT8 recipe was statistically indistinguishable from FP8 on aggregate metrics and the most faithful 8-bit reproduction of the FP8 reference (study-reported LPIPS 0.243 against 0.277 and 0.306 for 4-bit baselines). The lesson generalizes past Ideogram: a model can top an Elo table while its quantized build quietly changes fine detail your editing pipeline depends on. For a model whose selling point is pixel-exact editing, that is the divergence to test first, with your own images, before switching production work.

Verdict

For individuals and companies under $5M annual revenue (BFL’s terms), FLUX 3 Image Klein is a strong candidate to displace larger open models in a local workflow: the INT8 build runs on 8 GB-class RTX cards (vendor-reported), the distilled checkpoint delivers single-digit-second generation on mid-range hardware, and day-one ComfyUI and Diffusers support means no waiting on the ecosystem. Pick INT8 as the default consumer build, FP8 only on cards whose tensor cores actually execute it natively, BF16 when you are fine-tuning, and leave NVFP4 to whoever owns a GB300.

The two reasons to hold off are the license’s unexamined corners and the evidence gap. If your organization is near or above the revenue threshold, or if you intend to fine-tune and redistribute, the release post alone does not answer your questions; the license text does. And if the switch depends on Klein actually beating Qwen-Image 2.1 or Krea 2 on your kind of work, wait for independent benchmarks or run the comparison yourself. At 5.6B parameters with an 8 GB floor, the cost of running that test is an afternoon, not a budget request.

Frequently Asked Questions

Which build is recommended for consumer GPUs with 8 GB of VRAM?

With the INT8 build, FLUX 3 Image Klein runs on NVIDIA GeForce RTX GPUs with as little as 8GB of VRAM, through ComfyUI or Diffusers, extending what FLUX.2 [klein] started to even smaller hardware.

Does the FP8 build offer performance benefits on Ampere GPUs?

Ampere has no FP8 tensor cores, so the FP8 checkpoint executes by dequantizing each linear to bf16 every forward; in practice this overhead is modest (FP8 runs at 172.9 s/image, ≈5% over NF4’s 164.5), so the published FP8 variant is usable on Ampere.