How to Measure Post-Quantum TLS Adoption in Cloudflare Traffic Logs
Cloudflare logs TLS key-exchange groups to measure hybrid post-quantum adoption. The metric tracks encryption only, not authentication, and classical shares often reflect non-
The topic guide
Infrastructure & Runtime
The serving stack, network fabric, and cloud-account substrate beneath production AI, where every throughput claim collides with rebuild windows, egress invoices, and control-plane risk.
134 published articlesFollow this topic via RSS
Latest analysis
Start with these guides
Foundational reading and comparisons to help you get your bearings.
MLX vs llama.cpp on Apple Silicon: Which Runtime to Use for Local LLM Inference
MLX and llama.cpp both run quantized LLMs on Apple Silicon unified memory. Here is what each project documents, and how to measure which wins on your Mac.
Ollama vs LM Studio: Picking a Local LLM Runtime in 2026
Both wrap llama.cpp, so Ollama vs LM Studio is a headless MIT daemon versus a proprietary desktop GUI, with one ceiling: neither is a production serving engine.
Prefill-Decode Disaggregation: The Architecture Shift Redefining LLM Serving
Prefill-decode disaggregation separates compute-bound prefill from memory-bound decode onto dedicated hardware, eliminating phase interference.
pgvector vs Pinecone vs Qdrant: Picking a Vector Database in 2026
pgvector, Pinecone, and Qdrant split on deployment, not on which ANN index is fastest. The call is where vectors live, who runs the cluster, and what filtered search costs.
What I cover
Production AI runs on infrastructure that was never designed for it. Inference serving is a moving target as prefill and decode pull apart onto different hardware, KV caches spill into tiered storage, and collective communication libraries get rewritten to claw back bandwidth. Every benchmark win on synthetic workloads has to survive long-context synthesis, multi-tenant interference, and the unglamorous math of tokens-per-dollar before it counts.
The fabric underneath is just as contested. Vector databases are converging with the OLTP stack, serverless runtimes are quietly absorbing what connection poolers used to own, and overlay networks keep colliding with cloud-provider NAT and egress policy in ways that turn architecture diagrams into invoices. Storage density is outrunning rebuild windows, forcing erasure-coding choices that used to be theoretical. Cheaper-inference research keeps threatening the assumption that scale must mean GPU farms, while denser GPU farms keep proving it.
I examine that tension on the merits, tracking serving architectures, networking and peering economics, retrieval and caching layers, GPU and storage hardware, and the cloud-account dependencies that quietly underwrite the whole stack. I compare vendor claims against published numbers, flag when a throughput headline hides a quality regression, and pay attention to the boring failure modes that take down platforms more often than the exciting ones do.
The archive
1–24 of 130 articles · Newest first
Why GPU Utilization Misleads LLM Inference Capacity Planning
A preprint argues GPU utilization misleads LLM capacity planning by conflating memory-bound decode with compute saturation, urging per-phase metrics.
Running MoE LLMs on a Single GPU: What Expert Offloading Actually Costs
FluxMoE streams MoE weights from host DRAM to save VRAM, trading capacity for bandwidth. Author-reported gains on multi-GPU setups lack independent replication.
Running Kimi K3 From SSDs on a MacBook Pro: The Storage Tradeoff
Deltafin streams Kimi K3 weights from SSDs to run on 64 GB Macs, but self-reported benchmarks vary widely, limiting it to batch workloads rather than interactive use.
Speculative Decoding on AMD GPUs: What vLLM's Speedup Actually Costs
vLLM's ROCm speculative decoding speedup is self-reported and unreplicated. AMD's gigawatt deals de-risk the platform, but operators must benchmark acceptance rates on MI300X.
Cloudflare Runs Nearly 9 in 10 European CDNs: The Concentration Problem
A Hacker News claim that 9 in 10 European CDN users rely on Cloudflare lacks independent verification. Operators must audit failover paths to avoid correlated failure and meet
Noisy Neighbors at the Fabric: Why Shared GPU Clusters Throttle Your Jobs
A new NCCL shim recovers 13-38% bandwidth on shared GPU clusters by tuning collective patterns. Test for cross-tenant interference before buying more fabric.
Running Your Own Nitter Instance: What the Post-Takedown Comeback Requires
Self-hosting Nitter in 2026 is a maintenance contract, not a setup task. Operators must rotate banned X tokens, track upstream commits, and absorb legal exposure from active C
Running Local LLMs on a $60 Used GPU: What AMD's BC-250 Can and Can't Do
A claimed $60 AMD BC-250 with 16GB VRAM shifts the local LLM bottleneck from capacity to software support. Verify ROCm and llama.cpp backend compatibility before buying, as un
Private Vector Search vs TEEs: Can RAG Retrieval Be Outsourced Safely?
Spruce claims 0.21-2.97s private retrieval via MPC, challenging TEE defaults. Compare self-hosted Qdrant, TEEs, and cryptographic outsourcing for privacy-sensitive RAG.
Running LLMs in the Browser: Can Privacy Be Verified Instead of Promised?
Browser LLM inference is local but not private. The host page controls the GPU, model bytes, and network egress, making 'data never leaves device' a policy promise, not a ver
Does Cloudflare's Adaptive Intelligence Change the Bot Defense Build-vs-Buy Math?
Cloudflare's bundled bot defense changes procurement math, but efficacy claims remain self-reported. Audit standalone spend and measure false positives before migrating.
Cloudflare Compresses Its Cache With Zstandard: The Storage-vs-CPU Trade
Cloudflare's zstd cache proposal trades storage for CPU. We verify the math: zstd saves 5% over gzip but only 0.45% over brotli, while non-zstd clients pay a 10ms re-encode.
Getting an AI Agent Into Cloudflare's BotBase: What Operators Must Verify
Cloudflare's agent directory proposal shifts identity burden to operators. With 20% of the web behind its network, unlisted agents face default blocking or per-route charges,
Running GLM-5.2 on vLLM: MTP Support, Weight Errors, and Throughput
GLM-5.2 on vLLM v0.28.0: enable MTP, fix k_norm warnings, and verify FP8 precision. Measured gains from research on other models, not GLM-5.2.
Automating Kubernetes Capability Drops: What KubeCap's Rules Miss
KubeCap automates Kubernetes capability minimization via LLM-inferred rules, cutting permissions by 55% in Go workloads. Validate inferred drops against runtime behavior to CI
Alibaba's ScaleSense: When Learned Autoscaling Beats Provisioning Rules
Alibaba's ScaleSense uses learned query-level estimation to cut AnalyticDB costs by up to 5.22x. This audit covers the risks of opaque autoscalers and the limits of vendor-own
AWS Cognito Postmortem: The Real Cost of Free Managed Auth
A founder's Cognito postmortem highlights hidden auth costs. This guide maps decision axes for startups, separating verified AWS capabilities from unverified customization and
How Cloudflare Saved 100 TB in 1.1.1.1's DNS Cache and What Operators Can Copy
Cloudflare claims 100 TB saved in 1.1.1.1 cache, but the mechanism is unverified. Operators can cut memory now by tuning Redis eviction policies and admission controls, not by
Where Simple RAG Breaks: Multi-Document QA Needs Hierarchy, Not More Chunks
Flat RAG fails on similar-document corpora due to scope confusion and entity conflicts. HiQA proposes hierarchical augmentation, but hybrid retrieval remains the cheaper, unme
Codex on AWS Bedrock 10x Charges: Auditing Agent Token Bills
A user-reported 10x charge on AWS Bedrock remains unconfirmed by OpenAI or AWS. This runbook reconciles Codex client logs against provider metering to isolate cache misses, or
Cloudflare FedRAMP High Claim: Edge vs. GovCloud for Government AI
Cloudflare claims FedRAMP Class D status, but the Marketplace shows no listing. This guide maps edge security layers to impact levels and compares latency costs for.
GPU Memory Explained: Why LLM Throughput Collapses When VRAM Runs Out
LLM decode speed tracks memory bandwidth, not TFLOPS. When VRAM fills, reads shift to PCIe, causing a throughput cliff. Use this bandwidth budget worksheet to plan KV cache.
Task-Based OAuth Consent: Scoping AI Agent Permissions Per Action
Cloudflare's task-based OAuth consent moves authorization from install time to runtime. Compare blast radius, revocation, and fatigue to redesign agent API security.
V8 Isolates vs MicroVMs vs Wasm: Where Spectre Still Draws the Line
Cloudflare's 2026 Spectre revisit is unverified. This guide maps V8 isolates, Firecracker microVMs, and Wasm sandboxes to define a data-classification rule for edge secrets.
