groundy
articlessearch

infrastructure & runtime

128 articles·rss

The serving stack, network fabric, and cloud-account substrate beneath production AI, where every throughput claim collides with rebuild windows, egress invoices, and control-plane risk.

latest
A cutaway solid-state storage enclosure feeds a laptop through a narrow copper connection carrying small data tiles.

Running Kimi K3 From SSDs on a MacBook Pro: The Storage Tradeoff

Deltafin streams Kimi K3 weights from SSDs to run on 64 GB Macs, but self-reported benchmarks vary widely, limiting it to batch workloads rather than interactive use.

9 min
11 min

Speculative Decoding on AMD GPUs: What vLLM's Speedup Actually Costs

12 min

Cloudflare Runs Nearly 9 in 10 European CDNs: The Concentration Problem

12 min

Noisy Neighbors at the Fabric: Why Shared GPU Clusters Throttle Your Jobs

archive1–50 of 124 · page 1 / 3
  1. Running Your Own Nitter Instance: What the Post-Takedown Comeback Requires
  2. Running Local LLMs on a $60 Used GPU: What AMD's BC-250 Can and Can't Do
  3. Private Vector Search vs TEEs: Can RAG Retrieval Be Outsourced Safely?
  4. Running LLMs in the Browser: Can Privacy Be Verified Instead of Promised?
  5. Does Cloudflare's Adaptive Intelligence Change the Bot Defense Build-vs-Buy Math?
  6. Cloudflare Compresses Its Cache With Zstandard: The Storage-vs-CPU Trade
  7. Getting an AI Agent Into Cloudflare's BotBase: What Operators Must Verify
  8. Running GLM-5.2 on vLLM: MTP Support, Weight Errors, and Throughput
  9. Automating Kubernetes Capability Drops: What KubeCap's Rules Miss
  10. Alibaba's ScaleSense: When Learned Autoscaling Beats Provisioning Rules
  11. AWS Cognito Postmortem: The Real Cost of Free Managed Auth
  12. How Cloudflare Saved 100 TB in 1.1.1.1's DNS Cache and What Operators Can Copy
  13. Where Simple RAG Breaks: Multi-Document QA Needs Hierarchy, Not More Chunks
  14. Codex on AWS Bedrock 10x Charges: Auditing Agent Token Bills
  15. Cloudflare FedRAMP High Claim: Edge vs. GovCloud for Government AI
  16. GPU Memory Explained: Why LLM Throughput Collapses When VRAM Runs Out
  17. Task-Based OAuth Consent: Scoping AI Agent Permissions Per Action
  18. V8 Isolates vs MicroVMs vs Wasm: Where Spectre Still Draws the Line
  19. Fine-Tuning DeepSeek Without NVIDIA: What the Ascend SuperPOD Run Shows
  20. Cloudflare WebMCP: The Security Baseline for Agent-Ready Sites
  21. Cloudflare AI Search vs Self-Hosted RAG: Where the Build-vs-Buy Line Lands
  22. Cloudflare H1 2026 DDoS Report: DNS Floods and Sizing Past 1 Tbps
  23. daVinci-kernel shifts the RL kernel bottleneck from reward shaping to skill libraries
  24. Calibrated LLM Monitoring: Conformal Prediction with Drift Detection
  25. Cloudflare AI Crawler Controls: Block, Charge, or Allow Bots Per Route
  26. Postgres LISTEN/NOTIFY Scales: When to Drop Redis for Job Fan-Out
  27. Tailscale on Azure: Measure Direct vs DERP Routing to Control Latency and Egress
  28. Accelerate vs Megatron Core: The Model Size Curve for Distributed Training
  29. Why cgroups, not permission prompts, bound AI agent CPU and memory
  30. vLLM Configs Shift Energy, Latency, and Accuracy: A 9,000-Run Study
  31. AWS Estimated Billing Was Off by $1.7B: Reconciling Actual Cloud Spend
  32. Cloudflare Attribution vs Custom Logs: The Per-Path AI Crawler Decision
  33. Spectral Compute CUDA Translation: vLLM Procurement vs Porting Cost
  34. Running MiniCPM-V-4.6 on Fermi: What 6 GB of VRAM Forces
  35. LLM Inference Without a GPU: Pure CPU vs Hybrid CPU-GPU Scheduling
  36. RL-Found CUDA Kernels Beat cuBLAS: Kernel Tuning Shifts to Reward Design
  37. pgvector vs Pinecone vs Qdrant: Picking a Vector Database in 2026
  38. Ollama vs LM Studio: Picking a Local LLM Runtime in 2026
  39. Beyond Quantization: LLM Efficiency Is Now a Memory-Bandwidth Problem
  40. How Sparse LLM Weights Cut GPU Inference Cost Without Quantization
  41. GLM-5.2: vLLM Int4 Drops MTP Without Patches, SGLang FP8/NVFP4 Keeps It
  42. Serving DeepSeek on Azure: Compliance Without Owning the GPU Fleet
  43. Vercel Makes WAF Mitigated Traffic Free: Recompute Your Edge Cost Model
  44. GLM 5.2 Hosting Compared: Vercel AI Gateway vs Self-Hosted vLLM
  45. Cloudflare DMARC Management GA: What to Configure Before p=reject
  46. Claude Code Permissions vs OS Privilege Isolation: What the Gap Costs
  47. LLM Memory Without the RAM: What SSD-Backed Paging Actually Costs
  48. Triton Kernels Pass Tests but Run Slow: The GPU Kernel Eval Gap
  49. Vercel Edge Config: What Global Feature Flags Actually Cost at the Edge
  50. Serverless GPU Inference on GCP: What the Cold Starts Actually Cost
about this beateditorial framing

Production AI runs on infrastructure that was never designed for it. Inference serving is a moving target as prefill and decode pull apart onto different hardware, KV caches spill into tiered storage, and collective communication libraries get rewritten to claw back bandwidth. Every benchmark win on synthetic workloads has to survive long-context synthesis, multi-tenant interference, and the unglamorous math of tokens-per-dollar before it counts.

The fabric underneath is just as contested. Vector databases are converging with the OLTP stack, serverless runtimes are quietly absorbing what connection poolers used to own, and overlay networks keep colliding with cloud-provider NAT and egress policy in ways that turn architecture diagrams into invoices. Storage density is outrunning rebuild windows, forcing erasure-coding choices that used to be theoretical. Cheaper-inference research keeps threatening the assumption that scale must mean GPU farms, while denser GPU farms keep proving it.

This beat covers that tension on the merits. We track serving architectures, networking and peering economics, retrieval and caching layers, GPU and storage hardware, and the cloud-account dependencies that quietly underwrite the whole stack. We compare vendor claims against published numbers, flag when a throughput headline hides a quality regression, and pay attention to the boring failure modes that take down platforms more often than the exciting ones do.