groundy

infrastructure & runtime

102 articles·rss

Top in infrastructure & runtime

infra

daVinci-kernel shifts the RL kernel bottleneck from reward shaping to skill libraries

The daVinci-kernel preprint argues RL-tuned CUDA kernels plateau because skill libraries are mis-architected, not because rewards are mis-shaped. It co-evolves skill.

jul 30
infra

Calibrated LLM Monitoring: Conformal Prediction with Drift Detection

A cloud-native architecture packages conformal prediction as a sub-2ms microservice to replace raw logprob confidence with finite-sample coverage guarantees, gated by.

jul 28
infra

Cloudflare AI Crawler Controls: Block, Charge, or Allow Bots Per Route

Cloudflare ships per-route AI crawler controls for Search, Agent, and Training bots. Site owners must model pay-per-crawl yield against ad inventory and measure Agent.

infra

Postgres LISTEN/NOTIFY Scales: When to Drop Redis for Job Fan-Out

DBOS benchmarks suggest Postgres LISTEN/NOTIFY handles high fan-out, but the 2026-07-24 data remains unverified. Use durable queues for small payloads, but keep Redis for.

infra

Tailscale on Azure: Measure Direct vs DERP Routing to Control Latency and Egress

Tailscale on Azure VMs silently falls back to DERP relays when direct paths fail, adding latency and egress costs. Measure direct success rates to avoid hidden bills.

infra

Accelerate vs Megatron Core: The Model Size Curve for Distributed Training

Accelerate simplifies FSDP and DeepSpeed for 7B to 70B models. Megatron Core delivers higher MFU for 400B+ training. The ND-Parallel feature remains unverified.

infra

Why cgroups, not permission prompts, bound AI agent CPU and memory

AgentCgroup shows tool calls drive 15.4x memory spikes. Framework prompts gate actions but do not limit CPU or memory. Wrap agent process trees in cgroups to prevent OOM.

infra

vLLM Configs Shift Energy, Latency, and Accuracy: A 9,000-Run Study

A 9,000-run study shows vLLM attention kernels and prefix caching shift energy, latency, and accuracy. Benchmark configs per workload before production.

more in this beat1–50 of 94 · page 1 / 2
  1. jul 19infraAWS Estimated Billing Was Off by $1.7B: Reconciling Actual Cloud Spend
  2. jul 19infraCloudflare Attribution vs Custom Logs: The Per-Path AI Crawler Decision
  3. jul 18infraSpectral Compute CUDA Translation: vLLM Procurement vs Porting Cost
  4. jul 18infraRunning MiniCPM-V-4.6 on Fermi: What 6 GB of VRAM Forces
  5. jul 17infraLLM Inference Without a GPU: Pure CPU vs Hybrid CPU-GPU Scheduling
  6. jul 17infraRL-Found CUDA Kernels Beat cuBLAS: Kernel Tuning Shifts to Reward Design
  7. jul 15infrapgvector vs Pinecone vs Qdrant: Picking a Vector Database in 2026
  8. jul 14infraOllama vs LM Studio: Picking a Local LLM Runtime in 2026
  9. jul 14infraBeyond Quantization: LLM Efficiency Is Now a Memory-Bandwidth Problem
  10. jul 13infraHow Sparse LLM Weights Cut GPU Inference Cost Without Quantization
  11. jul 11infraGLM-5.2: vLLM Int4 Drops MTP Without Patches, SGLang FP8/NVFP4 Keeps It
  12. jul 11infraServing DeepSeek on Azure: Compliance Without Owning the GPU Fleet
  13. jul 10infraVercel Makes WAF Mitigated Traffic Free: Recompute Your Edge Cost Model
  14. jul 10infraGLM 5.2 Hosting Compared: Vercel AI Gateway vs Self-Hosted vLLM
  15. jul 10infraCloudflare DMARC Management GA: What to Configure Before p=reject
  16. jul 09infraClaude Code Permissions vs OS Privilege Isolation: What the Gap Costs
  17. jul 09infraLLM Memory Without the RAM: What SSD-Backed Paging Actually Costs
  18. jul 09infraTriton Kernels Pass Tests but Run Slow: The GPU Kernel Eval Gap
  19. jul 09infraVercel Edge Config: What Global Feature Flags Actually Cost at the Edge
  20. jul 09infraServerless GPU Inference on GCP: What the Cold Starts Actually Cost
  21. jul 09infraVercel In-Function Concurrency: What It Changes for Stateful Node.js
  22. jul 09infraRunning LLMs on AMD GPUs With ROCm: What Actually Works
  23. jul 08infraCloudflare Meerkat: What Globally Distributed Consensus Costs at the Edge
  24. jul 08infraVercel CDN Now Honors External Origin Cache-Control: Audit Your Headers
  25. jul 08infraAI Found Real Bugs in Cloudflare's Circl Crypto Library
  26. jul 08infraPruning RAG Context: What to Cut Before the LLM Sees It
  27. jul 08infraCloudflare's x402 Gateway: What Per-Request API Billing Actually Needs
  28. jun 30infraDoubao 2.1 Pro: What 180 Trillion Daily Tokens Means for Inference Infrastructure
  29. jun 30infraEvery CUDA Kernel Pays a Launch Tax: The Host-to-Device Walkthrough
  30. jun 29infraVercel Montreal Region: Audit Residency Before You Migrate
  31. jun 29infraGLM-5.2 on vLLM and Ascend: Open Weights Beyond NVIDIA
  32. jun 28infraHow Vercel Runs Its Own CDN in Front of Discourse: A Self-Dogfooding Case Study
  33. jun 28infraVercel Runtime Logs Surface CDN Cache Hits, Not the Eviction Cause
  34. jun 28infraMultimodal Knowledge Graph RAG vs Vector RAG: What MKG-RAG-Bench Shows
  35. jun 28infraVercel Observability Now Tracks Redirects and Rewrites Beside Function Errors
  36. jun 28infraCloudflare Workflows Saga Rollbacks: Compensating Actions in Serverless Orchestration
  37. jun 27infraStatic Corpus RAG: The Bible Case for Separating Churn from Algorithm Complexity
  38. jun 27infraVercel's KIKO Milano Black Friday Case Study: What the Scaling Claims Skip
  39. jun 27infraVercel Postgres vs Neon vs Supabase: When the Bundled DB Wins
  40. jun 27infraFine-Tuning a 20B LLM With RLHF on a 24GB GPU: What Fits
  41. jun 27infraVercel Flat Rate CDN Beta: Break-Even Math for Spiky Workloads, Tax for the Rest
  42. jun 26infraWhere DeepSeek Weights Actually Run on Vercel's AI Gateway
  43. jun 26infraVercel's Anti-Lock-In Pitch: What the Open-Source Bet Still Locks In
  44. jun 26infraVercel Adds Tag-Based CDN Cache Invalidation: Surrogate Keys at the Edge
  45. jun 26infraGLM 5.2 Fast on Vercel AI Gateway: What Routing Through Wafer Actually Buys
  46. jun 26infraVercel CDN Cache Tags vs Path Purging: When Tag Invalidation Wins
  47. jun 26infraPrisma Joins the Vercel Marketplace: The ORM Becomes the Database Vendor
  48. jun 26infraOpenAI on AWS Bedrock: Routing Math to Run Before You Move Traffic
  49. jun 25infraVercel's Function Observability: What Native Metrics Replace and What They Don't
  50. jun 25infraAWS Databases on the Vercel Marketplace: The Cross-Cloud Latency Tax
about this beateditorial framing

Production AI runs on infrastructure that was never designed for it. Inference serving is a moving target as prefill and decode pull apart onto different hardware, KV caches spill into tiered storage, and collective communication libraries get rewritten to claw back bandwidth. Every benchmark win on synthetic workloads has to survive long-context synthesis, multi-tenant interference, and the unglamorous math of tokens-per-dollar before it counts.

The fabric underneath is just as contested. Vector databases are converging with the OLTP stack, serverless runtimes are quietly absorbing what connection poolers used to own, and overlay networks keep colliding with cloud-provider NAT and egress policy in ways that turn architecture diagrams into invoices. Storage density is outrunning rebuild windows, forcing erasure-coding choices that used to be theoretical. Cheaper-inference research keeps threatening the assumption that scale must mean GPU farms, while denser GPU farms keep proving it.

This beat covers that tension on the merits. We track serving architectures, networking and peering economics, retrieval and caching layers, GPU and storage hardware, and the cloud-account dependencies that quietly underwrite the whole stack. We compare vendor claims against published numbers, flag when a throughput headline hides a quality regression, and pay attention to the boring failure modes that take down platforms more often than the exciting ones do.