Microsoft's Azure Linux Goes General-Purpose: The Container Base-Image Play
Microsoft's Azure Linux 4.0 extends the internal CBL-Mariner into a Fedora-based server OS for VMs. Preview gaps remain, and AKS teams should test now but wait for GA.
The archive · Page 6 of 6
Infrastructure & Runtime
The serving stack, network fabric, and cloud-account substrate beneath production AI, where every throughput claim collides with rebuild windows, egress invoices, and control-plane risk.
The archive
121–136 of 136 articles · Newest first
Cloudflare Acquires VoidZero, the Company Behind Vite's Rust Toolchain
Cloudflare acquired VoidZero, putting Vite, Rolldown, and Oxc maintainers on a deploy-target vendor's payroll. MIT licensing stays. Roadmap neutrality is the open question.
Tesla V100 local LLM value depends on software support
A Tesla V100 can still run compatible CUDA workloads, but its value as a local LLM purchase depends on a specific workload, a compatible software build and safe physical integration.
Why LLMs Still Botch Kubernetes Manifests: The Training-Data Gap
A 1.5B-parameter model hits 91.5% on Kubernetes YAML generation, but the remaining failures are syntactically valid manifests that deploy and quietly violate cluster intent.
Gemma 4 31B on Cloud TPU vs GPU: The Serving Cost Crossover Point
TPU v6e Flex-start delivers 308M tokens per dollar for Gemma 4 31B prefill, undercutting H100 rates for open-weight serving, but production decode costs remain unquantified.
ObjectCache Moves KV Reuse to S3-Class Storage: Why Layerwise Retrieval Beats Full-Prefix Cache Hits
ObjectCache retrieves KV cache per-layer from S3, adding 5.6% TTFT at 64K context but 56-75 ms at 4K. Long-context deployments where DRAM is the bottleneck benefit most.
vLLM 0.21 Makes Prefill-Decode Disaggregation Actually Practical
vLLM v0.21 reportedly adds bi-directional KV cache transfers between prefill and decode nodes, making P/D ratios dynamic and requiring new NIXL transfer telemetry.
OpenRAG: The Open-Source RAG Platform Challenging Pinecone
OpenRAG combines Langflow, OpenSearch, and Docling into a single deployable RAG platform. Here's how it compares to managed services like Pinecone.
MLX vs llama.cpp on Apple Silicon: Which Runtime to Use for Local LLM Inference
MLX and llama.cpp both run quantized LLMs on Apple Silicon unified memory. Here is what each project documents, and how to measure which wins on your Mac.
When prefill and decode disaggregation helps LLM serving
Prefill and decode disaggregation lets LLM serving separate prompt processing from token generation. Its value depends on latency targets, KV transfer, and worker utilization.
Google LiteRT: Running LLMs on Your Phone Without the Cloud
Google's LiteRT (formerly TensorFlow Lite) powers on-device GenAI on Android, Chrome, and Pixel. What developers need to know about private, offline inference.
BitNet b1.58 makes CPU inference an option for specific models
BitNet b1.58 makes CPU inference an option for specific models. It does not establish that GPU farms are obsolete, or that a full-precision model can be replaced without evaluating the task.
WebAssembly AI: Running Models in the Browser
How WebAssembly, WebGPU, and frameworks like Transformers.js and WebLLM run AI models in the browser: the real performance trade-offs and when to use them.
Tailscale Peer Relays are a tailnet fallback when direct paths fail
Tailscale Peer Relays let a tailnet use an explicitly configured device as a relay when direct connectivity is unavailable.
DNS Persist 01: how persistent ACME validation works
DNS Persist 01 binds a persistent DNS authorization to a certificate authority and an ACME account.
The Complete Guide to Local LLMs
Why running AI on your own hardware is becoming the default choice for privacy-conscious developers and enterprises that need data sovereignty, cost control, and low latency.
