vLLM 0.21 Makes Prefill-Decode Disaggregation Actually Practical
vLLM v0.21 reportedly adds bi-directional KV cache transfers between prefill and decode nodes, making P/D ratios dynamic and requiring new NIXL transfer telemetry.
The archive · Page 6 of 6
Infrastructure & Runtime
The serving stack, network fabric, and cloud-account substrate beneath production AI, where every throughput claim collides with rebuild windows, egress invoices, and control-plane risk.
The archive
121–130 of 130 articles · Newest first
OpenRAG: The Open-Source RAG Platform Challenging Pinecone
OpenRAG combines Langflow, OpenSearch, and Docling into a single deployable RAG platform. Here's how it compares to managed services like Pinecone.
MLX vs llama.cpp on Apple Silicon: Which Runtime to Use for Local LLM Inference
MLX and llama.cpp both run quantized LLMs on Apple Silicon unified memory. Here is what each project documents, and how to measure which wins on your Mac.
Prefill-Decode Disaggregation: The Architecture Shift Redefining LLM Serving
Prefill-decode disaggregation separates compute-bound prefill from memory-bound decode onto dedicated hardware, eliminating phase interference.
Google LiteRT: Running LLMs on Your Phone Without the Cloud
Google's LiteRT (formerly TensorFlow Lite) powers on-device GenAI on Android, Chrome, and Pixel. What developers need to know about private, offline inference.
Microsoft's BitNet: How 1-Bit LLMs Could Make GPU Farms Obsolete
Microsoft's BitNet runs ternary-weight LLMs on ordinary CPUs, with up to 6x faster inference and 82% lower energy than full-precision baselines.
WebAssembly AI: Running Models in the Browser
How WebAssembly, WebGPU, and frameworks like Transformers.js and WebLLM run AI models in the browser: the real performance trade-offs and when to use them.
Tailscale Peer Relays: The Missing Piece for True P2P Networking
Tailscale Peer Relays became generally available on February 18, 2026, enabling high-throughput peer-to-peer relaying within your own infrastructure. This feature eliminates the performance bottleneck of DERP servers when NAT traversal fails, delivering true mesh networking even in restrictive network environments.
DNS-Persist-01 Validation: Let's Encrypt's Model for Permanent ACME Certificate Authorization
DNS-Persist-01 proposes persistent DNS TXT records for ACME certificate validation, removing per-renewal DNS updates as certificate lifetimes shrink toward 47 days by 2029.
The Complete Guide to Local LLMs
Why running AI on your own hardware is becoming the default choice for privacy-conscious developers and enterprises that need data sovereignty, cost control, and low latency.
