Fine-Tuning DeepSeek Without NVIDIA: What the Ascend SuperPOD Run Shows
SLAI T-Rex reports 34.22% MFU for full-parameter DeepSeek-V4 post-training on Ascend. The single-cluster result narrows the CUDA moat for fine-tuning, but cost data remains.
The archive · Page 2 of 6
Infrastructure & Runtime
The serving stack, network fabric, and cloud-account substrate beneath production AI, where every throughput claim collides with rebuild windows, egress invoices, and control-plane risk.
The archive
25–48 of 130 articles · Newest first
Cloudflare WebMCP: The Security Baseline for Agent-Ready Sites
Cloudflare's WebMCP proposal is unverified, but the MCP ecosystem is not. With 91.8% of servers unauthenticated, treat agent endpoints as strict APIs, not crawler extensions.
Cloudflare AI Search vs Self-Hosted RAG: Where the Build-vs-Buy Line Lands
Cloudflare's AI Search lacks verifiable pricing and access control docs. For Postgres 13+ teams, pgvector offers a documented, self-hosted alternative that avoids vendor.
Cloudflare H1 2026 DDoS Report: DNS Floods and Sizing Past 1 Tbps
Cloudflare's H1 2026 DDoS report claims DNS floods exceeded 1 Tbps. This article verifies the structural risks of UDP amplification and anycast absorption, while noting the.
daVinci-kernel shifts the RL kernel bottleneck from reward shaping to skill libraries
The daVinci-kernel preprint argues RL-tuned CUDA kernels plateau because skill libraries are mis-architected, not because rewards are mis-shaped. It co-evolves skill.
Calibrated LLM Monitoring: Conformal Prediction with Drift Detection
A cloud-native architecture packages conformal prediction as a sub-2ms microservice to replace raw logprob confidence with finite-sample coverage guarantees, gated by.
Cloudflare AI Crawler Controls: Block, Charge, or Allow Bots Per Route
Cloudflare ships per-route AI crawler controls for Search, Agent, and Training bots. Site owners must model pay-per-crawl yield against ad inventory and measure Agent.
Postgres LISTEN/NOTIFY Scales: When to Drop Redis for Job Fan-Out
DBOS benchmarks suggest Postgres LISTEN/NOTIFY handles high fan-out, but the 2026-07-24 data remains unverified. Use durable queues for small payloads, but keep Redis for.
Tailscale on Azure: Measure Direct vs DERP Routing to Control Latency and Egress
Tailscale on Azure VMs silently falls back to DERP relays when direct paths fail, adding latency and egress costs. Measure direct success rates to avoid hidden bills.
Accelerate vs Megatron Core: The Model Size Curve for Distributed Training
Accelerate simplifies FSDP and DeepSpeed for 7B to 70B models. Megatron Core delivers higher MFU for 400B+ training. The ND-Parallel feature remains unverified.
Why cgroups, not permission prompts, bound AI agent CPU and memory
AgentCgroup shows tool calls drive 15.4x memory spikes. Framework prompts gate actions but do not limit CPU or memory. Wrap agent process trees in cgroups to prevent OOM.
vLLM Configs Shift Energy, Latency, and Accuracy: A 9,000-Run Study
A 9,000-run study shows vLLM attention kernels and prefix caching shift energy, latency, and accuracy. Benchmark configs per workload before production.
AWS Estimated Billing Was Off by $1.7B: Reconciling Actual Cloud Spend
A $1.7 billion AWS estimated billing error exposes why console projections are direction indicators, not bookable numbers. Teams must reconcile against the Cost and Usage.
Cloudflare Attribution vs Custom Logs: The Per-Path AI Crawler Decision
Cloudflare Attribution packages AI crawler traffic as business insights, but rollups hide the per-path decisions operators actually need. Compare vendor dashboards against.
Spectral Compute CUDA Translation: vLLM Procurement vs Porting Cost
Spectral Compute proposes CUDA binary translation for AMD and Intel GPUs, but vLLM already supports native HIP. The real question is whether translation overhead beats the.
Running MiniCPM-V-4.6 on Fermi: What 6 GB of VRAM Forces
MiniCPM-V-4.6 runs on a 2011 Fermi GPU with 6 GB VRAM. The study shows software staging recovers multimodal inference, shifting the constraint from hardware procurement to.
LLM Inference Without a GPU: Pure CPU vs Hybrid CPU-GPU Scheduling
ATSInfer schedules LLM tensors across CPU and integrated GPU at tensor granularity, reporting up to 3.29× decode speedup where pure-CPU inference is memory-bandwidth-bound.
RL-Found CUDA Kernels Beat cuBLAS: Kernel Tuning Shifts to Reward Design
CUDA-L2 used reinforcement learning over 1,000 kernel configurations to beat cuBLAS by 19.2% on HGEMM, shifting the scarce kernel skill from hand-tuning to reward design.
pgvector vs Pinecone vs Qdrant: Picking a Vector Database in 2026
pgvector, Pinecone, and Qdrant split on deployment, not on which ANN index is fastest. The call is where vectors live, who runs the cluster, and what filtered search costs.
Ollama vs LM Studio: Picking a Local LLM Runtime in 2026
Both wrap llama.cpp, so Ollama vs LM Studio is a headless MIT daemon versus a proprietary desktop GUI, with one ceiling: neither is a production serving engine.
Beyond Quantization: LLM Efficiency Is Now a Memory-Bandwidth Problem
A new survey reframes LLM efficiency as a memory-bandwidth co-design problem, pushing 2027 fleet sizing toward HBM capacity over peak GPU FLOPS.
How Sparse LLM Weights Cut GPU Inference Cost Without Quantization
A three-layer sparse matmul kernel achieves 1.64x kernel-level and 1.41x end-to-end speedups, making moderate unstructured sparsity a viable third optimization lever.
GLM-5.2: vLLM Int4 Drops MTP Without Patches, SGLang FP8/NVFP4 Keeps It
GLM-5.2's speculative decoder speeds decode, but vLLM int4 drops MTP without a community patch. SGLang FP8/NVFP4 keeps MTP intact. Format, not kernel speed, decides serving.
Serving DeepSeek on Azure: Compliance Without Owning the GPU Fleet
DeepSeek on Azure through Vercel's AI Gateway lets regulated teams route the model inside Microsoft's perimeter, turning a binary compliance ban into a per-token cost call.