VLA Robot Models on Jetson: Action Decoding, Not Size, Sets Latency

If you are shortlisting vision-language-action (VLA) checkpoints for a Jetson AGX Thor or a Ryzen AI Max+ PRO 395 by parameter count, you are optimizing a variable that explains at most 15% of end-to-end latency variance on those platforms. That is the central measured result of VLA-HWBench, a benchmark released in October 2026 that profiles 18 open VLA models across 19 configurations on both SoCs, with a seven-stage breakdown of every inference call. What actually sets the latency budget is how a model decodes actions: models that generate long action chunks in a few flow-matching steps keep up with control, and token-by-token decoders, as a group, do not.

One boundary before the numbers. These results come from controlled single-model configurations measured against LIBERO’s execution rate, not from full robot stacks with real sensor pipelines, actuator dynamics, or network noise. Treat them as a stage-level map of where inference time goes on this hardware, not as base rates for a deployed control loop. With that frame, the map is genuinely useful, and it overturns the most common selection heuristic in this corner of robotics.

The control-rate question, and the wrong shortlist

The standard way to pick a VLA model is to sort a leaderboard by task success and then filter by parameter count, on the assumption that a smaller checkpoint is the safer real-time pick. The leaderboard part of that workflow is real: the public VLA leaderboard tracks 112 models across 32 benchmarks, and describes itself as enabling “fair comparison of different approaches to robot learning.” What it does not track is latency or control rate at all. There is no axis on which a model can fail for being too slow to run a closed loop.

Vendor framing does not fill the gap. NVIDIA’s glossary entry on reasoning VLA models emphasizes that “reasoning VLAs break down complex tasks into manageable sub-problems and articulate their reasoning process in an interpretable form,” which is a claim about accuracy and interpretability, not about whether the model can emit actions at your control frequency on edge silicon.

The parameter-count filter fails on measured grounds. VLA-HWBench fit a least-squares regression of log latency on log parameter count, one point per checkpoint, and it explains only 9% of latency variance on the Jetson and 15% on the AMD platform, according to the paper. Size is close to noise as a latency predictor. The same measurement campaign found that only 10 of 19 configurations on the Jetson and 9 of 19 on the AMD platform generate actions as fast as LIBERO executes them. Roughly half the field fails the control-rate test on both SoCs, and you cannot predict which half from the checkpoint size or the leaderboard rank.

Inside the seven-stage call

Prior latency studies of VLA models split the call into at most three phases. VLA-HWBench instead instruments seven stages per call, from CPU preprocessing through vision encoding, prefill, and action generation, and runs every model both on its released software and on alternative backends including TensorRT and FlashRT (the latter two only on the Jetson). That granularity is what makes the results actionable: a single end-to-end number tells you a model is slow; a stage breakdown tells you which assumption about why is wrong.

The headline stage finding: “Action generation is the largest stage in 14 of the 19 configurations on the Jetson, latency spans 82×82\times on the Jetson and 157×157\times on the AMD platform, and parameter count explains at most 15% of its variance.” Those spans translate to 77 ms end-to-end for the fastest model (X-VLA) against 6.3 seconds for the slowest (VLA-0) on the Jetson, and 114 ms to 17.9 seconds on the Ryzen. The ranking is stable across platforms (Spearman’s ρ=0.91), so a model that is slow on the Jetson is very likely slow on the AMD box too.

Two cautions on those spans. They are across-model spreads, not a tuning range for any single model, and they cover wildly different control scenarios: 6.3 seconds per call is unusable for most manipulation loops, while 77 ms buys roughly 13 Hz before you count anything else in the loop. Neither number is a task-success claim. The benchmark measures speed; whether the fast models also do the task well is a separate question this evidence does not answer.

Action decoding is the lever

If size does not predict latency, what does? The paper’s answer is the action head. Quoting the authors: “Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not.”

The mechanism is straightforward. Current releases generate anywhere from 5 to 50 actions per call. A flow-matching or diffusion-style action head produces a whole chunk of future actions in a small number of denoising steps, so one expensive call amortizes across many control ticks. A token-by-token autoregressive decoder pays a sequential cost per action token, and that cost compounds with chunk length. The benchmark’s own contrast makes the amortization concrete: π0.5’s call takes 4.4× longer than VLA-Adapter’s, yet π0.5 spends less time per action, because its chunk is long enough to hide the slow call. Call latency and per-action cost can rank models in opposite orders, which is exactly the kind of inversion a leaderboard or a parameter count will never show you.

This framing is consistent with how the architecture families have been described elsewhere. The Wikipedia overview of VLA models notes that SmolVLA, a 450M-parameter model, “employs flow-matching for continuous control, and asynchronous inference to decouple the VLM backbone from the action execution,” while achieving performance comparable to much larger models such as Octo, OpenVLA, and π0. A small model with a flow-matching head and asynchronous execution is designed around precisely the constraint the benchmark measures.

The exceptions matter for model selection

“Action generation dominates” is true for 14 of 19 Jetson configurations, which means it is false for five. The paper names them: the four parallel decoders and Evo-1 spend more of the call in prefill or vision encoding, and π0.5 on the AMD platform spends more time in its FP32 vision tower, which encodes three image slots, one of them empty.

This is not a footnote. It changes the diagnostic you run. If you pick a parallel decoder or Evo-1 and assume the action head is your problem, you will optimize the wrong stage: those models are bottlenecked in prefill or vision, where batching, precision, and image-slot handling matter more than decoding strategy. And π0.5’s AMD profile shows the bottleneck can move with the platform. The empty image slot is a software artifact, not a hardware law, which suggests a configuration fix, but you only find it with a stage-level profile, not an end-to-end number.

Backend gains are real, bounded, and move the bottleneck

VLA-HWBench’s second experiment swaps backends one at a time and finds that kernel fusion with 4-bit weights, in the FlashRT backend, gives the largest gain: π0.5 and GR00T N1.7 run 2.6–2.9× faster than PyTorch with torch.compile. That is a substantial speedup, and its limits are just as important as its size. Per the paper, FlashRT “exists for only these two models and only on the Jetson.” There is no measured equivalent on the AMD platform, and no evidence the gain transfers to other checkpoints. Backend availability is a first-class selection constraint, not an afterthought: a model that misses your control budget on PyTorch might meet it on FlashRT, but only if it is one of the two models FlashRT supports.

Optimization also does not remove the bottleneck; it relocates it. After optimization, CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call, because optimization shortens the GPU stages while leaving host-side work untouched. Once you apply the fast backend, you may be shopping for faster tokenization and image preprocessing on the SoC’s CPU cores rather than more GPU kernels. Budget for that second round of work, or the 2.6–2.9× headline will not survive contact with your pipeline.

A selection framework for a control-rate budget

Putting the measured findings together, the decision sequence for these two platforms looks like this:

The ordering is the argument. Latency feasibility is a gate, task success is a ranking, and the gate is invisible in the tooling most teams use to build their shortlist.

I would also carry one operational rule out of this: never compare candidate models on per-call latency alone. With chunk sizes ranging from 5 to 50 actions per call, the π0.5 versus VLA-Adapter inversion shows per-action cost can reverse the ranking. Express every candidate in time-per-action against your control rate, then re-check the ranking.

Limits of the evidence

Everything above comes from one benchmark: 18 open models, 19 configurations, two SoCs, their released software versions, and LIBERO’s control rate as the reference. That is a narrow base for a general claim, and several things are untested. Other SoCs may shift stage balances. Full robot stacks add camera drivers, state estimation, safety monitors, and network hops that this benchmark does not include, and those additions eat the same control budget. Control frequencies above or below LIBERO’s execution rate will re-rank the “keeps up” verdicts. Energy use is unmeasured, which matters on battery-powered platforms. And the latency results say nothing about task success: the model that meets your rate budget still has to do the task.

Some of this will also go stale quickly. The absolute numbers (77 ms to 6.3 s on Jetson, 114 ms to 17.9 s on AMD), the FlashRT and TensorRT model rosters, and the 18-model field will shift with new checkpoints and SDK releases. What should survive is the method: profile the call by stage on your own hardware, filter by decoding strategy and backend availability, and treat parameter count and leaderboard rank as the last things you check, not the first. The paper’s own summary holds up as the shopping rule: where the time goes depends on how a model generates actions, not on its size.

Frequently Asked Questions

How much of latency variance does parameter count explain on Jetson and AMD platforms?

VLA-HWBench fit a least-squares regression of log latency on log parameter count, one point per checkpoint, and it explains only 9% of latency variance on the Jetson and 15% on the AMD platform, according to the paper.

Which models support the FlashRT backend and what speedup does it provide?

Per the paper, FlashRT “exists for only these two models and only on the Jetson.” There is no measured equivalent on the AMD platform, and no evidence the gain transfers to other checkpoints. Backend availability is a first-class selection constraint, not an afterthought: a model that misses your control budget on PyTorch might meet it on FlashRT, but only if it is one of the two models FlashRT supports.

What happens to the bottleneck after applying the FlashRT backend optimization?

Optimization also does not remove the bottleneck; it relocates it. After optimization, CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call, because optimization shortens the GPU stages while leaving host-side work untouched. Once you apply the fast backend, you may be shopping for faster tokenization and image preprocessing on the SoC’s CPU cores rather than more GPU kernels.