VOTE: Vision-Language-Action Model Optimization for Real-Time Robotic Manipulation

98.0%

average success on LIBERO

58.3%

on SimplerEnv WidowX, vs. 51.3% for CogACT

46 Hz

on a Jetson AGX Orin, where CogACT runs out of memory

1 token

per action chunk, no action tokenizer

Abstract

Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural language. However, current VLA models suffer from two drawbacks: (i) generation of massive tokens leading to high inference latency and increased training cost, and (ii) insufficient utilization of generated actions resulting in potential performance loss. To address these issues, we develop a training framework to finetune VLA models for generating significantly fewer action tokens with high parallelism, effectively reducing inference latency and training cost. Furthermore, we introduce an inference optimization technique with a voting ensemble strategy to combine current and previous action predictions, improving the utilization of generated actions and overall performance. Our results demonstrate that we achieve superior performance, achieving significantly higher success rates and 39× faster inference than OpenVLA with 46 Hz throughput on edge platforms, demonstrating practical deployability.

Rollouts

VOTE policies in simulation, with the instruction each episode was given.

SimplerEnv · WidowX

put the spoon on the towel

put the carrot on the plate

put the eggplant in the basket

stack the green block on the yellow block

SimplerEnv · Google robot

pick the coke can

move near

open the drawer

close the drawer

open the top drawer

open the middle drawer

LIBERO

pick up the black bowl next to the ramekin and place it on the plateSpatial

pick up the ketchup and place it in the basketObject

put the wine bottle on the rackGoal

turn on the stove and put the moka pot on itLong

pick up the black bowl on the stove and place it on the plateSpatial

pick up the milk and place it in the basketObject

open the middle drawer of the cabinetGoal

put both moka pots on the stoveLong

Method

Where the time goes

In current VLAs the VLM backbone dominates action latency: decoding the action tokens takes at least half of the total time for SpatialVLA, CogACT and OpenVLA, CogACT adds a diffusion head on top, and SpatialVLA feeds extra 3D visual tokens into a longer input sequence. Many output tokens also mean padding as many empty action embeddings during training.

Bar chart of latency in milliseconds for VLM prefill, VLM decode, action head, diffusion and total, for SpatialVLA, CogACT and OpenVLA.

One <ACT> token per action chunk

VOTE adds a special token <ACT> to the tokenizer and appends it to every instruction as the target label. After one forward pass, the final-layer hidden state of <ACT> goes to an MLP action head: it downsamples the hidden state from 4096 to 1024 dimensions and passes it through 4 bottleneck blocks, each projecting back up to 4096 and down to 1024, to predict a chunk of continuous 7-DoF actions (end-effector translation, rotation and gripper). Training combines an L1 loss on the actions with the token-level cross-entropy. No action tokenizer is needed, and only one token is generated per chunk.

Ensemble voting at test time

Each inference predicts several future steps, so at step t the robot also holds predictions for t made at earlier steps. VOTE puts the current prediction and the last K predictions for step t into a committee and splits it by cosine similarity to the current prediction (threshold τ = 0.5). The larger side wins, and the executed action is the mean of its members. Because the split is made relative to the newest prediction, it always counts as one vote for agreement, while modes that disagree with the majority are filtered out instead of being averaged in. We use K + 1 = 8.

Vote action ensemble with K = 3: four inferences at t = 0 to 3 for the instruction 'put the spoon on the towel' each predict several steps; at each step the committee of current and past predictions for that step is marked similar or not similar, and the ensembled output averages the winning side.

Vote action ensemble, with K = 3. At t = 3, more than half of the candidates differ from the current prediction, so the current one is discarded and the executed action is the mean of the three historical predictions that voted not-similar.

Results

SimplerEnv WidowX (average of four tasks, 24 trials each) and LIBERO (chunk 8). Success rate (%); latency on an RTX A6000.

Method	SimplerEnv WidowX			LIBERO
Method	Avg.	Latency (ms)↓	Speed↑	Spatial	Object	Goal	Long	Avg.
RT-1-X	1.1	—	—	—	—	—	—	—
Octo-Base	16.0	—	—	—	—	—	—	—
Octo-Small	30.0	—	—	—	—	—	—	—
OpenVLA	1.0	240	1.0	84.7	88.4	79.2	53.7	76.5
RoboVLM	31.3	—	—	—	—	—	—	—
Diffusion Policy	—	—	—	78.3	92.5	68.3	50.5	72.4
Octo	—	—	—	78.9	85.7	84.6	51.1	75.1
TraceVLA	—	—	—	84.6	85.2	75.1	54.1	74.8
π₀	27.1	—	—	96.8	98.8	95.8	85.2	94.2
π₀-FAST	32.1	470	0.5	96.4	96.8	88.6	60.2	85.5
SpatialVLA	42.7	400	0.6	88.2	89.9	78.6	55.5	78.1
CogACT	51.3	220	1.1	—	—	—	—	—
OpenVLA-OFT	—	—	—	96.2	98.3	96.2	90.7	95.3
VOTE	58.3	78	3.1	98.8	99.8	97.6	95.6	98.0

Speed is relative to OpenVLA. — = not reported. ■ best ■ second best in each column.

Inference on a desktop GPU and on an edge device.

Model	Chunk	NVIDIA RTX A6000				NVIDIA Jetson AGX Orin
Model	Chunk	Latency (ms)↓	VRAM (GB)↓	Throughput (Hz)↑	Speedup↑	Latency (ms)↓	VRAM (GB)↓	Throughput (Hz)↑	Speedup↑
OpenVLA	1	240	14.35	4.2	1.0	836	14.35	1.2	1.0
SpatialVLA	4	400	7.82	10.1	2.4	1949	7.82	2.1	1.7
CogACT	16	220	29.33	72.4	17.7	—	OOM	—	—
VOTE	8	78	14.40	102.6	24.4	346	14.40	23.1	19.3
VOTE	16	78	14.40	205.2	48.8	346	14.40	46.2	38.6

Latency = time to generate one action chunk; throughput = actions per second; each model queried 100 times with a 224×224 image. Speedup is relative to OpenVLA. OOM = out of memory.

Action ensemble strategies on SimplerEnv. Success rate (%).

Strategy	Google robot VM	Google robot VA	WidowX	Average
None	60.9	56.3	24.0	47.1
Temporal	70.5	60.3	30.2	53.7
Adaptive	71.9	60.3	49.0	60.4
Average	72.1	59.6	53.2	61.6
Ensemble voting	74.9	60.2	58.3	64.5

VM = Visual Matching, VA = Visual Aggregation. Average averages all historical predictions; Temporal is the ACT temporal ensemble; Adaptive is CogACT's adaptive ensemble. ■ best ■ second best in each column.

Action head architecture and chunk size on LIBERO. Success rate (%).

Action head	Spatial	Object	Goal	Long	Average
Isotropic, chunk 8	98.0	99.5	96.0	94.0	96.9
Isotropic, chunk 16	96.0	98.5	94.0	91.0	94.9
Bottleneck, chunk 8	98.8	99.8	97.6	95.6	98.0
Bottleneck, chunk 16	97.6	97.8	96.8	93.8	96.5

Isotropic keeps every linear layer at the 4096-d hidden size; Bottleneck is VOTE's 4096→1024 head. ■ best ■ second best in each column.

BibTeX

@inproceedings{lin2026vote,
  title     = {VOTE: Vision-Language-Action Model Optimization for Real-Time Robotic Manipulation},
  author    = {Juyi Lin and Amir Taherin and Arash Akbari and Arman Akbari and Lei Lu and Guangyu Chen and Taskin Padir and Xiaomeng Yang and Jayanth Srinivasa and Xue Lin and David Kaeli and Yanzhi Wang and Pu Zhao and Xuan Shen},
  booktitle = {International Conference on Artificial Neural Networks (ICANN)},
  year      = {2026},
  eprint    = {2507.05116},
  archivePrefix = {arXiv},
  url       = {https://arxiv.org/abs/2507.05116}
}

VOTE

Vision-Language-Action Model Optimization for Real-Time Robotic Manipulation

VOTE in 100 seconds. Why VLA decoding is slow and wastes earlier predictions, how the single <ACT> token and ensemble voting fix it, and the results on LIBERO, SimplerEnv and a Jetson Orin.

Overview of VOTE. The LLM generates one special token; its hidden state goes through an MLP action head that predicts the next actions in parallel. At test time, the current prediction and past predictions for the same step form a committee that votes on the action to execute.