The Shift in Robot Manipulation Data Collection
For years, training robotic manipulation models has been held back by a fundamental bottleneck: data collection. Traditionally, experts in labs gather demonstrations using proprietary hardware, process them offline, and eventually release a static, monolithic dataset. Once that dataset is shipped, it is essentially dead—it cannot evolve, and models hit a performance ceiling defined by that fixed set of examples. The team behind AXIS is overturning this paradigm.
By shifting the demonstration gathering process into the web browser, AXIS introduces a scalable, living data engine that treats datasets not as a product to be shipped, but as a project that continuously grows. By decoupling the lightweight, latency-sensitive frontend interface from the compute-intensive backend processing, they have created an infrastructure that allows for massive, distributed data collection.
Architectural Foundations of the AXIS Data Engine
The AXIS architecture relies on a strategic split between the user environment and the computational heavy lifting:
- The Browser Frontend: Operators control a Franka Research 3 robot equipped with a parallel-jaw gripper directly via a MuJoCo WebAssembly interface. This interface supports diverse input methods—keyboards, mice, virtual joysticks, or gamepads—ensuring that the barrier to entry is minimal.
- Asynchronous Processing: Physics simulation and UI rendering run independently of the React thread, ensuring that state-action logs remain precisely aligned with the simulator.
- Backend GPU Orchestration: While the browser handles interaction, heavy tasks like ray-traced rendering are offloaded to clusters of 8x RTX 4090 GPUs, while training and evaluation are managed by 8x A100 GPU clusters.
- Dynamic Task Generation: Rather than relying on static, pre-recorded tasks, the TaskGen system breaks down natural language instructions into concrete scene and object configurations. It leverages an image-to-3D pipeline to generate meshes, rescales them, and validates the physical layout through a rigorous supervision layer.
Dataset Scale and Quality Control
The latest snapshot of the AXIS data engine represents a significant leap in scale, containing 207 unique tasks across 50,129 episodes. These trajectories include comprehensive metadata—embodiment details, simulator versions, success markers, and RGB-D observations from third-party and wrist-mounted cameras.
Quality is ensured through a multi-stage post-processing pipeline. Static samples (where joint movement is below 5e-3) are discarded, and high-frequency noise is smoothed using a Savitzky-Golay filter (window 15, polynomial order 3). To ensure consistency, data is resampled using cubic splines from the browser-native 6–8 Hz range to a 20 Hz target. While this smoothing process sacrifices some raw data fidelity—reducing mean acceleration from 1.3539 to 0.4885 and mean jerk from 11.5899 to 2.2243—it increases the overall stability required for training high-performance policies.
Performance Benchmarking: LIBERO-Plus
To measure the effectiveness of the data gathered, the team evaluated policies on the LIBERO-Plus benchmark, comparing the AXIS-augmented training against vanilla checkpoints. The models utilized a PaliGemma Gemma-2B backbone paired with a Gemma-300M action expert.
| Training Condition | LIBERO-Plus Success Rate (%) |
|---|---|
| Vanilla π0.5 (Baseline) | 83.9 |
| AXIS-25% Snapshot | 84.7 |
| AXIS-50% Snapshot | 85.7 |
| AXIS-100% Snapshot | 88.8 |
| RoboCasa365 Control (Matched) | 57.5 |
The results demonstrate clear scaling benefits, with the AXIS-100% snapshot achieving an overall success rate of 88.8, notably outperforming both the baseline and the volume-matched control group.
Key Observations
- Scaling Efficacy: Pretraining performance improves consistently as more data is added, confirming the benefits of the AXIS engine’s approach to expansion.
- Perturbation Impact: The augmentation pipeline specifically improves performance in scenarios involving sensor noise and camera variations, demonstrating the value of the backend IsaacSim randomization.
- Regression Risks: Despite aggregate gains, specific axes like Light and Language showed slight regressions (1.7 and 1.3 points, respectively), suggesting that future iterations need more targeted balancing in the augmentation pipeline.
