Why Format Belongs in the Purchase Order
Robot datasets are episodic: sequences of synchronized observations and actions, plus metadata that makes them interpretable. Every training stack expects that structure in a particular shape, and reshaping terabytes of multi-camera episodes after the fact is slow, error-prone work your researchers should never have to do. Specify the format when you specify the data, and require a loader script that reads the delivery unmodified.
LeRobot
LeRobot is Hugging Face's open-source robot learning stack, and its dataset format has become the default for the open policy ecosystem. A LeRobotDataset stores tabular streams, states, actions, timestamps, in Parquet files, with camera streams as compressed video chunks, plus JSON metadata describing features, episodes, and statistics. Datasets load through the LeRobot library in a few lines, stream from the Hugging Face Hub, and drop directly into the reference implementations of current policy architectures.
Choose LeRobot when your team trains with PyTorch, prototypes against open policy implementations, or wants the dataset shareable on the Hub internally with zero friction. It is our default recommendation for lab buyers and the format most of our deliveries ship in.
RLDS
RLDS, Reinforcement Learning Datasets, is the episodic standard built on TensorFlow Datasets. An RLDS dataset is a collection of episodes, each a sequence of steps carrying observation, action, reward, and boundary flags, serialized as TFRecords with a declared schema. It is the lingua franca of the Open X-Embodiment effort, which aggregated demonstrations from dozens of labs into a shared RLDS corpus, and it remains the shape expected by TensorFlow and JAX training pipelines descended from that line of work.
Choose RLDS when your pipeline is TF or JAX based, when you co-train against Open X-Embodiment mixtures, or when your tooling already speaks TFDS. Conversion between RLDS and LeRobot is well-trodden but not free; if both ecosystems matter to you, ask for both exports at delivery.
GR00T Compatible
NVIDIA's Isaac GR00T is a foundation model line for humanoid and manipulation policies, and its fine-tuning pipelines consume LeRobot-style episodic data with one addition: a modality configuration that maps your specific cameras, state vectors, and action dimensions onto the slots the model expects. "GR00T compatible" therefore means clean LeRobot structure plus a correct modality mapping and naming convention, validated against the fine-tuning tooling.
Choose GR00T-compatible delivery when your roadmap includes fine-tuning GR00T-family models. The practical risk is not the container but the mapping: a camera stream mislabeled at collection time surfaces as silent degradation at fine-tuning time. We validate the modality configuration against the target pipeline as part of delivery QA.
Side by Side
| Format | Container | Native Ecosystem | Best For |
|---|---|---|---|
| LeRobot | Parquet + compressed video + JSON metadata | PyTorch, Hugging Face Hub | Open policy stacks, lab workflows, fast iteration |
| RLDS | TFRecord episodes with declared schema | TensorFlow, JAX, TFDS | Open X-Embodiment co-training, TF/JAX pipelines |
| GR00T compatible | LeRobot structure + modality configuration | NVIDIA Isaac GR00T fine-tuning | GR00T-family foundation model fine-tuning |
What to Require From Any Vendor
- The format named in the agreement, with a loader script that reads the delivery without modification.
- Per-episode integrity checks and synchronized timestamps across every stream, with results delivered alongside the data.
- Camera calibration files, intrinsics and extrinsics, for every camera in every configuration used.
- A data card: collection conditions, operator protocol, device details, known limitations.
- Versioned exports if you take delivery in more than one format, so both stay traceable to the same capture.
Format is one section of a complete dataset spec. The rest of the checklist, task definition, diversity axes, episode counts, and acceptance criteria, is covered in how to spec a custom robot training dataset.
Buyer Questions
Can one collection be delivered in more than one format?
Yes. The underlying capture is the same synchronized episode data. Conversion to a second format is an export step, not a second collection, so ask for it up front and version both exports together.
Is GR00T a separate format from LeRobot?
GR00T fine-tuning pipelines consume LeRobot-style episodic data with an additional modality configuration that maps your cameras and state vectors to what the model expects. If your data is clean LeRobot, GR00T compatibility is a mapping exercise.
What should I require from a vendor regardless of format?
Per-episode integrity checks, synchronized timestamps across every stream, camera calibration files, a data card describing collection conditions, and a loader script that reads the delivery without modification.
Every collection we run is delivered in LeRobot, RLDS, or GR00T-compatible form, your choice, within 60 days of a signed spec. Get a collection quote.