Resources
Egocentric video versus teleoperation versus UMI data
Separate observations of human activity from robot actions and gripper-based demonstrations before choosing training data.
Published 2026-10-27 · 5 min read
| Question | Evidence or decision |
|---|---|
| Required learning signal | Observation, commanded action, measured state or gripper trajectory |
| Source evidence | Capture system, schema, clocks and processing |
| Compatibility | Robot or policy interface and a tested sample |
| Quantity | Accepted episodes and distinct task coverage |
Egocentric video records a first-person observation of activity. Robot teleoperation records a person controlling a robot, potentially with paired states and commands. UMI is a particular handheld-gripper demonstration and policy framework. They are not interchangeable labels for people manipulating objects.
Start with what the model must learn and which native signals are actually available. A video showing a successful action is not automatically an executable robot-action trajectory.
Human video provides observations, not implicit robot commands
An egocentric recording may show task order, objects, failures and context. Camera placement, continuity and action visibility affect usefulness. Report pose or depth generated later as derived annotations, separate from what the source recorded.
Do not infer robot joint states, gripper opening or force values from a descriptive video label. If those are missing, say so. The buyer may use the footage for a different objective, but that does not make missing control signals available.
Teleoperation needs the robot-side record
Ask for robot platform, control mode, observation timestamps, state and action schema, units, control frequency and resets. A commanded action and the measured resulting state are distinct signals. Identify latency and any resampling rather than treating row alignment as physical simultaneity.
Document operator interventions, failed attempts, episode boundaries and safety constraints. Keep derivatives linked to native logs. Different robots or control interfaces may require adaptation before data can support a target policy.
UMI is a framework, not every wrist video
The original UMI project combines handheld grippers with a designed interface and relative-trajectory policy representation. A claimed UMI package should identify its capture and processing implementation, trajectories, gripper information and applicable calibration or timing records.
Ask what is native to the device and what is reconstructed or estimated. UMI-style hardware variations need their own documentation; the research project does not certify a commercial supplier’s implementation or the suitability of a particular dataset.
Evaluate the package in the intended workflow
Request a pilot that imports into the actual training or evaluation pipeline. LeRobot documentation illustrates a format separating visual data, tabular signals and metadata; a format label alone does not guarantee all required features are populated.
Compare unique task coverage, accepted episodes, actual action signals, permissions and adaptation work. Do not compare video hours with robot-action rows as if they were the same quantity. Fiund confirms availability and scope after review, without implying that all three products are in stock.
Frequently asked questions
Is head-camera footage teleoperation?
No. Teleoperation involves controlling a robot; the delivered action and state record must be specified.
Does LeRobot format prove a dataset has robot actions?
No. Inspect the actual schema and source of each field.