Skip to content
Solutions · Physical AI & Robot Data

UMI Data Collection for Robot Learning Teams

UMI data collection for robot learning teams, with ready-made kitchen, bedroom and staged retail episodes recorded on a UMI handheld, or new collection to your task list. Depth is available as an add-on, and episodes ship in LeRobot format.

Book a UMI Data Strategy Session Free 30-minute call · mutual NDA included
Ready-madeKitchen, bedroom and staged retail UMI sets.
DepthAdd-on stream for UMI episodes.
LeRobotDelivery format, with MCAP, HDF5 and RLDS on request.
The Handheld Method

What UMI Data Is and How It Is Recorded

UMI data comes from a handheld gripper with a wrist camera, carried by a person through the task, so demonstrations are recorded with no robot in the scene. Chi et al. introduced the method in the Universal Manipulation Interface (UMI) paper at RSS 2024 and open-sourced the code and hardware guide.

One wrist camera

A GoPro with a 155-degree fisheye lens is the only sensor in the paper's design, and two side mirrors add implicit stereo.

Pose from visual-inertial tracking

The paper's inertial-monocular SLAM, based on ORB-SLAM3, recovers the gripper's 6-DoF pose at metric scale.

Gripper width from markers

Fiducial markers on the fingers track how far the gripper is open in each demonstration.

A second gripper for two-handed tasks

The authors extend the setup with a second gripper for bimanual tasks, and the policy gets the relative pose between the two.

In the paper, each demonstration becomes a synchronized sequence of wrist images, end-effector pose and gripper width. Our ready-made sets and new collection are recorded on a UMI handheld.

Ready-Made Sets

Ready-Made UMI Sets for Kitchen, Bedroom and Staged Retail Scenes

Three families of UMI handheld episodes are ready to license, so your team can train and evaluate before scoping its own program.

Kitchen scenes

UMI demonstrations recorded in kitchen settings, among counters, cabinets and everyday objects.

Bedroom scenes

UMI demonstrations recorded in bedroom settings, around furniture and soft and rigid objects.

Staged retail

Real UMI demonstrations recorded in a staged mock store set, arranged like a working shop.

Licensed non-exclusive

Ready-made data is licensed non-exclusive and can be sublicensed to your own client.

Sample pack on request

A representative sample pack has the same modalities, calibration and delivery format as the full set.

Room to extend

New UMI collection adds scenes, objects or episodes to a family when your evaluation needs more.

New Collection

New UMI Collection to Your Task List

New UMI collection records fresh handheld demonstrations in the scenes and task families your policy is evaluated on. Before the pilot, we agree three things:

1 The task list, written as objects, actions and success conditions.
2 The streams each episode carries, with depth where your model uses it.
3 Acceptance criteria and reject rules for every delivered episode.
Custom Tasks

Your tasks on a robot, on two paths.

Custom collection covers a new task on an existing robot, or a new robot or embodiment. For joint-space actions or multi-finger hands, we record by teleoperation on YAM or OpenArm-class bimanual arms or the Unitree G1.

A new task on an existing robot
A new robot or embodiment
Non-exclusive or exclusive terms
Add-Ons

Depth as an Add-On for UMI Episodes

The paper's design records RGB from one wrist camera and notes the lack of direct depth perception from a monocular view. Teams whose models take depth can add a depth stream to UMI collection.

Depth on request

A depth stream joins the wrist images in each episode, in a format agreed in scoping. Bimanual arm programs can add depth too.

Where depth helps

Models with RGB-D or point-cloud input train on it directly, and your team gets a 3D record of each grasp.

Tactile and force-torque streams

Tactile or force-torque readings are add-ons on robot setups that carry the sensors, and our tactile manipulation data page covers that capture.

Compare the Options

Ready-Made UMI Sets, New UMI Collection and Custom Tasks Compared

All three ship in LeRobot format, with MCAP, HDF5 and RLDS on request.

OptionReady-made UMI setsNew UMI collectionCustom tasks
What you getRecorded UMI handheld episodes in kitchen, bedroom and staged retail scenesNew UMI handheld episodes recorded to your task list and scenesYour tasks on an existing robot, such as bimanual arms or the Unitree G1, or on a new robot or embodiment
LicenseNon-exclusive, sublicensable to your own clientNon-exclusive or exclusiveNon-exclusive or exclusive
Add-onsStreams as recorded in each set, shown in the sample packDepth on requestDepth on bimanual arms, plus tactile or force-torque where the hardware carries them
Best fitTraining and evaluation on real UMI episodes nowMore scenes, objects or episodes in the handheld formJoint-space actions, multi-finger hands or a robot-specific action space
From Handheld to Robot

How UMI Data Maps to a Robot

UMI data transfers because the robot sees and acts the way the handheld did, and the paper describes the resulting policies as hardware-agnostic.

Same fingers, same camera position

On the robot, the paper places the GoPro in the same position relative to the same 3D-printed fingers, and the authors report wrist-camera video almost indistinguishable between demonstration and deployment.

Actions relative to the gripper

End-effector poses are expressed relative to the gripper's current pose, and the policy outputs relative poses and gripper widths.

Filtering for each robot

Kinematic limits are unknown at recording time, so the paper filters trajectories by kinematics for each embodiment.

Calibration-free mounting

With the camera on the gripper in both settings, the paper reports that mounting UMI on a robot needs no camera-robot-world calibration.

The paper trains Diffusion Policy on this data. For its in-the-wild cup arrangement task, three demonstrators collected 1,400 demonstrations across 30 locations, and the policy succeeded in 43 of 60 trials (71.7%) in unseen environments. Our page on training data for ACT and Diffusion Policy covers both policies, and our teleoperation and capture services cover tasks recorded on a robot.

Delivery and Licensing

Delivery Formats and License Terms

Episodes ship in LeRobot format, and each carries provenance and consent records.

LeRobot by default

Every stream and add-on is packaged in LeRobot format for policy training.

MCAP, HDF5 and RLDS

On request, for loaders or storage that expect them.

Ready-made terms

Licensed non-exclusive, and sublicensable to your own client.

Custom terms

New and custom collection runs on non-exclusive or exclusive terms. Exclusivity covers the data and its resale rights, separate from the hardware that recorded it. Exclusive terms are priced above non-exclusive, and term and scope are agreed once a pilot is defined.

Signed participant and site agreements before capture permit commercial training, with a per-episode chain of custody you can audit.

Scoping

What Shapes the Engagement

We scope every UMI program against these eight factors with you, then quote it as a whole.

1

Ready-made, new or custom

A ready-made set, new UMI collection or custom tasks on a robot.

2

Task families and scenes

Kitchen, bedroom and staged retail, or tasks written for your evaluation.

3

Target robot

The arm and gripper your policy will run on.

4

Streams and add-ons

Which streams each episode carries, whether depth is added and the delivery format.

5

Episode length

From a single grasp to a multi-step routine.

6

Acceptance and QC criteria

What counts as a reject, such as a dropped object or an incomplete task.

7

Pilot size

How many episodes and scene variations your team needs to train and evaluate.

8

Exclusivity and sublicense rights

Non-exclusive ready-made data, or custom collection on non-exclusive or exclusive terms.

Start Today

Have a Task for UMI Demonstrations?

Send us the task, the scenes and your target robot. We'll match a ready-made UMI set or scope new collection, and send a representative sample pack with the same modalities, calibration and delivery format you'd get in production.

Book a Strategy Session →
Ask us about
Ready-made kitchen, bedroom and staged retail UMI sets
New UMI collection to your task list and scenes
Depth as an add-on stream
Custom tasks on bimanual arms or the Unitree G1
Non-exclusive, sublicensable or exclusive terms
Questions

Frequently Asked Questions on UMI Data Collection

UMI data collection records manipulation demonstrations with a handheld gripper and wrist camera, so a person performs the task without a robot in the scene. Policies trained on it run on a robot fitted with the same fingers and camera position, as the UMI paper describes.

Three families of UMI handheld episodes, recorded in kitchen, bedroom and staged retail scenes, with the retail set recorded in a mock store. They are licensed non-exclusive, can be sublicensed to your own client and ship in LeRobot format, with a representative sample pack on request.

Yes. Depth is an add-on for UMI collection, agreed in scoping with the other streams. The paper's design records RGB from one wrist camera, so depth suits models that take RGB-D or point-cloud input.

UMI records a person moving a handheld gripper, so actions are end-effector trajectories and gripper widths. Teleoperation records a robot driven by an operator, so joint states and actions come from that robot. Tasks that need joint-space actions or multi-finger hands are recorded by teleoperation on bimanual arms or the Unitree G1.

We scope and quote each program as a whole, based on the option, task list, streams and add-ons, pilot size and license. Exclusive terms are priced above non-exclusive.

Scope Your UMI Data Collection

A 30-minute strategy call. Tell us the tasks, scenes and target robot, and we'll match a ready-made UMI set or scope new collection.

Discuss Your Project