DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy

We present DemoDiffusion: a simple and scalable method for enabling robots to perform manipulation tasks by imitating a single human demonstration, without requiring any paired human-robot data. DemoDiffusion refines a re-targeted human demonstration trajectory using a pre-trained generalist diffusion policy, efficiently bridging the embodiment gap and enabling the robot to succeed even on tasks where the pre-trained generalist policy fails entirely.

Overview

Experiments

1. Task Rollouts (x2 speed for robot)

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

Human Demonstration

DemoDiffusion

2. Comparison to Baselines (x2 speed for robot)

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion


3. Zero-Shot Generalization to New Objects (x2 speed for robot)

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion

Pi-0

Kinematic Retargeting

DemoDiffusion



4. Detailed Results for Hand Reconstruction and Robot Execution

Task: Drag the Basket to the Right