Language Conditioned Imitation Learning

Chahyon Ku, Carl Winge, Aaron Fernades

University of Minnesota
Minnesota Robotics Institute 2023

Video Proposal Video Student Lecture Code

Abstract

We explore language conditioned imitation learning for tabletop robot manipulation in simulation (RLBench) and real life (UR5e)

Video Presentation

Poster

Poster

Demonstrations

Evaluations

Reach Target 1 Color Evaluation

Model trained and evaluated on 1 color (red) reach target task.

Reach Target 3 Colors Evaluation

Model trained and evaluated on 3 color (red, green, blue) reach target task with language conditioning.

Reach Target 6 Colors Evaluation

Model trained and evaluated on 6 color (red, green, blue, cyan, yellow, magenta) reach target task with language conditioning.

Pick Lemon Evaluation

Model trained and evaluated on pick lime task.

Push Block Evaluation

Model trained and evaluated on 7 push block tasks (push {cyan, yellow, magenta, lime} {left, forward}).

References

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning, Jang et al, CoRL 2021