Huy Ha
@haqhuy
Followers
701
Following
1
Media
10
Statuses
22
I'm a Ph.D. student in Computer Science at Columbia University, advised by Professor Shuran Song
New York, USA
Joined October 2022
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
LINGORM FILL
• 1287288 Tweets
BRICS
• 475648 Tweets
John Kelly
• 169390 Tweets
Sara
• 93516 Tweets
#tusaş
• 79666 Tweets
ボボステ
• 57037 Tweets
#JO1ANNX
• 39590 Tweets
Hopkins
• 38033 Tweets
ドラフト
• 37655 Tweets
Türk Havacılık
• 35021 Tweets
#편도행_해적선_6년째_순항중
• 32264 Tweets
Şehitlerimize Allah'tan
• 28869 Tweets
サクラコ
• 28430 Tweets
Uzay Sanayii AŞ
• 28138 Tweets
HAPPY ATEEZ DAY
• 25004 Tweets
Penarol
• 21184 Tweets
Medina
• 13334 Tweets
$ORANGE
• 10130 Tweets
Pinned Tweet
I’ve been training dogs since middle school. It’s about time I train robot dogs too 😛
Introducing, UMI on Legs, an approach for scaling manipulation skills on robot dogs🐶It can toss, push heavy weights, and make your ~existing~ visuo-motor policies mobile!
12
86
436
How can we put robotics on the same scaling trend as large language models while not compromising on rich low-level manipulation and control?
3
43
263
For stability, we track gripper actions in world frame instead of body frame, like most prior works. So, when our robot is pushed, its arm will compensate the perturbation.
To support real world deployment, we mount an iPhone on the dog’s 🍑, running a custom iOS ARKit app.
3
6
51
@YihuaiGao
@zipengfu
@SongShuran
To learn more, check out our video, website, paper, and code!
📽️:
🌍:
📃:
💻:
1
4
28
UMI on Legs starts with UMI, a handheld gripper with a camera, that can be used to collects robot data without robots. From this data, we train manipulation policies which predicts gripper actions from image inputs.
But how should the legs move to track those gripper actions?🤔
1
3
24
For precision, we give the controller a trajectory of targets into the future, which allows anticipation of upcoming motion for dynamic trajectory tracking. For instance, the controller will brace before a fast toss, and mobilize all its legs during a push.
1
3
18
To find out more check out our paper!
Arxiv:
Website:
Code:
Or, just find out how we render the visualizations 🙃
2
1
16
A giant shoutout to my co-first author,
@YihuaiGao
, whose real world robot engineering wizardy and optimism are inspiring 🚀 Special thanks to
@zipengfu
and Jie Tan for their RL, control, and quadruped wisdom 🐕 Last but not least, thank you to my advisor,
@SongShuran
1
1
15
In a massively parallelized simulation, the robot learns through trial and error how to track UMI trajectories with stability and precision.
1
1
14
Our framework, scaling up and distilling down, provides a comprehensive solution by leveraging language guidance. We scale up automatic language and success labeled robot data generation and distill this data down to an expressive language conditioned multi task policy.
1
1
3
Using no human demonstrations, language annotation, and reward specification, our framework outputs robust visuo motor policies which can be used for real world deployment.
1
1
5
@ARX_Zhang
Thank you so much for your amazing hardware! Wouldn't have been possible if we hadn't switched to ARX5 arms 🦾🦾🦾
0
0
1
@sidharthtalia
We wanted to use odometry provided by Unitree. However, they only enable odometry during joint control mode. Further, we found that the lidar spinning can cause vibrations and shakes on the arm 🤦This is why we disconnect the lidar entirely during all of our experiments
1
0
1