Vikash Kumar
@Vikashplus
Followers
4K
Following
3K
Media
253
Statuses
1K
Studying intelligent embodied behaviors. Ad. Prof. @CMU_Robotic | Sr. research scientist @OpenAI @GoogleAI @AIatMeta | @berkeley_ai @UWcse #MuJoCo
NewYorkCity
Joined February 2016
โ๏ธOn the way to @corl_conf for my first-ever keynote. โ๐ ๐๐ ๐ฅ๐๐๐ค โ ๐ฅ๐๐ ๐๐ฆ๐ฅ๐ฆ๐ฃ๐ ๐๐ค ๐๐๐ช๐ฅ๐๐๐๐ ๐๐ฆ๐ฅ ๐ ๐ก๐ฅ๐๐๐๐!. a bit excited โจ & a bit nervous๐คช .(+2 more talks. details๐)
5
11
134
#๐ฅ๐ผ๐ฏ๐ผ๐๐ด๐ฒ๐ป๐ -- A universal multi-task agent on a data-budget. ๐ช with 12 non-trivial skills.๐ช can generalize them across 38 tasks.๐ช& 100s of novel scenarios!. ๐w/ @mangahomanga @jdvakil @m0hitsharma, Abhinav Gupta, @shubhtuls
6
69
253
Lack of scale & diversity in robot datasets is demanding a change towards scalable alternatives- LLMs, Sim2Real, etc. GenAug presents a powerful recipe for using text2image generative models to demonstrate widespread generalization of robot behaviors to novel scenarios. ๐งต(1/N)
6
45
196
๐ข#๐ฅ๐ผ๐ฏ๐ผ๐๐ถ๐๐ฒ - a unified robot learning framework . โ
Designed for genralizn first robot-learning era.โ
Diverse (500 envs, 8 domain).โ
Single flag for Sim<>Real.โ
TeleOper Support.โ
Multi-(Skill x Task) realworld dataset.โ
pip install robohive. ๐งต๐
3
33
194
#MuJoCo 3.1.4 drops๐ง. Lots of new features - . - Gravity compensation. - Non-linear least squares (ala SysIDโ
). - MJX features (yup, still no tendons๐ซค). - many more.
3
24
179
๐ข Announcing a breakthrough in science robotics.@SciRobotics - ๐๐๐จ๐ช๐๐ก ๐ฟ๐๐ญ๐ฉ๐๐ง๐๐ฉ๐ฎ. ๐ณ๏ธโ๐ any object .๐ any rotation .๐ a low-cost hand (D'Claw).๐ทsingle camera. A single policy capable of in-hand reorientation of.novel & complex objects .(thread๐)
6
31
179
Just wait until you realize that this is full-blown physics with contacts, not just a visual rendering!. ๐ช#MuJoCo 3.0 is packed with features.๐ XLA: Accelerated physics with JAX.๐ Non-convex collisions.๐Deformable bodies.Give it a try -
1
15
169
Human Dexterity is the epitome of motor control. My PhD goal was to study this grand challenge. Amazes me to-date, we found 1st behavioral prior for human dexterity #MyoDex. It's a step change๐ชin our understanding, & ability to synthesize, physiological motor control. #proud.
Excited to present MyoDex at @icmlconf today (poster #200 at 10:30AM). We propose:.๐งช Open-Sourced MyoSuite benchmark w/ 50+ contact-rich manipulation tasks.๐ง The MyoDex prior for quickly learning novel tasks w/ RL.๐ฆพ All validated on a biologically accurate musculoskeletal arm
2
24
156
What most fails to realize. - We can solve problems that are an order more complex than ones found in robotics.- Most #RL algorithms donโt direct scale to higher dim search space .- contact discontinuities,once considered evil, arenโt daunting anymore. 1/N๐งต.
How does the brain control the numerous muscles of the body? Let's say you want to rotate two balls in your hand, how does your brain achieve that?. Read our article in @NeuroCellPress to learn more!.
2
12
144
New paper - RelayPL. Takeaway: Goal conditioned hierarchical policy when infused with unstructured play data and a relabelling trick can solve REALLY LONG horizon tasks!. Website: With A. Gupta, @coreylynch , @svlevine, @hausman_k
1
28
120
It has been a long roller coaster journey that started in 2008 when I was an intern under Emo Todorov. 12 years and now it's in the hands of the open source community. Excited for the next phase . Big congratulations to team MuJoCo and @DeepMind.
Weโve acquired the MuJoCo physics simulator ( and are making it free for all, to support research everywhere. MuJoCo is a fast, powerful, easy-to-use, and soon to be open-source simulation tool, designed for robotics research:
0
11
117
What do robot-learning & horse-racing have in common?. All candidates can never be simultaneously compared in identical real world conditions. Yet horse-racing has a leader-board & robot-learning doesnโt! . Presenting Ranking Based Robotics Benchmarks 1/N
1
30
118
Introducing a game-theoretic framework MBRL-GAME that (a) unifies existing MBRL methods (b) derives algorithms that are stable & performant (c) delivers ๐๐๐๐ผ results on OpenAI gym, ROBEL, and hand manipulation suite. Web: Paper:
2
21
102
ATLAS is such a show-off ๐. Had a phenomenal visit to @BostonDynamics discussing "Foundation Models for Low Level Motor Control". (Recording coming soon). Thanks @KuindersmaScott for the invite #bigfan. He is onto something cool, keep your eyes peeled and fingers crossed.
4
16
98
๐ขMajor update to #RoboHive v0.7๐ฅ."Unified Robot Learning Framework". Seamlessly work across #Gym & #Gymnasium w multiple teleop interfaces(sim & hardware) on latest #MuJoCo. ๐ ๐๏ธ pip install robohive
1
15
90
๐๐๐: Surprisingly simple method at the intersection of representation, imitation, and reinforcement learning that leverages features from standard pre-trained image classification models as representations to deliver complex contact-rich behaviors. 1/N
1
10
85
Learned controller (white) vs classical controller (grey) compete head to head @ #corl2022 .PS: both are being tele-oprerated
2
15
83
Woohoo,.#R3M won the best paper award at .#ICRA Scaling robot learning workshop. (). @SurajNair_1, @aravindr93 , @chelseabfinn, Abhinav Gupta
6
6
82
๐ข#๐ด๐๐๐บ๐๐๐๐๐๐๐-human parametric model. ๐ฐfeature-rich,full-body.๐ฆดanatomically accurate.โ๏ธphysiologically realistic.โก๏ธhardware accelerated.๐ฉปcomprehensive articulation.โ๏ธw/inertial, contact details.๐ฅamongst the most comprehensive models to date. PIP INSTALL MYOSUITE๐ซฐ
5
13
80
Can we solve robotics using off-domain, non-robotics datasets? . I'm bullish onโก๏ธ LLMs for high-level plans ๐ human videos for low-level robot skills. ๐ข๐ฏ๐ถ๐ท๐ด๐จ๐ต -- zero-shot manipulation in the wild from human videos!. with @mangahomanga , Abhinav Gupta, @shubhtuls.๐งต๐
2
7
77
We are building something unimaginable ๐ง. Hiring #ML, 3D #vision experts and tech-artists at all levels. Will love to help out. DMs open.
Between Apple SPG shutdown, Cruise layoffs, and shakeup at Google Robotics there are >1,000 top notch AI & Robotic engineers on the market in the Bay Area ๐. Bay area continues to be top place imo for robotics - 5yrs ago i moved from NYC here for this very reason.
4
6
77
Why does Reinforcement Learning(RL) struggle with high-dim control problems?โก๏ธExploration. Announcing the biggest speed-up in RL-algos in recent history ๐๐ฎ๐ฃ๐๐ง๐๐๐จ๐ฉ๐๐ ๐ผ๐๐ฉ๐๐ค๐ฃ ๐๐๐ฉ๐๐ฌ๐๐ฎ๐จ!.โ
compatible with all RL-libs.๐fully automatic.โno demos required.
(1/8) In July, we presented SAR at #RSS2023. We show that SAR enables SOTA high-dim control by applying the neuroscience of muscle synergies to reinforcement learning!. Check out our thread on SAR below ๐. Site (code, vids): Paper:
1
9
76
Human possess a remarkable ability to perform a broad spectrum of tasks with varying degrees of precision. Can we imparting similar levels of versatility to robots merely by watching human videos?. ๐ข H2R - an agent capable of common coarse-manipulation skills.1/N๐งต
2
12
66
The uniqueness of human intellect lies at the juncture of 1. cognitive decision making, &.2. musculoskeletal motor-control to express them. Introducing #MyoSuite -- a framework to investigate these two facets of intelligence.
1
8
62
Often overlooked in AI, morphological representations are one of the strongest form of prior behind ALL intelligent being . - acquired over generations.- has no replacement.- more fundamental+critical than acquired representations (world models, concepts, etc).
ใใใงใฆใใ็ๆณใๅพใ่้จใฏใฉใใใซใใใๆๅฐ้ใฎๅถๅพกใงใจใใซใฎใผๅน็ใฎ้ซใๆญฉ่กใๅฎ็พใใ้ณฅใใใ. #BirdBot #robot #robotics #Biorobotics #BioRob #biomimicry #ใใคใชใใใฏใชใผ #็็ฉๆจกๅฃ #ใใใงใฆใญใใใ
1
10
57
Presenting the newest member of the #ROBEL family: .โญ๏ธD'Manusโญ๏ธ- a learning platform for tactile aware manipulation . ๐ Low cost ($3500).๐10 DoF prehensile hand.๐ Surface tactile sensing (ReSkin).๐ช Robust (>10k operational hours) .โ
Open-source. 1/n
3
11
63
Reproducibility in robotics is desired, but extremely challenging, especially with hardware results. Our effort in making #ALOHA reproducible is paying off. Multiple folks have been able to reproduce and extend it to new systems. A thread ๐.
2
10
56
๐๐๐ญ๐ฅ๐๐ฃ๐๐๐ฃ๐ ๐๐ง๐ค๐ฃ๐ฉ๐๐๐ง๐จ ๐ค๐ #๐๐๐ข2๐๐๐๐ก๐. ๐ข #ICRA workshop deliberating a transformative paradigm that has pushed many frontiers. ๐ will it keep scaling/delivering?. ๐ open/missed opportunities?. ๐ฉAccepting submissions. See you at @ieee_ras_icra
1
12
55
Whatโs truly remarkable about @UnitreeRobotics and @BostonDynamics is they let their results do most of the talking! . It almost seems like Unitree has caught up (in technology) with Boston Dynamics despite having 2 decades of age gap. Anyone knows about their business numbers?.
Daily Training of Robots Driven by RL.Segments of daily training for robots driven by reinforcement learning. Multiple tests done in advance for friendly service humans.๐.The training includes some extreme tests, please do not imitate. #AI #Unitree #AGI #EmbodiedIntelligence
4
1
54
Both cost roughly the same - one is way more reusable and could have saved > 500,000$ in waste at @NeurIPSConf . ๐กIdea: Buy TVs & present posters, donate to local communities post conferences
It costs $89-$199 for a poster printing.Estimated $260.000-$597.000 for ~3k posters (main conference). $0.5M dollars go to the trash bin right after 3 hours ๐ญ. I feel sad, is there any more sustainable way to reduce this waste?? ๐ฅน #NeurIPS2024 #NeurIPS
8
3
54
The official RL library of #PyTorch is finally out๐ข๐ฃ.Thrilled to have played a small role. Take it for a whirl ๐ช๏ธ Looking forward to your valuable feedback.
TorchRL has its paper!.With Albert Bou, @MatteoBettini98, Sebastian Dittert, @Vikashplus, @shagunsodhani, Xiaomeng Yang, @gdefabritiis and @VincentMoens .
1
8
50
๐ขAnnouncing: #MyoChallenge @ #NeurIPS2022.๐ฏGoal: Physiological Dexterity.๐กTask: Develop controllers for physiologically realistic musculoskeletal #MyoHand to solve complex dexterous manipulation tasks.๐ฒ๐ die reorientation.โช๐ simultaneous rotation of two Baoding balls.1/3
3
9
51
When discussions extend well beyond research, & you learn more than you teach them. @rlfromlux wraps up his visit in style w/ phenomenal results (coming ๐) & by beating me in the farewell match (or did he ๐คฃ). ๐จ๐ We're #HIRING postdocs, interns, full-time.(details๐)
1
3
52
๐ Quite pleased to share our #GenerativeAI efforts in robotics-.#๐๐ฒ๐ป๐๐๐ด has been nominated for the ๐๐ฒ๐๐ ๐ฆ๐๐๐๐ฒ๐บ ๐ฃ๐ฎ๐ฝ๐ฒ๐ฟ ๐๐๐ฎ๐ฟ๐ฑ๐
of #RSS2023! . w- @ZoeyC17 @shokiami @abhishekunique7 at @uwcse, @MetaAI.
Lack of scale & diversity in robot datasets is demanding a change towards scalable alternatives- LLMs, Sim2Real, etc. GenAug presents a powerful recipe for using text2image generative models to demonstrate widespread generalization of robot behaviors to novel scenarios. ๐งต(1/N)
3
2
49
How robot videos should be - uncut and at 1x . Kudos to the @1x_tech team for this amazing feat.
3
3
48
While others r moving to Imitation Learning, #OG of model-based control @BostonDynamics deploys new SPOT capabilities with RL! . What's exciting? ideological camps in robotics are breaking down. The conversation is about mixing, not choosing between different approaches. (1/2)
2
4
46
Rethinking dexterous manipulation from the first principles-.โ
Algorithm (MTRF) that can learn behaviors autonomously without *any* human intervention.โ
Paradigm that can learn multiple tasks simultaneously.โ
New D'Hand that is robust to over ~2000 hours of on-hardware training!!.
After over a year of development, we're finally releasing our work on real-world dexterous manipulation: MTRF. MTRF learns complex dexterous manipulation skills *directly in the real world* via continuous and fully autonomous trial-and-error learning. Thread below ->
0
8
44
Early years is all about building representations (Vision, motor, world, morphology) strong enough to simplify decision making and execution. My 2023 realization is representation>control.
Lots of roboticists talk about how inspiring watching babies learn is. Total nonsense, worst manipulation learning algorithm I've ever seen. 2 months of basically constant supervision and no applications to real-world tasks at all. It's like grad school all over again.
4
2
43
๐Nominated for Best Manipulation Paper at @ieee_ras_icra. #HopMan outlines a new paradigm towards a generalizable universal agent โก๏ธNo RL, no imitation, only zero-shot translation of Human Interaction Plans๐ฏ. โฒ๏ธToday.- Talk: WeAA1-CC.2 <10:30-12:00>.- Poster: <13:30-15:00>.
Excited to share our latest on generalizable zero-shot manipulation in the real world!. We can train a single goal-conditioned policy that scales to over 100 diverse tasks in unseen scenarios, including real kitchens and offices. w/ @Vikashplus Abhinav Gupta @shubhtuls . 1/n
3
5
43
๐ข #๐ ๐๐ผ๐ฆ๐๐ถ๐๐ฒ ๐ฎ.๐ฌ.Towards generalizable agents โ๏ธ๐คพโโ๏ธ๐คณ๐โโ๏ธ. If generalization was easy.it would be stupid for evolution.to discover its solution in.complex neural & morphological architectures!. #MyoSuite 2.0 is packed with evolutionary secrets!๐ฐโจ.(1/N)๐งต
1
9
38
@svlevine @JitendraMalikCV My favorite from @JitendraMalikCV @ keynote panel . โI strongly pushed for robot learning benchmarks initially. I thought the field was wrong for not having one. Iโve done some robotics now. Unlike Vision benchmark, robotics ones are not obvious, or easy. I accept I was wrongโ.
1
3
41
Three projects at #ICML2023 this week w amazing students & colleagues๐. 1โฃ ๐ ๐๐ผ๐๐ฒ๐
โก๏ธ@CaggianoVitt @SudeepDasari .2โฃ ๐๐๐ฉโก๏ธ@JasonMa2020 @yayitsamyzhang @obastani @dineshja.3โฃ ๐ฉ๐ถ๐๐๐ฎ๐น ๐๐ฒ๐
๐๐ฒ๐ฟ๐ถ๐๐โก๏ธ@taochenshh M.Tippur, S.Wu, E.Adelson @pulkitology . Details๐งต๐
1
3
39
Embodied beings๐discussing embodied intelligence๐ฆพ๐ค@ieee_ras_icra. Topics:.- 3 fingers on an ๐elephant?๐คฏ.- @Vikashplus on an ๐ in 2013๐คซ.- Humanoid: future & timelines.- Legs, Hands: mistake/opportunity?. @pulkitology @mukadammh @shaneguML @CaggianoVitt @yuvaltassa @erikfrey
1
4
39
#ImitationLearning is progressing fast. Will it overshadow #Sim2Real's success? โ๏ธ. Let's debate ๐ฃ๏ธ. We are creating a safe platform at @ieee_ras_icra w\ @MyoSuite. ๐ฉSubmissions open
1
9
38
๐It was an honor to be recognized with the.**Early Career Keynote** at @corl_conf๐. Inspiring to share the stage with the best minds in #Robotics, especially now when all eyesโacademic, startup, & VCโare on the #EmbodiedAI frontier. ๐งตon how I prepared and what I shared๐
1
2
37
Can't wait to finally release what I've been working on over the last 2 years. Robot loving people -- Stay tuned. Something exciting is gonna happen ๐
1
0
37
Officially released! #ROBEL's a bet to stimulate & facilitate real-world progress in robotics. It has benefitted a wide diversity of projects (from dexterous manipulation to agile locomotion) & breath of researchers (from novice to theoreticians) at @UCBerkeley and @GoogleAI.
Check out ROBEL, an open-source platform for cost-effective robots and curated benchmarks designed to facilitate research and development on real-world hardware. Learn more below โ
2
12
36
โก๏ธALERT: Postdoc opportunities available at @MetaAI . ๐ข We are especially looking for robot learning candidates.๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค๐ค. Send me a ping if you curious and wanna know more .
0
6
37
Congratulations #CoRL2022 best papers winners! .@hausman_k, @pathak2206, @dineshjayaraman and collaborators
0
3
36
Here I though they we supposed to help clean instead !. (-ve result from an overnight run)
1
0
35
๐In search of something interesting? Stay tuned .A bunch of interesting releases lined up on @MyoSuite!
1
2
34
Last week, we presented our *single-task* multi-scene agent Today we are announcing #CACTI - a scalable *multi-task* multi-scene framework that delivers agents capable of visuomotor skill generalization to 100s of scenes โก๏ธ1/N๐งต
1
7
31
Very excited to be featured in Robotics Talent Spotlight. People will be tuning-in from around the world to discuss our present and future plans in Embodied AI. And, we are hiring for both research and engineering positions!.
1
2
33
Robotics is a journey that demands tools at the intersection of hardware and software. TRI betting into restricting mobility for better manipulability. An intriguing design choice. What benefits does this have over wheeled locomotion?.
5
2
32
Reproducibility in robotics research is necessary but has proven challenging, especially with hardware results. we took a lot of effort into packaging our research on #ALOHA for reproducibility. And its is spreading! First signs coming in. A few more are in works. .
2
5
31
Hidden behind the limelight of the new electric ATLAS, @BostonDynamics also unveiled impressive dynamic manipulation abilities. Given the progress in high-level semantic reasoning via VLMs, task-specific robust low-level (MPC) planners are the key missing ingredient!. BD? ๐
6
4
32
The gap between quadruped and biped is narrowing fast. To the best of my knowledge CHIMP from. @CMU_Robotics was the first to attempt this idea during #DRC. Any other ? .
Wait?! ๐ฎ A Cyber "Monkey" Dog?! Lite3 robot dog stands bipedally. #DEEPRobotics AI+ algorithm boosts Lite3 learning more than you can imagine. #robotics #robot #robotdog #ai #quadrupedrobot #tech
1
6
29
#RoboAgent is - .โก๏ธ 6x better than .๐คฏ needs 18x less data! . Appearing today in the Robot-Learning workshop @NeurIPSConf (Hall-B). w @mangahomanga @m0hitsharma @jdvakil Abhinav Gupta, @shubhtuls.
#๐ฅ๐ผ๐ฏ๐ผ๐๐ด๐ฒ๐ป๐ -- A universal multi-task agent on a data-budget. ๐ช with 12 non-trivial skills.๐ช can generalize them across 38 tasks.๐ช& 100s of novel scenarios!. ๐w/ @mangahomanga @jdvakil @m0hitsharma, Abhinav Gupta, @shubhtuls
0
4
31
#Robotics is progressing through a transformative self-discovery๐ toward the next era. I have been a proponent of both.- ๐๐-๐๐๐๐๐๐ ๐
๐ฟ &.- ๐๐๐๐๐๐๐๐ ๐๐๐๐ ๐๐ข๐๐๐s. @TUDarmstadt, I'll argue if at all to learn from #humans?๐คจ. Detail:
1
4
31
A perfect example why - Robotics is a quest at the intersection of hardware and software.
LocoMan.= Quadrupedal Robot + 2 * Loco-Manipulator. Powered by dual lightweight 3-DoF Loco-Manipulators and the Whole-Body Controller, LocoMan achieves various challenging tasks, such as manipulation in narrow spaces and bimanual-manipulation. ๐๐๐
0
3
31
Happening today at 2:00 @corl_conf.
โ๏ธOn the way to @corl_conf for my first-ever keynote. โ๐ ๐๐ ๐ฅ๐๐๐ค โ ๐ฅ๐๐ ๐๐ฆ๐ฅ๐ฆ๐ฃ๐ ๐๐ค ๐๐๐ช๐ฅ๐๐๐๐ ๐๐ฆ๐ฅ ๐ ๐ก๐ฅ๐๐๐๐!. a bit excited โจ & a bit nervous๐คช .(+2 more talks. details๐)
2
6
30
Real world is the litmus test for Robotics. One can't assume information or cheat. Let's bring robots out of the labs and simulations into the real world. Shout out to @chris_j_paxton and team who have been passionately working hard over an year to make it happen!.
The future of robot butlers starts with mobile manipulation. Weโre announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge!.- Full robot stack โ
.- Parallel sim and real evaluation โ
.- No robot required โ
๐.
2
7
28
An approachable blog of our #ALOHA and #ACT efforts -. A few follow up works from us that also might be of interest .- MTACT - - MobileAloha -
โ๏ธ Are you curious about Deep Learning and robotics?. The Action Chunking Transformer by @tonyzzhao et al. is a fascinating and critically important piece of research!. As a weekend fun project, I wrote a blog post to help others and me understand it:.
0
4
28
An independent study identifies #MuJoCoโs as best and easy to use simulator for deformable objects studies ๐ช.
We recorded data of two robots performing dynamic (left) and quasi-static (right) cloth manipulation. Why? .The dynamic motion produces a large deformation on cloths, really challenging to simulate!.The quasi-static requires accurate simulation of frictional and inertia forces.
0
3
29
โค๏ธExtremely grateful for this special honor from my alma mater @IITKgp! Thanks for nurturing & providing the launch platform. ๐Sincere gratitude to mentors(EmoTodorov @svlevine @fox_dieter17849 @gupta_abhinav_ ) & collaborators. ๐ท๐พ to the whole team!
6
2
29
Daily Reminder: .- These aren't renderings. - These are real. It's happening โถ๏ธ.
2
2
28
"Touch at the very least refines, & at the very best disambiguate visual estimates during in-hand manipulation"-- @Suddhus . A pearl of crucial wisdom, yet challenging to convincingly leverage & exploit in robotic dexterous manipulation. Reliable surface sensing is the key!.
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation. paper page: Neural perception with vision and touch yields robust tracking and reconstruction of novel objects for in-hand manipulation.
1
2
29
โจCuriosity around the sample efficiency of #๐ฅ๐ผ๐ฏ๐ผ๐๐ด๐ฒ๐ป๐ triggered multiple requests on its (i.e. #RoboSet) training data distribution๐ค. Attached is a stop motion of the initial object configurations for a particular scene
#๐ฅ๐ผ๐ฏ๐ผ๐๐ด๐ฒ๐ป๐ -- A universal multi-task agent on a data-budget. ๐ช with 12 non-trivial skills.๐ช can generalize them across 38 tasks.๐ช& 100s of novel scenarios!. ๐w/ @mangahomanga @jdvakil @m0hitsharma, Abhinav Gupta, @shubhtuls
0
1
28
Visiting beautiful Boston for a few days for a few talks at @BostonDynamics @MITCSAIL @amazon robotics, @Northeastern. ๐ค
0
0
29
Time and again @UnitreeRobotics has changed the game with Roboticsโs capabilities at a reasonable cost. For H1 .- form factor is lean .- motors are strong .- battrey reasonable .- cost needs to come down a bit. Reliability for continuous deployment - juryโs out there testing โณ.
Unitreeย H1 Breaking humanoid robot speed world record [full-size humanoid] ย Evolution V3.0 ๐ฅฐ.The humanoid robot driven by the robot AI world model unlocks many new skills!.Strong power is waiting for you to develop!.#Unitree #AI #subject3 #BlackTech
1
2
26
๐Exciting updates on recent submissions๐. ๐Grateful for the students, mentors, & collaborators for these fun projects -.@CaggianoVitt @SudeepDasari @JasonMa2020 @yayitsamyzhang @obastani @dineshjayaraman @ZoeyC17 cameron @tonyzzhao @abhishekunique7 @chelseabfinn @svlevine
3
2
28
๐ช Heading to #ICRA2023 this week6โฃcontributions from our group+collaborators!. Feeling extremely grateful and lucky to be a part of these collaborations.A thread with details๐งต๐
2
3
28
For a change there is scientific rigor๐ฌ on twitter, not lofty claims!! ๐ฃ. While there are many technical reasons to read this report, I wanna call it for the exquisite demonstration of candor and professionalism by the genesis team (@zhou_xian_ et al) and open source community.
Weโre excited to share some updates on Genesis since its release:. 1. We made a detailed report on benchmarking Genesis's speed and its comparison with other simulators (. 2. Weโve launched a Discord channel and a WeChat group to foster communications.
0
1
28
๐๐ค๐ฉ๐ค๐ง ๐๐๐๐๐ซ๐๐ค๐ง ๐๐จ ๐ ๐ง๐๐๐ ๐ค๐ช๐ฉ ๐ค๐ ๐๐ฃ๐ฉ๐๐ก๐ก๐๐๐๐ฃ๐๐.Over the last 2 years, I have been focusing on understanding physiological behavior synthesis, & building @MyoSuite. In this talk, I summed up the lessons learned and its relevance to Robot Learning
The first workshop on Learning Dexterous Manipulation at @RoboticsSciSys is starting now! Check out our speaker lineup at or tune in via zoom at if you are not in person.
3
5
27
๐จOur group has new openings (postdocs, interns, full-time) at the intersection of .1. multi-sensory embodied foundation models (with spatial reasoning).2. human motion tracking, reconstruction, & synthesis. spread the word/ get in touch (DMs open)
2
6
27
๐คฏ How did we train a universal agent with merely 7k trajectories?. Join #RoboAgent team today @ieee_ras_icra .- Big Data in Robotics and Automation Session at 4.30 pm (CC-313) .- Poster Session 10.30 am - 12 pm (Board 03.05).
#๐ฅ๐ผ๐ฏ๐ผ๐๐ด๐ฒ๐ป๐ -- A universal multi-task agent on a data-budget. ๐ช with 12 non-trivial skills.๐ช can generalize them across 38 tasks.๐ช& 100s of novel scenarios!. ๐w/ @mangahomanga @jdvakil @m0hitsharma, Abhinav Gupta, @shubhtuls
1
5
28
I gave a positioning talk at the human modeling workshop during @ieee_ras_icra, Japan. ๐ง AGI vs Embodied SensoryMotor intelligence.๐ง Motor-behavior vs Behavioral-Intelligence. ๐คHow can we build embodied foundation models capturing human dexterity+agility.
1
4
27
- 0.6 m/s.- model based .- motion plan + state machine .- addition of arm sways etc for naturalness .=> immersive nibble locomotion. Would be exciting to see some error recover @julianibarz.
3
2
26
The next version of our #ALOHAโก๏ธ #MOBILE_ALOHA . @tonyzzhao @zipengfu & co. outdid themselves! Congratulations. ๐My favorites: Bloopers toward the ends of the video. Don't miss out.
Introducing ๐๐จ๐๐ข๐ฅ๐ ๐๐๐๐๐๐ -- Hardware!.A low-cost, open-source, mobile manipulator. One of the most high-effort projects in my past 5yrs! Not possible without co-lead @zipengfu and @chelseabfinn. At the end, what's better than cooking yourself a meal with the ๐ค๐งโ๐ณ
0
2
26
Quite surprised by reception of #RoboAgent. Most knew about it in details, had great feedback, and share new extensions they are working on. Kudos to @tonyzzhao, @mangahomanga @m0hitsharma @jdvakil @ZoeyC17 @ZhaoMandi student leads on this research stream #ACT #GenAug, #CACATI
๐คฏ How did we train a universal agent with merely 7k trajectories?. Join #RoboAgent team today @ieee_ras_icra .- Big Data in Robotics and Automation Session at 4.30 pm (CC-313) .- Poster Session 10.30 am - 12 pm (Board 03.05).
0
3
27
Team #RoboAgent is honored to be featured as a spotlight story on @CarnegieMellon 's homepage ๐ . ๐ w @mangahomanga @m0hitsharma @jdvakil Abhinav Gupta, @shubhtuls
#๐ฅ๐ผ๐ฏ๐ผ๐๐ด๐ฒ๐ป๐ -- A universal multi-task agent on a data-budget. ๐ช with 12 non-trivial skills.๐ช can generalize them across 38 tasks.๐ช& 100s of novel scenarios!. ๐w/ @mangahomanga @jdvakil @m0hitsharma, Abhinav Gupta, @shubhtuls
0
6
27
I often get asked by VCs+academics- when will robotics happen?. ๐ชIโve always responded with huge optimism about a short timeline but with brutal honestly about the technical+non technical challenges!. For once, a form factor that I wouldnโt be afraid of bringing into my home.๐ .
Unitree G1 mass production version, leap into the future!.Over the past few months, Unitree G1 robot has been upgraded into a mass production version, with stronger performance, ultimate appearance, and being more in line with mass production requirements. We hope you like it.๐ฅณ
1
2
27
Another big realization of 2023 . ๐คcompliant gripper prongs are the easiest/simplest change that will give you the biggest performance boost. #ROI. Now try communicating & getting a paper accepted about that !!.
Dyson Robot Learning Lab is Hiring full-timers and interns! ๐ค. 1x Research Scientist:ย 1x Data Engineer (Data Collection & ML Training):ย 3x PhD Internship:ย Come join our lab of 12; located in London, UK ๐ฌ๐ง
2
3
27
If you can induce a training distribution that is super set of your test distribution, itโs almost stupid to do anything but #sim2real. We are organizing an @ieee_ras_icra workshop on Expanding frontiers of #sim2real - Details to follow soon.
We trained ANYmal to go into confined spaces such as under collapsed buildings. To be presented at #ICRA2024. Title: Learning to walk in confined spaces using 3D representation.Arxiv: Video: Summary Page:
0
2
27
To further mark the role of off-domain datasets in robotics - . Ecstatic to announce๐ขnext in the line of our .๐๐ผ๐๐ป๐ฑ๐ฎ๐๐ถ๐ผ๐ป ๐ ๐ผ๐ฑ๐ฒ๐น๐ ๐ณ๐ผ๐ฟ ๐ฅ๐ผ๐ฏ๐ผ๐๐ถ๐ฐ๐.from our group & collaborators.#RRLโก๏ธ#PVRโก๏ธ#R3Mโก๏ธ#VIPโก๏ธ#H2Rโถ๏ธ#๐๐๐ฉ. ๐งต๐.
Excited to share our #ICML2023 paper โจLIVโจ!. Extending VIP, LIV is at once a pre-training, fine-tuning, and (zero-shot!) multi-modal reward method for (real-world!) language-conditioned robotic control. Project: Code & Model: ๐งต:
1
4
26
RL suffers from ineffective search due to credit assignment problems in high dimension spaces. Not reward tuning, but exploration based methods like #LATTICE, #DepRL (@rlfromlux), SAR (Cameron), MyoDex (@CaggianoVitt @SudeepDasari) are breaking new grounds. Exciting times!!!.
Tomorrow Wed 13 Dec 10:45 a.m. CST, @a_marinvargas and I will present Lattice at #NeurIPS2023 .Lattice is a new latent exploration method for Reinforcement Learning! Also top-ranked solution of the @MyoChallenge!. Poster #1401:
0
4
24
@svlevine Behind every successful real-world training is a reset mechanism that no one talks about. Highlighting the unsung backstage heroes of our show.
0
2
24
An unpopular opinion in hot humanoid era ๐ฆพ๐ฆฟ. My bet is on @hellorobotinc as a safe and welcoming form factor most poised to enter unstructured home environments. Itโs a step in the right direction but more work is needed.
We still have a lot of work in front of us but our dream of home robots that benefit everyone is coming into focus!.
3
3
24
- search is still very inefficient. There is a lot of room to improve.- sim & real distribution is slowly collapsing. This has transformed robotics, & itโs just the beginning .- the future is robust, not optimal control. @MyoSuite is an enabling research platform for all of above.
1
1
24
PhD students should try to organize at least 1 workshop before they graduate. โก๏ธ Best opportunity to mark+catalyze your research subfield.โก๏ธ Best way to build academic network.โก๏ธBest opportunity to closely interact with senior members.โก๏ธTeaches skills school/publications doesn't.
Interested in encouraging debate/discussion on a focussed topic in Robot Learning? .Consider submitting a workshop proposal to CoRL this year. ๐
Deadline: June 22nd, 2023.โก๏ธDecisions: June 30th, 2023.๐คWorkshops: November 6, 2023.
0
3
23
I really like the transparency in communication .- It's autonomous. - It's 16% human speed!.- It's wired (no video gimmick to hide it).=> It's amazing ๐.
Figure 01 is now completing real world tasks. Everything is autonomous:. -Autonomous navigation & force-based manipulation.-Learned vision model for bin detection & prioritization.-Reactive bin manipulation (robust to pose variation).-Generalizable to other pick/place tasks
3
2
24
๐๐ฎ๐ฟ๐๐ถ๐ป'๐ ๐๐ต๐ฒ๐ผ๐ฟ๐ ๐๐ผ๐๐น๐ฑ ๐ฏ๐ฒ ๐๐ฟ๐ผ๐ป๐ด if natural selection picked such complex embodiment to just make computation a challenge for our๐ง . Morphological complexity is what enables our nimbleness.๐๐ ๐ฎ๐ด๐ฒ๐ป๐๐ ๐ป๐ฒ๐ฒ๐ฑ ๐บ๐ผ๐ฟ๐ฝ๐ต๐ผ๐น๐ผ๐ด๐ถ๐ฐ๐ฎ๐น ๐ฝ๐ฟ๐ถ๐ผ๐ฟ๐.
Want natural motion with RL and muscles, but mocap data is limiting? biological objectives + constraints now achieve natural walking with 90 muscles! .Thanks @tgeijten for contributions with #Hyfydy and @CaggianoVitt & @Vikashplus for #MyoSuite . preprint.
0
4
23
Challenge with hands is not how to built the ONE, but how to build it to last; for it is what collides with the rest of the world for all of robots life. Hands are a robots portal into the real world. ๐ฆพ.
HOT ๐ฅ fastest, most precise, and most capable hand control setup ever. Less than $450 and fully open-source ๐คฏ.by @huggingface, @therobotstudio, @NepYope. This tendon-driven technology will disrupt robotics! Retweet to accelerate its democratization ๐. A thread ๐งต
3
1
24
๐จWe're building something unique toward @vkhosla's #1. ๐ We're #hiring for a role in language-based reasoning between human embodiments.<intern๐/FTE(preferably๐บ๐ธ)>. ๐ชGot a knack for complex multi-modal reasoning? Got an exceptional understanding of #LLM APIs?. ๐ฉDrop me a DM.
Entrepreneurs, with passion for a vision, invent the future they want. These are my predictions for abundant, awesome, technology-based, Possible Tomorrows (2035-2049) . if we allow them to happen! #TED2024 @TEDTalks
0
5
24
Itโs RL - the ๐- stealing the show again!! . Embodied motor control requires generalization far exceeding LLM & VLMs. AGI experts should pay attention. @JitendraMalikCV has been claiming this as eloquently as possible for quite some time now.
Unitree B2-W Talent Awakening! ๐ฅณ.One year after mass production kicked off, Unitreeโs B2-W Industrial Wheel has been upgraded with more exciting capabilities. Please always use robots safely and friendly. #Unitree #Quadruped #Robotdog #Parkour #EmbodiedAI #IndustrialRobot
0
1
24
Appearing in #RSS2023 this week, 4โฃstrong papers from our group!. From incredible lead authors ๐ช .๐ฏ#GenAug - @ZoeyC17(best system paper nomination).๐ฏ#ALOHA - @tonyzzhao .๐ฏ#SAR - Cameron Berg.๐ฏ#LIV - @JasonMa2020 . Details in thread ๐งต๐งต๐งต.(all times GMT+9, Korea local time)
2
2
24