Toru
@ToruO_O
Followers
857
Following
184
Media
15
Statuses
86
理一@東大(UTokyo) → Course 6 @MIT → PhD student @berkeley_ai 🌈 she/her/hers I like capybaras :D
Joined October 2020
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
ميلان
• 1229650 Tweets
دورتموند
• 564576 Tweets
BRICS
• 303350 Tweets
Özgür Özel
• 86765 Tweets
Ümit Özdağ
• 81194 Tweets
#النصر_الاستقلال
• 76173 Tweets
Eminem
• 60175 Tweets
NEVER BORED ON NINGNING DAY
• 59426 Tweets
Blade
• 56788 Tweets
Siya
• 49435 Tweets
Milan
• 48218 Tweets
Celtics
• 42158 Tweets
Dortmund
• 40314 Tweets
Türkiye Türk'tür
• 29300 Tweets
Öcalan
• 25048 Tweets
Yunus Akgün
• 24835 Tweets
Monaco
• 21158 Tweets
Knicks
• 21131 Tweets
Happy Birthday Darling
• 20653 Tweets
Leao
• 15539 Tweets
Pulisic
• 14520 Tweets
#راكان_السهلي_في_شاعر_الرايه
• 12333 Tweets
Denji
• 11758 Tweets
LİDERE SADAKAT ŞEREFİMİZDİR
• 10576 Tweets
Pinned Tweet
Achieving bimanual dexterity with RL + Sim2Real!
TLDR - We train two robot hands to twist bottle lids using deep RL followed by sim-to-real. A single policy trained with simple simulated bottles can generalize to drastically different real-world objects.
5
59
218
Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies?
Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌
8
75
280
It’s year 2023 and PPO is still going strong 🙂
open-sourcing a simple & performant PPO that trains Cartpole in <1m (with IsaacGym)
same power as IsaacGymEnvs' default but everything in 4 files / under 1k lines of code
3
21
180
Update -- to appear in NeurIPS 2023 : D
TL;DR - a simple, general, flexible framework for novelty-based RL exploration
"MIMEx: Masked Input Modeling for Exploration"
arxiv:
code:
w/
@ajabri
[a rare two-author & student-only work]
New work! 🧵=>
In RL (and perhaps in life), exploration is about maximizing one's knowledge of the environment so as to act more optimally in the long run.
It is the *only* way through which an agent can learn to improve upon itself from scratch, and thus of central importance.
2
2
46
3
4
66
New work! 🧵=>
In RL (and perhaps in life), exploration is about maximizing one's knowledge of the environment so as to act more optimally in the long run.
It is the *only* way through which an agent can learn to improve upon itself from scratch, and thus of central importance.
2
2
46
Weekly research progress: didn't train any neural network but I proudly present this beautiful camera mount
4
0
40
Unfortunately neither my co-author nor I will be at NeurIPS 2023 but I still put in some effort to make this poster 🥹
3
0
33
To address the first challenge, we develop HATO ("dove" 🕊️ in Japanese), a low-cost Hands-Arms TeleOperation system using Meta Quest 2.
Our system demonstrates precise motion control capabilities and human-like dexterity -- we even controlled our robot to play Hollow Knight!
1
6
29
@YuZhang
The system can be used to control a flexible number of DoFs with joint retargeting & fingertip IK.
Video below shows how multiple hands-arms setup (Allegro,
@PSYONICinc
) can be teleoperated to solve rubik's cube, open nutella bottle, disassemble chair, and tear paper. [4x speed]
1
2
11
@YuZhang
@PSYONICinc
@hellorokoko
Personally, I still think a setup like HATO+gloves or ALOHA/GELLO+gloves will be great for teleoperating bimanual hands. Progress is happening -- look forward to more exciting development on this front in the next few months! (
@kenny__shaw
;)))
1
0
10
Thank you Nathan! We believe in a future where our robots can learn to play video games with visuotactile+proprioceptive data :) hopefully they will then have some work-life balance too
This is cool. Leverage the engineering ingenuity of prosthetic hands for tactile + dexterity with low-dof controls. Also, the teleoperated video game play (control the hand to operate a game controller) is pretty fun :)
0
0
5
0
0
8
But exploration is hard, especially in challenging environment with sparse rewards!
To encourage exploration, we derive intrinsic reward based on masked prediction loss on input trajectory sequences: the higher the loss, the more "novel" the trajectory, and the higher the reward.
1
0
6
@YuZhang
@PSYONICinc
So why didn’t we go with it? Gloves are promising, but suffer from issues such as overheating, drift and recalibration (
@hellorokoko
).
As mentioned in an earlier tweet (), HATO enables *much* faster data collection on many tasks that require hand dexterity.
1
0
6
@YuZhang
@PSYONICinc
@hellorokoko
@kenny__shaw
And shout-out to Franklin
@YuZhang
, an amazing undergraduate student who has contributed a lot to setting up these teleoperation systems despite having no prior experience at all!
Franklin is applying to PhD programs this year, and I would not hesitate to recommend :)
0
0
6
Indeed, we find that hands, even when used with limited DoFs, can perform a large set of tasks & outperform grippers because of the additional stability! This also allows for smoother and more intuitive teleop (e.g. no retargeting issue), leading to much faster data collection :D
Very incredible to see how capable VR-controlled robot hands can be. While there is a lot of debate on grippers vs hands, why not think of hands as just grippers that offer more redundancy and stability? Congrats on the great work!
1
2
23
0
0
6
To say that this is depressing would be an understatement. The implication goes far beyond abortion rights. Clarence Thomas wrote in his concurring opinion that contraception, same-sex relationships, and same-sex marriage should also be reconsidered. A big step backwards.
My thoughts on the Supreme Court's decision to overturn Roe v. Wade.
18K
172K
706K
0
0
5
~ introducing ~
"MIMEx: Masked Input Modeling for Exploration"
arxiv:
code:
a simple, general and flexible framework for intrinsic reward methods; superior results when compared to common exploration baselines like RND and ICM
1
0
5
To tackle the latter challenge, we adapt two
@PSYONICinc
prosthetic hands with touch sensors for research, enabling visuotactile data collection with HATO.
We learn skills that can complete long-horizon, high-precision tasks and generalize to different environment settings.
1
0
4
@QinYuzhe
@zhaohengyin
@HaozhiQ
Thank you Yuzhe, our sim2real buddy 🤜🏻🤛🏻 Really learned a lot from you too!!!
0
0
3
We use 3D keypoints extracted from RGBD images to represent an object.
Specifically, we segment and track object parts from the RGB frames (left), take mask centers as object part centers (middle), and estimate 3D object keypoints using noisy depth information (right).
1
0
2
Link to arxiv:
Shout out to amazing collaborators that made this happen!!!
@zhaohengyin
@HaozhiQ
@pabbeel
@JitendraMalikCV
1
0
2
Intuitively, using trajectory-level bonus encourages more novel action sequences rather than one-step transitions, giving rise to more complex behaviors and allowing more efficient exploration; it also allows for more flexible control over the difficulty of prediction problem.
1
0
3
By setting up prediction problems on trajectory sequences, MIMEx obtains intrinsic rewards that consider transition dynamics across longer time horizons and extract richer exploration signals.
1
0
3
@PSYONICinc
Project page:
Huge thanks to amazing collaborators --
@YZ_Franklin
,
@qiyang_li
,
@HaozhiQ
,
@brenthyi
,
@svlevine
, and
@JitendraMalikCV
!!! from
@berkeley_ai
🤖
0
0
3
We tackle two key challenges:
1. Lack of affordable and accessible teleoperation systems for bimanual multifingered hands
2. Lack of good hand hardware equipped with touch sensing
1
0
3
@Bubble_eio
@ajabri
Thank you for your interest! The "noise" baseline is simply adding random action noise as in the original PPO implementation, so you can reproduce it by running without exploration (e.g. as in the no_expl config file).
1
0
2
@HarryXu12
Thank you Harry!! Really excited to contribute to the comeback of RL & Sim2Real :))))))
0
0
2
@PSYONICinc
We have also released a comprehensive software suite that supports efficient data collection, multimodal data processing, scalable policy learning, and smooth policy deployment.
See our Github repo for more details:
1
0
2
[motivation: the default learning code that came with IsaacGymEnvs was such a pain to work with]
hope this could help someone else!
1
0
2
Our policy is robust against random perturbation, and adapts quickly to sustain continuous manipulation.
In the video below, we randomly poke / push objects during policy deployment; our policy could reorient and translate the perturbed objects back to stable in-hand poses.
1
0
1
@Bubble_eio
@ajabri
Both should be possible -- you can set the isaacgym option "headless=True" to enable headless mode
1
0
0
@BoyuanChen0
Thank you Boyuan!!! Totally agree and glad to see that awesome people like
@BoyuanChen0
are working on this problem too 🤜🏻🤛🏻
0
0
1
@TairanHe99
Thank you Tairan! Maybe our robots can learn to play video games on their own in the near future 😆
0
0
1
And there’re a bunch of other smaller details, e.g. orthogonal init for networks with different gains for hidden layers/policy output/value output, use eps 1e-5 instead of the default in Adam optimizer, not clipping value loss, using ELU over other activations, etc.
2
0
1
More videos of interesting emergent behaviors from our policy --
- skillfully adjusting the finger gaits and grasps of both hands to recover objects from unstable states back to stable poses
- adapting finger movements to objects of different shapes and sizes
1
0
0
@HedayatianSaeed
I don't really have a rigorous explanation (it's DRL after all!), but Engstrom ey al. (2020) and Andrychowicz et al. (2021) also show empirical results that confirm clipping VF loss does not help or even hurts performance :)
0
0
1
@PSYONICinc
We empirically investigate the effects of dataset size, sensing modality, and visual input preprocessing on policy learning.
Results and analysis can be found in our paper:
1
0
1
Overall it’s a surprising amount of details, and I tried to bring the best of everything together, but I probably definitely also overfitted to IsaacGymEnvs tasks :) thankfully, it's very easy to tune and extend everything with this code!
1
0
1
@RemiCadene
@brenthyi
Recipe: ZD10-100 flex sensors, breadboards, jumper wires, your favorite microcontroller; hand hardware and workstation; using workstation as an interchange station, take in sensor readings with Arduino and send them to the robot hand!
1
0
1