Muhammad Firmansyah Kasim
@mfkasim
Followers
1K
Following
70
Statuses
3K
Researcher at @UniofOxford | Lecturer in math at @TrinityOxford
Oxford
Joined April 2009
Really good work parallelising RNN!
Did you know that you can parallelize *nonlinear* RNNs over their sequence length!? Our @NeurIPSConf paper "Towards Scalable and Stable Parallelization of nonlinear RNNs," which introduces quasi-DEER and ELK to parallelize ever larger and richer dynamical systems! 🧵 [1/11]
0
0
10
RT @allinwithchris: WATCH: @chrislhayes honors Shaban Al Dalu, a Palestinian teen who burned to death after an Israeli strike. "That young…
0
10K
0
@_fikri_auliya Kalau diplot index vs time, bakalan jadi sawtooth signal dengan periode 2*(n-1), jadi T bisa direduksi jadi T=T%[2*(n-1)]. Habis itu tinggal dibagi aja kasusnya apakah T<=n-1 atau tidak.
0
0
2
@_fikri_auliya Kalau tensor programming di deep learning, pakai for-loop (apapun variablenya) hukumnya makruh
0
0
1
@francoisfleuret You might want to check our paper on how to compute nonlinear RNN in parallel:
0
0
3
@ZikriIsmal @aria_ghora @__nggih @nmonarizqa @rizalzaf @imcimon Tergantung jobdesc-nya. Kita biasanya nyebutnya ML research engineer, karena kerjaannya dekat ke riset. Skillsetnya juga beda dari SE: tensor programming, baca paper+implementasi, linear algebra
0
3
12
Can we parallelize RNN? We can! Now you can see how on our repo: This is the code implementation of the paper I shared last week:
0
1
8
@aria_ghora Non-convergence karena Newton's method memang ga ada jaminan bakalan converge, jadi kalau initial guess-nya jauh, bisa diverge bahkan. Paper-nya lumayan berat di matematikanya, jadi kalau ada pertanyaan, tanya aja.
1
0
0
Another drawback is scalability. Our method scales poorly: O(n^3) time complexity with the number of dimensions n. However, we can still achieve speed up using GRU with 64 dimensions (which is quite common value for experiments).
0
0
0