RT @MayankMish98: Accelerating inference using Ladder-Residual - no custom kernels! - device agnostic! - code in pure PyTorch

@teortaxesTex Yea this is a question I'm interested in as well lol.

@kalomaze @teortaxesTex Yea I'm thinking about fixed-depth computation here.

@sbmaruf @agihippo Not known to the public, at least.

RT @yoavgo: deepseek published their V3 model a month ago and that's where all the efficiency stuff was disclosed and discussed. why are pe…

@alvations RNNs?

RT @chinwei_h: 📜 MatterGen published by Nature 📢 A generative model capable of discovering new materials. Super excited for the team! Chec…

@teortaxesTex Problem-"I don't know" pairs are model specific.

RT @riccardograzzi: I just had the pleasure of presenting "Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues" at @Autom…

@Love2Code

RT @Yikang_Shen: Granite 3.1 has been released! The new Granite LLM family has 128K context, better performance, and Apache 2.0 license. ht…

@CFGeek @teortaxesTex It takes the previous layer (iteration) as input and produces the current layer. It doesn't use later layers as input for the next time-step. It's still 'parallel' across timesteps.

@CFGeek @teortaxesTex Then I don't see what difference that has with UTs, so are they all bad approaches?

@CFGeek @teortaxesTex Do you see RNNs as "feedback recurrence" or is that a separate thing to you?

@teortaxesTex @kalomaze I have hope for some diffusion-like tricks though.... Or DEQs?

@teortaxesTex @kalomaze Computational expressivity-wise I still like UTs over Transformers for many reasons. Just think the original tweet is an oversimplification. There's interplay between computation and 'knowledge' in these models it seems to me we need more 'knowledge' for the UT.

RT @marktenenholtz: “So actually, pre-2017, we were convinced that LSTMs were the way to go for NLP. Attention was invented as an architect…