We recently released our latest paper from
@NormalComputing
on "Thermodynamic Natural Gradient Descent" (TNGD), showcasing a groundbreaking approach to AI optimization.
But how does it work?
TNGD combines the power of GPUs and innovative thermodynamic computers called
Got my reading for the weekend:
@NormalComputing
has really been putting out some interesting/informative papers & basically teaching (although challenging) for free!
A thread on our new paper Thermodynamic Bayesian Inference
250 years later, Bayes’s theorem is still the gold standard for probabilistic reasoning. But for complicated models it’s too hard to implement exactly, so approximations are used. For example, the complexity of Bayesian
Introducing Thermodynamic Bayesian Inference.
We propose physical systems whose thermal equilibrium states correspond to Bayesian posteriors. This allows for sampling posteriors with improved time and energy costs. Deep dive coming soon.
this wonderful blog post () by
@janhkirchner
reinterprets neural circuits in the ventral stream as performing crud operations on a residual stream and i highly recommend anyone interested in neuro x interp checks it out
A cool feature of the "fewshot" LLM library is that you define all your prompts as pydantic types, which can include images!
Furthermore, it uses bayesian optimization (using optuna) to pick the most helpful examples (including images) for fewshot examples.
🎉 If you're looking for a career transforming opportunity, come meet the founders of these breakout startups in NYC at our next Founders You Should Know showcase on Oct. 2nd. Application link in comments!
- Kareem Amin from Clay (
@kareemamin
,
@clayrunhq
)
- Abiel Gutierrez from
Go beyond RAG without fine-tuning your model!
In this talk from
@klettphoebe
of
@NormalComputing
, you'll learn about Extended Mind Transformers and how they select and attend to the most relevant information at each generation step.
She demonstrates EMT's state-of-the-art
was inspired to play around with claude-generated manim animations :P
here's an animation of DDR3 state flow--the device starts in IDLE and transitions to various other states based on the allowed transition, specified by the arcs in gray
the red arcs are inhibitors and
In thermodynamics, heat flows from the hotter block to the cooler one until both blocks reach the same temperature.
But what if you want to transfer all the heat from block A to block B, not just equalize temperatures?
Just chop up the blocks like in this animation: