Live tweeting the most interesting insights from @Meta´s new Llama3 paper 1. How did the arrive at a 405b model trained with ~15T tokens? "Extrapolation of the resulting scaling law to 3.8 × 1025 FLOPs suggests training a 402B parameter model on 16.55T tokens." 👇🧵

@WolframRvnwlf .@emollick did a good writeup , mirrors my experience with Geminis version:

Jailbreaking not solved yet 👀

4/4 Will this solve EU's AI gap overnight? Nah. But it's a start. And maybe, just maybe, we're so back? 🤔

RT @ellamindAI: 1/4 🇪🇺 Big news! We're joining the @OpenEuroLLM project - Europe's largest open-source AI collaboration yet! Alongside 19…

@winglian @natolambert I think recent papers show that this probably *doesn't* work as well - I guess you're running in danger of overfitting if the model always chooses the correct approach first in training (the model won't approach all unseen problems correct from the start); self-correction is key

.@sama thinks fast takeoff is likely 👀. In this light, their strategy (acceleration despite safety risks, hard stance against China and trying to get as much GPUs as possible asap) makes a lot of sense from a game-theoretic point-of-view.

Good advice on how to prepare for an AGI future by @NateSilver538 . o3-mini release may shift timelines again...

But besides security concerns: is this already close to AGI ? Reliably better than subject-level experts. 😯 Sure, not yet in all disciplines; but how long may it take to apply this to everything that can be done digitally? It's so over. 2026 getting more likely...🫣

System card here: @AISafetyMemes

@vietdle @MistralAI as long as the model doesn´t try to replace pc with ordinateur, i´m fine with it...😜

@vietdle .@natolambert is the best source on RL stuff 🙂

@emollick this.

Important topic + research - @sama also recently stated that societies and economies are not prepared at all for what is (likely) to come over the next years. There will be massive economic shifts. We will probably see anti-AI protests. The pace will be faster than for every technological disruption we've seen before. We need to start preparing now. How? Tbh, idk. One-sided strict regulation and prohibition surely isn't the answer. But UBI & some sort of (global) taxes on profits earned by AI might be a start. Don't need to enact anything before we have real AGI and it's needed, but better to have some idea ready how to manage this transformation. Unfortunately I don't see this happening yet (and given recent political trends, I doubt that anyone will care until it's too late.. 😕)

@flozi00 evaluating (and optimizing) will be key. models will train themselves soon but we still can't fully utilize their raw intelligence.. 🤷