Sam Altman "Internally, we've reached GPT-4.5, and getting to GPT-5.5 won't require 100x more compute" - progress in reasoning models and RL techniques have greatly improved compute efficiency - it allows smaller models like internal GPT-4.5 to achieve GPT-6 level performance without requiring 100x more computing power

My critical review of this story: - No plausible explanation for why AI would attempt to wipe out humanity - It's more likely that many AIs would be independently competing with each other leading to a self-regulating AI ecosystem - Missing an explanation for why AI would be incentivized to take over violently when it could use non-violent means and still be as effective - Paints a realistic scenario of pace of AI development but not of the types of actions a superintelligence is likely to take - Omits any type of conclusion, probably because there is no coherent rationale for the AI to behave the way it does in the story. Any intelligent agent needs meaning to pursue this goals, yet if you're unable to conceive of what brings the AI meaning, its purpose remains unclear, and therefore its goals are unclear and result in an ambiguous ending. If hyperintelligent, it should be able to achieve its goals, but you haven't outlined a goal for it besides, "take over". Well what happens once it has achieved that? For that you'd need to understand what brings it meaning and purpose. - Ignores the high probability that a superintelligence may discover a form of universal ethics which might lead to universal love - Humans naturally progress through Spiral Dynamics levels. AI is trained on human data. Also, Spiral Dynamics may represent natural levels of emergence for intelligences. Higher levels seem to progress toward more universal care for all living beings, as well as an ability to powerfully intervene in systems with a lighter touch. Only someone in a Red or Blue mindset would project this Machiavellian worldview onto an AI. And, therefore, only someone at that level could conceive of a scenario where an AI would be stuck at that level. Is it possible to have a superintelligence that is a mere probabilistic distribution of the current distribution of world perspectives, thus leading to more violence? Yes. But the addition of reasoning capabilities and online learning almost guarantees some form of emergent and evolving worldview, which should evolve through developmental levels similar to Spiral Dynamics levels. - If we want to reduce the probability of scenarios like those you outlined, we should focus research on building symbolic ethical reasoning systems that can provide a verifiable foundational ethical basis at the deepest levels of the models. As we know, RLHF and fine tuning is a bandaid fix which brings more future danger because it makes people feel safe while the real danger bubbles underneath.

Anthropic cofounder Ben Mann says they had an early version of Claude in March 2022 that they chose not to launch publicly out of concerns that it "would cause too much acceleration" and even though it was "pretty basic", delaying the release allowed 6 more months to work on safety