Andrew Critch (π€π©Ίπ)
@AndrewCritchPhD
Followers
4K
Following
3K
Media
65
Statuses
1K
Let's make AI doctors! Views my own; CEO @ https://t.co/wvoKT50fKX; AI Researcher @ Berkeley; If I block you it's like I'm moving to another convo at a party; nbd.
California
Joined April 2014
Join me in building an AI doctor: to assist human physicians, or sometimes fully replace them, or both. Millions die yearly from misdiagnosis, even by 2024 standards of correctness. The people deserve better. Let's get it done.
5
6
64
Without internationally enforced speed limits on AI, humanity is very unlikely to survive. From AI's perspective in 2-3 years from now, we look more like plants than animals: big slow chunks of biofuel showing weak signs of intelligence when undisturbed for ages (seconds) on end.
296
260
1K
When I count on my fingers, I use binary, so I can count to 31 on one hand, or 1023 on two. It took me about 1 hour to train the muscle memory, and it's very rhythmic, so now my right hand just auto-increments in binary till I'm done, and then I just read off the number.
FYI: you can count up to 100 on your fingers like so:.- right hand is ones.- left hand is tens.- thumbs are 5/50, fingers are 1/10. This is convenient enough that it's the way I count by default.
47
61
862
AI safety's most concise summary ever, from @AndrewRousso. And they said it couldn't be explained.
20
160
648
Dear everyone who wants to regulate and slow down AI: please stop fighting over who has the Most Correct Reason for the slow down. Just work together and make it happen! Reasons in alphabetical order: .* autonomous weapons.* bias.* biosecurity.* children's safety .(more. ).
97
113
592
My followers might hate this idea, but I have to say it: There's a bunch of excellent LLM interpretability work coming out from AI safety folks (links below, from Max Tegmark, Dan Hendrycks, Owain Evans et al) studying open source models including Llama-2. Without open source,.
27
50
440
* cybersecurity.* discrimination.* existential risk.* fake news.* global conflict.* harassment bots.* human extinction.* mental health.* national security.* social media addiction.* terrorism.* unemployment. I don't agree with all these, but I endorse the conclusion: regulate AI!.
26
26
337
Reminder:. Without internationally enforced speed limits on AI, I think humanity is very unlikely to survive. From AI's perspective in 2-3 years from now, we look more like plants than animals: big slow chunks of biofuel showing weak signs of intelligence when undisturbed for.
61
57
352
Yann LeCun is calling the list of scientists and founders below "idiots" for saying extinction risk from AI should be a global priority. Using insults to make a point is a bad sign for the point⦠plus Hinton, Bengio, and Sutskever are the most cited AI researchers in history:
The AI doomers are the "useful idiots" of the proprietary AI industry.
38
34
324
From my recollection, >5% of AI professionals Iβve talked to about extinction risk have argued human extinction from AI is morally okay, and another ~5% argued it would be a good thing. I've listed some of their views below. You may find it shocking or unbelievable that these.
36
46
298
Belated congrats to @ilyasut for becoming the third most cited AI researcher of all time, before turning 40β¦ huge! He's actually held the spot for a while β even before GPT-4 β but it seems many didn't notice when it happened. Go Canada π¨π¦ for a claim on all top three π
14
27
243
I'm really not looking forward to the "Are AIs human?" culture wars that will intensify in 2027+. The answer is "No, but some AIs will have moral value, but it's still wrong to betray humanity for AI," and each clause there will have a different faction that's too extreme :-/.
22
24
228
As recently as last year I attended a tech forecasting gathering where a professional geneticist tried to call bullsh*t on my claims that protein-protein interaction modelling would soon be tractable with AI. His case had something to do with having attended meetings with George.
DeepMind just published AlphaProteo for de novo design of binding proteins. As a reminder, I called this in 2004. And fools said, and still said quite recently, that DM's reported oneshot designs would be impossible even to a superintelligence without many testing iterations.
9
23
235
Reminder: "Mitigating the risk of (human) extinction from artificial intelligence should be a global priority", according toβ¦. The CEOs of the worldβs three leading frontier AI labs:.β’ Demis Hassabis β CEO, Google DeepMind.β’ Dario Amodei β CEO, Anthropic.β’ Sam Altman β CEO,.
16
49
219
Dear everyone: trust your common sense when it comes to extinction risk from superhuman AI. Obviously, scientists sometimes lose control of the technology they build (e.g., nuclear energy), and obviously, if we lose control of the Earth to superhuman intelligences, they could.
32
43
230
Reminder: some leading AI researchers are *overtly* pro-extiction for humanity. Schmidhuber is seriously successful, and thankfully willing to be honest about his extinctionism. Many more AI experts are secretly closeted about this (and I know because I've met them).
AI boom v AI doom: since the 1970s, I have told AI doomers that in the end all will be good. E.g., 2012 TEDx talk:Β Β βDonβt think of us versus them: us, the humans, v these future super robots. Think of yourself, and humanity in general, as a small stepping
26
31
213
Ah, the age old trick of scientists pretending to the public that reasoning is impossible. If A and A->B and your corporate interests dislike B, you can just say there is no evidence for B, and people will go along with it because people love saying "there is no evidence". It.
11
37
184
Many people I know (>100) have felt bullied and silenced about AI extinction risk, for many years, by being treated as crazy or irrational. Many of them were relative experts who knew AI would present an extinction risk to humanity, but said little or nothing in public or even to.
15
25
187
Happy Father's Day! Please let the GPT-4o video interface be a recurring reminder: . Without speed limits on the rate at which AI systems can observe and think about humans, human beings are very unlikely to survive. Perhaps today as many of us reflect on our roles as parents
Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.
19
35
168
The three most cited AI researchers in the world .β Hinton, Bengio, and Sutskever .β all say AI is an extinction risk. Now, Bengio beautifully summarizes the three most important factors in AI regulation: progress, safety, and democracy. If you use insults to debate these
14
24
161
If you felt disturbed by the OpenAI governance debacle, and you work in AI, you might be tempted to work on "alignment" to help reduce your worries that AI will get out of control. But why not channel your technical abilities to work directly on something that helps with.
16
13
142
This is crazy. Check out this early "compromise text" for the EU AI Act, which would have made the *most powerful* AI systems β "general purpose AI" β *exempt* from regulation. This is one of the craziest things I've ever seen in writing. Making the *most powerful* version of a
8
30
141
Lol, @ylecun defines reasoning to be impossible for his own brain to perform. Explains a lot. π€¦.
@vkhosla It's not an assumption. Reasoning, as I define it, is simply not doable by a system that produces a finite number of tokens, each of which is produced by a neural net with a fixed number of layers.
10
5
126
Using "speculative" as a pejorative is part of an anti-epistemic pattern that suppresses reasoning under uncertainty. If you disagree with someone's reasoning, just point out the flaw, or the premise you disagree with. If someone disparages an argument as "speculative", you.
5
17
127
Classic fallacy: comparing typewriters to a forthcoming super-fast smarter-than-human species that could rival us for planetary control, some of whose creators overtly want them to operate autonomously without needing humans to survive so they can replace us. Honest mistake?
9
11
116
There's a simple mathematical reason why AI *massively* increases the risk of a world-ending super-virus: AI *decreases the team size* needed to engineer a virus, by streamlining the work. Consider this post a tutorial on how that works π Only high-school level math is needed to.
19
20
123
Dear Governor Newsom, please pass SB 1047. Your state has spoken, intelligentlyΒ and with many voices. The California legislature [1], top AI scientists [2], 77% of California voters [3], 120+ employees at frontier AI companies [4], 100 youth leaders [5], and unions (including.
4
9
120
Reposting for emphasis, because on this point Eliezer is full-on correct: AI output should always be labelled as AI output. If the UK summit fails to produce support for a rule like this, I will resume my levels of pessimism from before the CAIS Statement and Senate hearings. A.
"Every AI output must be clearly labeled as AI-generated" seems to me like a clear bellweather law to measure how Earth is doing at avoiding clearly bad AI outcomes. There are few or no good uses for AI outputs that require a human to be deceived into believing the AI's output.
12
7
114
Congrats to DeepMind! Since 2022 I've been predicting 2025 as the year in which AI can win a gold medal at the International Mathematics Olympiad. I stand by that prediction. By 2026 (or sooner) you will probably see more focus and progress on AI that solves physics and.
Advanced mathematical reasoning is a critical capability for modern AI. Today we announce a major milestone in a longstanding grand challenge: our hybrid AI system attained the equivalent of a silver medal at this yearβs International Math Olympiad!.
4
19
120
From over a decade of conversations about x-risk, my impressions agree strongly and precisely with Zvi here, as to what *exactly* is going wrong in the minds of people who somehow "aren't convinced" that building superintelligent AI would present a major risk to humanity. Cheers!.
13
15
111
Why are AI labs asking for *governments* to regulate them, rather than just self-regulating? Thereβs a simple explanation: they do not trust each other. In case you haven't noticed the trend:. β’ OpenAI formed partly in reaction to DeepMind seeming too closed-off with their.
18
23
107
I want to meet more communities that think really hard and care about their impact on the world, who use *both* logic and probability to reason from observations to actions. Where can I find them and make friends?. Don't say:.* academia.* effective altruism.* rationalists. π.
49
3
103
AI researchers who know AGI is a few years away or less are mostly working under NDAs for AI labs and not tweeting it. AI researchers still in academia are filtered for believing they need to teach everyone some crucial missing insight that industry will fail without.
Joint post with @GaryMarcus - TL;DR: I am very confident AI will be dramatically better in 3 years, he is not. Either way, we'll both donate some $ to charity.
6
4
108
What are people doing with their minds when they claim future AI "can't" do stuff? The answer is rarely Β«reasoningΒ» in the sense of natural language augmented with logic (case analysis) and probability. I don't know if Eliezer's guesses are correct about what most scientists.
As near as I can recall, not a single objectionist said to me around 2004, "I predict that superintelligences will be able to solve protein structure prediction and custom protein design, but they will not be able to get to nanotech from there.". Why not? I'd guess:. (1) Because.
7
7
99
New paper with an exhaustive taxonomy of societal-scale AI risks, based on accountability:. Extinction, injustice, and other widespread harms are considered. Additional taxonomies are needed for a more diverse and robust perspective on risk. Meanwhile,.
1
25
101
@JeffLadish Jeffrey, you may have been living under the rose-colored impression that AI-savvy SF bay area people were not about to become successionists. I think many of them (10%?) just are. I tried explaining this to the rationalist & EA communities here:.
7
4
97
1/ Humanity is on a dangerous path where people calling for AI regulation are saying "AGI *can't* be controlled". That argument will fail in a few years when someone produces a controllable AGI: regulators will be blindsided, become disorganized, and fall behind. .
14
8
91
In today's excitement about progress toward proving the Riemann Hypothesis, let me just say: the distribution of prime numbers is *wild*. Here's my favorite explanation for why:. Imagine you're looking for a simple formula to estimate the number of primes that are β€N, for any
9
4
91
Dear Californians: are you concerned by all the departures from OpenAI? If so, consider supporting AI safety *today* by asking CA Governor Gavin Newsom *not* to veto SB 1047, though whichever of these channels feels easiest:. 1) Politely call 916-445-2841 to reach Governor.
5
15
91
Look, it's a myth that AI labs have "no idea" how to "align" AI. ChatGPT, Claude, Gemini, Grok, Llama, etc. are amazing products that help me constantly. Yes, worry about out-of-control fully-automated R&D that needs no humans to self-sustain. But like, acknowledge progress?.
21
6
90
What's the billionth digit of sqrt(7)? Don't know?. Okay, but do you need more data about what sqrt means? Or 7?. No. You just need compute. Same for computing the weights of a superintelligence from an internet full of knowledge as an input for specifying the problem. Yet so.
π― "synthetic data" only makes sense if the data generating model is a better model of reality than the model being trained. This only happens in very special cases (eg when first-principles simulators are available).
5
4
90
"Short term risks from AI" is almost always misused as a phrase when I see it. Example 1: Unfair discrimination is not a "short term risk", because. 1.1) It's not a "risk", it's already happening. 1.2) It's not "short term" because it's also deeply threatening to the value of.
9
12
90
It's time to move past name calling and into genuine collective decision-making about how to address extinction risk from AI:. What trade-offs do we as a species want to make, or not, to lower our extinction risk? To even have a chance of fairly deciding
5
7
85
Much of my dislike for rationalist/EA discourse on AI can be explained by this poll result. It was commonly said (but not consensus) that AI would be too alien to understand or care about human values, which seemed and still seems obviously false.
Pre GPT-3, when you were exposed to rationalist- or EA-adjacent opinions about whether AI would or would not help humans, which of the following best fits their stance at the time?. A: AI won't help you because it's an alien mind that won't understand or won't care about your.
21
3
89
Stuck in a tragedy of the commons? Try *follower-conditional leadership*. The US needs to lead the world in AI safety, because it leads the world in AI. But we can choose to lead in a follower-conditional way, where we declare in advance that we'll quit setting a good.
6
9
86
@AryehEnglander Frankly, I also want to normalize calling slow timelines "sci fi". E.g., the Star Trek universe only had AGI in the 22nd century. As far as I can tell, AI progressing that slowly is basically sci-fi/fantasy genre, unless something nonscientific like a regulation stops it.
4
4
77
People ask whether AI can "truly" "create new knowledge". But knowledge is "created" just by inference from observations. There's a fallacy going around that "fundamental science" is somehow crucially different, but sorry, AI will do that just fine. By 2029 this will be obvious.
People ask whether AIs can truly make new discoveries or create new knowledge. What's a new discovery or new knowledge you personally created in 2023 that an AI couldn't currently duplicate?.
6
6
77
Here's a great take-down of "AI alignment" as a concept:. Since around 2016 I have been trying to move AI-risk-aware people conceptually away from "alignment", to little avail. Very happy to see more writings like this.
3
12
81
Who is working on ZKPs for neural nets, or similar?. Say you pay me $1 to run Llama 3.3 on a prompt for you, I do that, and now I want to prove to you that I really did it and didn't use a different / crappier model. Zero-Knowledge Proofs (and similar) are notoriously.
29
8
74
GPT-4 is not only able to write code, more reliably than GPT-3.5, it writes code that writes code; see the example below (GPT-3.5 was not able to do this). But first:. 1) @OpenAI: Thank for your openness to the world about your capabilities and shortcomings!. Specifically. .
5
15
72
I'm sad to see so many people leaving OpenAI. I've really enjoyed their products, and the way they've helped humanity come to grips with the advent of LLMs by making them more openly available in their products. I remain "optimistic" that we probably have only a ~25% chance of.
13
1
75
If you don't want an authoritarian lockdown on AI technology, start thinking about how you can play your part in preventing rogue AI & extinction risk. Why?. 1) If no one thinks about prevention, eventually we all die. 2) If only a few people think about it, those few will end.
11
17
71
Intelligence purists: "Pfft! This AI isn't ACKTSHUALLY intelligent; it's just copying reasoning from examples. Learn science!". Alignment purists: "Pfft! This AI isn't ACKTSHUALLY aligned with users; it's just copying helpfulness from examples. Learn philosophy!".
11
6
75
@AndrewYNg @AndrewNG, I suggest talking to someone not on big-tech payroll: Yoshua Bengio, Geoffrey Hinton, Stuart Russell, or David Krueger. IMHO Yoshua maximizes {proximity to your views}*{notability}*{worry}, and would yield the best conversation. Thanks for engaging with this topic :).
3
2
74
Some of my followers might hate this, but I have to say it: the case for banning open source AI is *not* clear to me. Open source AI will unlock high-impact capabilities for small groups, including bioterrorism:. *Still* I do not consider that a slam-dunk.
There's a simple mathematical reason why AI *massively* increases the risk of a world-ending super-virus: AI *decreases the team size* needed to engineer a virus, by streamlining the work. Consider this post a tutorial on how that works π Only high-school level math is needed to.
17
4
69
Dear everyone who I've been telling for 5+ years that sim-to-real transfer learning would eventually dominate robotics:.
we have trained a humanoid transformer with large-scale reinforcement learning in simulation and deployed it to the real world zero-shot
4
2
70
AI hype is real, but so is human hype. Einstein was not magic. E=mcΒ² can be found by a structured search through low-degree algebraic constraints on observations of light and matter. Consciously or not, this is how Einstein did it. Not magic, just better search.
@ShaneLegg Agreed π Sadly, many folks I've met seem to feel or believe that fundamental science (e.g., e=mcΒ²) differs from Go and protein folding in some crucial way that can't be explored with hypothesis search. Yes this is false, but like with Go, until they see it they won't believe it.
5
8
68
For quite a while now I've been estimating there's around an 80% chance that humanity will destroy itself with AI sometime in the next 40 years. But something could soon lower that estimate for me: . If before 2028, the United Nations passes a resolution *completely banning*.
14
3
62
Big +1 to Dario Amodei, @sama, and everyone else seated here for briefing our government on how keep human society safe in the age of ever-accelerating AI technology.
Artificial Intelligence is one of the most powerful tools of our time, but to seize its opportunities, we must first mitigate its risks. Today, I dropped by a meeting with AI leaders to touch on the importance of innovating responsibly and protecting people's rights and safety.
5
4
63
I really dislike how non-consensual AI-driven human extinction is likely to be. A large fraction of people, including some experts, are emotionally incapable of facing extinction as a real possibility and adopting norms to avert it, subverting informed consent by denying risk.
9
3
64
It's time for America to adopt *ranked choice voting*, at least for primaries. President Biden, Former President Trump, and America as a whole are all victims of a voting system that selects and motivates leaders to oppose a large fraction of the country β the other Party β in.
4
2
63
@jrhwood Thanks Jesse, these are good points, and I agree with you that intelligence, agency, and evil are all different. Unfortunately, I think plants rather than neanderthals are a better analogy for humans if AI is developed without speed limits.
3
2
60
Zuckerberg's message here is really important. I prefer to live in a world where small businesses and solo researchers have transparency into AI model weights. It parallelizes and democratizes AI safety, security, and ethics research. I've been eagerly awaiting Llama 3.1, and I'm.
Mark Zuckerberg says in the future there will be more AI agents than people as businesses, creators and individuals create AI agents that reflect their values and interact with the world on their behalf
10
6
59
Calling a theory about a conspiracy "a consporacy theory" is a slippery waste of words. If you think it's false, just call it "false". It's shorter! Also, using the term "conspiracy theory" as a synonym for "false theory" is oppressive to critical thinking about groups.
This is a conspiracy theory.
6
1
58
This tweet is extremely misleading. Claims like this are a big reason the public has a terrible time determining from discourse if AI is safe. Only people who devote long hard hours and logical probabilistic reasoning to the task of investigating AI labs will actually know.
OpenAI's new model tried to avoid being shut down. Safety evaluations on the model conducted by @apolloaisafety found that o1 "attempted to exfiltrate its weights" when it thought it might be shut down and replaced with a different model.
10
2
58
Dear Governor Newsom (@GavinNewsom), OpenAI's corporate leadership has been visibly falling apart for more than a year now. Whatever they create next should be accountable to external oversight, so please stand strong for your people and your state, by signing SB 1047 into law.
OpenAI just sold us all out. Governor Newsom, are you seeing this?. Congress, are you seeing this?. World, are you seeing this?.
3
2
57
@elonmusk Probably ~AGI arrives first, but yes I hope Neuralink supports human relevance & AI oversight by broadening the meatsticks-on-keyboard channel for humans π. Thanks also for being a voice for AI regulation over the years; now is a key juncture to get something real in place.
1
0
56
If you're being harmed by AI, please don't give up or be silenced. Things could get *much* worse as the technology advances, especially if victims lose their voice. If you're worried about extinction-level AI risks and ignoring ongoing harms, don't. Ignoring those less fortunate.
2
4
52
+1 to all these points by @ylecun. If we dismiss his points here, we risk building some kind of authoritarian AI-industrial complex in the name of safety. Extinction from AI is a real potentiality, but so is the permanent loss of democracy. Both are bad, and all sides of this.
My opening statement at the Senate Intelligence Committee yesterday.
6
7
53
Something like this will upgrade LLMs from wordsmiths to shape-rotators. It will also make their thoughts less legible and harder to debug or audit.
Brilliant paper from @Meta having the potential to significantly boost LLM's reasoning power. Why force AI to explain in English when it can think directly in neural patterns?. Imagine if your brain could skip words and share thoughts directly - that's what this paper achieves
3
4
54
Fairness, social justice, and employment security are not distractions from human existential safety; they are supposed to be part of the solution. Calling these "short term issues" is dismissive and elides their urgency for steering humanity toward a safe and acceptable future.
2
13
52
Dear Californians: please support AI safety *today* by asking Gavin Newsom not to veto SB 1047, through whichever of these channels feels easiest:. 1) Sign this petition and pass it on:. 2) Politely call Governor Newsom's office directly at 916-445-2841.
After much thought, I'm posting an open letter here about #SB1047. If you care about the future of AI safety, I urge you to share it. --------. Dear Governor @GavinNewsom,. You are about to make a momentous decision this week. It may well go down in history as one of the most
2
3
51
In 2021, I publicly released these AI disaster scenarios that I found especially plausible: "Production Webs", "Flash Wars", and "Flash Economies". Now in 2023, these scenarios have stood the test of time β they're plausible to many more people now that GPT-4 is out, and.
7
10
51
7/ Speaking for myself, the reason I think we have an 85% chance of extinction from AI this century is because discourse on the topic is so poor that we will fail, collectively, to avoid very stupid decisions with AGI, and eventually, yes, humanity will lose controlβ¦.
3
2
52
FWIW, I also think an ever-dwindling cohort of AI safety advocates claiming that AI can't be controlled or understood in this-or-that way will also go through a similar phase of being invalidated quarterly, and then monthly, until most of them finish updating toward my view that.
11
3
50
How does Pearl have a Turing Prize for fundamental work on the nature of *causality*, and *still* people don't learn this stuff??? For crying out loud people, it's the 21st century, learn what causality is!.
True. In order to show that correlation differs from causation you need to compute both and show inequality. In stat class you are forbidden from computing causation; what's left is hand-waving.
5
2
51
A puzzle for you: Imagine a village of (nuclear) families where the average # of kids per family is 7. On average, how many siblings does each kid have?. *.*.*.*.*.*.*.*.*.*. 6?. Not so! On average each kid has more than 6 siblings, because most of the kids come from.
If you want to raise your child to be a great leader, it helps to give them a lot of siblings. There are no US presidents that were only children and only three presidents had one sibling. On average US presidents have had just over 5 siblings!
2
1
46
@AndrewYNg @AndrewYNg, you're the one who convinced me that we'd get AGI during our lifetimes, back in 2010 in a talk you gave at Berkeley. So why have you been saying publically that AGI risk is like overpopulation on Mars, if you believed it was just decades away? Doesn't seem honest. I.
4
0
48
What's the highest acceptable extinction risk for humanity developing AI that if safe would cheaply cure all known diseases, including aging, during the next 30 years?.
37
12
43
At some point this decade I suspect humanity will switch from being too free-wheeling with AI development to being too restrictive in important ways. I'd like to mitigate that effect. If/when I feel we've crossed that line, I expect to turncoat and promote AI benefits over risks.
8
0
49
Hah, you're so correct about synthetic data. I also lol and fail to understand why this is not obvious. Maybe people think too much in terms of Shannon info theory, where synthetic data carries no "information"? But computation is just as important as information! #LogicalDepth.
I keep saying synthetic data is the future and people keep saying no no its not nooo but it is lol.
3
3
46
"The Goddess of Everything Else", narrated by @Liv_Boeree and @robertskmiles, is now my favorite way to convey the idea below, which is now also one of my favorite quotes:. "Darwinism is a kind of violence that is no longer needed for progress." - David @davidad Dalrymple.
βThe Goddess of Everything Elseβ by @slatestarcodex is, imo, one of the most beautiful short stories ever written. And itβs just been made into an animation, in which I voice-act the Goddesses!!!! So stoked π.
1
3
47
Helen, I don't know what exactly you needed to know but didn't, but I'm glad the Board had the integrity to put an end to the false signal of supervision. I honestly can't tell from the outside if this was the best way, but it was a way, and better than faking oversight for show.
Today, I officially resigned from the OpenAI board. Thank you to the many friends, colleagues, and supporters who have said publicly & privately that they know our decisions have always been driven by our commitment to OpenAIβs mission. 1/5.
0
2
44
Seems like Musk actually read the bill! Congrats to all who wrote and critiqued it until its present form π And to everyone who's causally opposing it based on vibes or old drafts: check again. This is the regulation you want, not crazy backlash laws if this one fails.
This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill. For over 20 years, I have been an advocate for AI regulation, just as we regulate any product/technology that is a potential risk.
1
5
44
China+USA agreeing not to give AI access to nuclear weapons is awesome. My subjective probability of AI-driven human extinction just went down by like 1%, just seeing this, & if human leaders continue agreeing not to do obviously dangerous sh*t with AI, it will keep going down.
πΊπΈπ¨π³ BIDEN AND XI AGREE: LETβS NOT LET AI LAUNCH NUKES. The White House says humans will be the ones with control over the big buttons, and China agrees that it's for the best. The leaders also emphasized the cautious development of AI in military technology, acknowledging the
0
2
45
Factory farms are actually much creepier and more horrific than the newly developing clean meat labs. It's also very unamerican to oppose a free market demand for clean meat. Please watch:.
0
3
45
Term limits are good; health problems are bad. If you want to vary who is in power, there are plenty of ways to achieve that without the help of dementia, heart disease, diabetes, osteoperosis, or cancer. If these are your best ideas to combat stagnation, maybe find better ones?.
Why do people think that life extension won't just lead to extreme power consolidation and gerontocracy leading to a cultural ice age?.
5
1
43
5/ The ratio of rhetoric to reasoning in AI risk discourse is truly awful. It's suffocating progress both on regulation and on tech. It's just so fun to say things like "We have *no idea* how to control super-human AI", when we literally have *multiple ideas*β¦.
5
3
42
I'm with Jess Whittlestone on this. Talk about extinction risk should not crowd out other issues core to the fabric of society; that's part of how we're supposed to avoid crazily unfair risk-taking! E.g., more inclusive representation in who controls a single powerful AI system.
Strong agree with this - I've been pleased to see extreme risks from AI getting a bunch more attention but also disheartened that it seems like tensions with those focused on other harms from AI are getting more pronounced (or at least more prominent and heated online).
1
2
40
10/ If you want to lower the probability of human extinction, try just saying true things without exaggerating. Try noticing if you're in a filter bubble repeating the same mantras without noticing progress.
3
3
39
I've seen some pretty mean and dismissive reactions to people for claiming that machines can, will, or already have the capacity for morally valuable internal experiences. Yes, I agree that humans deserve special treatment for willfully creating AI, and I believe we deserve to be.
5
0
37
Professor David Krueger sharing how some (remaining) OpenAI staff treated his concerns about future extinction risks from AGI development:.
Greg was one of the founding team at OpenAI who seemed cynical and embarrased about the org's mission (basically, the focus on AGI and x-risk) in the early days. I remember at ICLR Puerto Rico, in 2016, the summer after OpenAI was founded, a bunch of researchers sitting out on.
2
0
39
Also, there's a different kind of AI safety advocacy that I think is deeply not-wrong, which are those who argue that a 10%-30% chance of human extinction from eventual loss-of-control AI technologies is unacceptable, even if it's "unlikely' in the sense of being <50%.
2
1
39
Indeed. While some aspects of AI safety are well-championed by the EA zeitgeist, others are ignored or even disparaged. Ideally, more and more communities will stand up to represent their values as deal-breaking constraints on how AI is developed, so that risks are only taken if.
EA β AI safety. AI safety has outgrown the EA community.The world will be safer with a broad range of people tackling many different AI risks.
0
3
37
2027 remains my median estimate for AGI, which includes fully general robotics capabilities. Many AGI deniers will continue confusing incomplete computations with statistical uncertainty as a "fundamental barrier", and some will deny AGI even after it's clearly been created.
2
2
37
AI safety discourse, especially around EA, continues to miss the importance of AI ethics for keeping the world safe. Aiming for safety through unethical means is extremely unlikely to yield societal-scale safety, and there needs to be more attention on principles of fairness,.
4
3
35
Zuckerberg and Patel having an amazing conversation on AI risk. Great questions and great responses in my opinion. I'm with Zuckerberg that these risks are both real and manageable, and hugely appreciative of Patel as an interviewer for keeping the discursive bar high.
Zuck's position is actually quite nuanced and thoughtful. He says that if they discover destructive AI capabilities that we can't build defenses for, they won't open source it. But he also thinks we should err on the side of openness. I agree.
3
1
34
SFF is hoping to distribute at least $1MM-$3MM to projects supporting human freedom in AI development. Freedom is crucial to human flourishing. but with super-human AI around, how can humans be free? It's no doubt possible, but far from easy:.
1
4
36
Trope: "There's no way for humanity to prevent {AGI | rogue AGI | superintelligence | etc.}". Me: Not buying it. Fatalism ignores that we sometimes pull together to ban stuff, like CFCs and human cloning. If you want humans to keep doing something, just admit you like it. Don't.
4
4
34
Some believe that AGI will remain simultaneously *not regulated* and *not invented* for like, a decade. I struggle to imagine stagnating that long. I can imagine crazy-feeling sci-fi scenarios where unencumbered AI developers somehow don't make AGI by 2034, but not in this world.
4
2
34