Raphaël Millière
@raphaelmilliere
Followers
11,096
Following
2,591
Media
494
Statuses
2,589
Philosopher of Artificial Intelligence & Cog Science @Macquarie_Uni Past @Columbia @UniofOxford Also on other platforms Blog:
Sydney
Joined May 2016
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#NevaPlay
• 333898 Tweets
#MEGANxRM
• 269622 Tweets
Ravens
• 240812 Tweets
The NFL
• 239194 Tweets
SAROCHA X MODERN GURU
• 154312 Tweets
NAMJOON
• 149181 Tweets
Taylor Swift
• 110202 Tweets
Lamar
• 108230 Tweets
Linkin Park
• 106491 Tweets
Dybala
• 101050 Tweets
The Final Avatar
• 83339 Tweets
Mahomes
• 80954 Tweets
Flowers
• 75145 Tweets
Julian
• 66917 Tweets
Kansas City
• 60874 Tweets
Chester
• 55152 Tweets
Kelce
• 48059 Tweets
Xavier Worthy
• 44293 Tweets
Pacheco
• 32944 Tweets
#쯔위와_RunAway_할시간
• 30623 Tweets
Derrick Henry
• 26522 Tweets
#TZUYU_SOLO_RUNAWAYDAY
• 25542 Tweets
ABOUTZU RUN AWAY OUT NOW
• 24296 Tweets
サブスク解禁
• 21091 Tweets
Mark Andrews
• 20935 Tweets
#BALvsKC
• 14018 Tweets
Isaiah Likely
• 13692 Tweets
Rashee Rice
• 13213 Tweets
モモンガ
• 13195 Tweets
ちいかわ寿司
• 12762 Tweets
Chelsie
• 12748 Tweets
ハチワレ
• 10613 Tweets
Pinned Tweet
📄I finally preprinted this new chapter for the upcoming Oxford Handbook of the Philosophy of Linguistics, edited by the excellent Gabe Dupre,
@ryan_nefdt
& Kate Stanton. Before you get mad about the title – read on! 1/
5
22
118
I asked ChatGPT to rewrite Bohemian Rhapsody to be about the life of a postdoc, and the output was flawless:
155
2K
10K
I asked GPT-3 to write a response to the philosophical essays written about it by
@DrZimmermann
,
@rinireg
@ShannonVallor
,
@add_hawk
,
@AmandaAskell
,
@dioscuri
, David Chalmers, Carlos Montemayor, and Justin Khoo published yesterday by
@DailyNousEditor
. It's quite remarkable!
87
1K
3K
Yann LeCun kicking off the debate with a bold prediction: nobody in their right mind will use autoregressive models 5 years from now
#phildeeplearning
43
170
1K
Another day, another opinion essay about ChatGPT in the
@nytimes
. This time, Noam Chomsky and colleagues weigh in on the shortcomings of language models. Unfortunately, this is not the nuanced discussion one could have hoped for. 🧵 1/
34
240
1K
The release of impressive new deep learning models in the past few weeks, notably
#dalle2
from
@OpenAI
and
#PaLM
from
@GoogleAI
, has prompted a heated discussion of
@GaryMarcus
's claim that DL is "hitting a wall". Here are some thoughts on the controversy du jour. 🧵 1/25
27
201
1K
There's a lot of speculation about whether OpenAI's video generation model Sora has a 'physics engine' (bolstered by OAI's own claims about 'world simulation'). Like the debate about world models in LLMs, this question is both genuinely interesting and somewhat ill-defined. 🧵1/
30
164
949
More experiments with semantic guidance of a disentangled generative adversarial network for a paper in preparation. Here we start from a GAN-inverted picture of Bertrand Russell, and modify it in various ways using text prompts and a
#CLIP
-based loss.
7
126
580
One was really sassy: "I'll admit that my ideas are largely untested. I haven't spent years in academia toiling away at some low-paying job that I don't really enjoy just so that I can eventually get a job doing something that I don't really want to be doing in the first place."
6
42
516
Happy to share that the recordings of the
#phildeeplearning
conference are now available! Head to to find individual links in the program, or to for the whole Youtube playlist.
Very excited to share the final line-up and program of our upcoming conference on the Philosophy of Deep Learning!
Co-organized with
@davidchalmers42
&
@De_dicto
and co-sponsored by
@columbiacss
's PSSN &
@nyuconscious
.
Info, registration & full program:
23
112
349
10
124
450
Figure taken from a paper just published in
@NatureComms
. This is why deep learning researchers need to engage with cognitive science.
17
89
423
The recent update to the Midjourney beta with Stable Diffusion is pretty impressive, and arguably has the edge over DALL-E 2 for the generation of striking imagery - even though it's more stylistically opinionated.
11
57
411
I don't think lossy compression is a very helpful analogy to convey what (linguistic or multimodal) generative models do – at least if "blurry JPEGs" is the leading metaphor. It might work in a loose sense, but it doesn't tell the whole story. 1/
22
71
409
Very excited to share the final line-up and program of our upcoming conference on the Philosophy of Deep Learning!
Co-organized with
@davidchalmers42
&
@De_dicto
and co-sponsored by
@columbiacss
's PSSN &
@nyuconscious
.
Info, registration & full program:
23
112
349
📝New preprint! What does it take for AI models to have grounded representations of lexical items? There is a lot of disagreement – some verbal, some substantive – about what grounding involves. Dimitri Mollo and I frame this old question in a new light 1/
15
84
339
📄Now preprinted - Part I of a two-part philosophical introduction to language models co-authored with
@cameronjbuckner
! This first paper offers a primer on language models and an opinionated survey of their relevance to classic philosophical issues. 1/5
4
59
279
The recordings of our event "The Challenge of Compositionality for AI" are now available on !
@GaryMarcus
3
64
257
Can you reliably get image generation models like DALL-E 2 to illustrate specific visual concepts using made-up words? In this new preprint, I show that you can, using new approaches for text-based adversarial attacks on image generation. 1/12
10
61
250
Reminder: we invite high-quality submissions for the first issue of PHILOSOPHY AND THE MIND SCIENCES, a new peer-reviewed, fee-less, open-access journal dedicated to research at the interface of philosophy of mind, psychology and cognitive neuroscience. Please retweet! (thread)
9
161
242
In this new Vox piece written with
@CRathkopf
, we ask why people – experts included – are so polarized about the kinds of psychological capacities they ascribe to language models like ChatGPT, and how we can move beyond simple dichotomies in this debate.
12
41
243
PSA for LaTeX users: GPT-4 almost makes TikZ usable (almost).
Behold, this TikZ diagram of a simple perceptron with three input nodes, generated zero-shot:
7
14
241
Excited to announce this two-day online workshop on compositionality and AI co-organized with
@GaryMarcus
with a stellar line-up of speakers! Program (TBC) & registration: 1/
7
63
236
Following
@keithfrankish
's tongue-in-cheek suggestion, I prompted GPT-3 to write a short philosophy paper. The result puts its strengths & weaknesses on display. The mimicry of argumentation is uncanny, if very flawed, and often unintentionally funny.
22
80
222
With all the recent brouhaha about DALL-E 2, I somehow missed the release of the VQGAN-CLIP paper by
@RiversHaveWings
,
@BlancheMinerva
, and others. This will probably go down as the open source algorithm that democratized artistic uses of computer vision:
3
30
206
OpenAI unveiled its video generation model Sora two weeks ago. The technical report emphatically suggests that video generation models like Sora are world simulators. Are they? What does that even mean? I'm taking a deep dive into these questions in a new blog post (link below).
8
54
207
Are large pre-trained models nothing more than stochastic parrots? Is scaling them all we need to bridge the gap between humans and machines? In this new opinion piece for
@NautilusMag
, I argue that the answer lies somewhere in between. 1/14
12
55
197
📄Now preprinted - Part II of our philosophical introduction to language models! While Part I focused on continuity w/ classical debates, Part II is more forward-looking and cover new issues. 1/5
📄Now preprinted - Part I of a two-part philosophical introduction to language models co-authored with
@cameronjbuckner
! This first paper offers a primer on language models and an opinionated survey of their relevance to classic philosophical issues. 1/5
4
59
279
5
35
199
The prompt contained the essays themselves, plus a blurb explaining that GPT-3 had to respond to them. Full disclosure: I produced a few outputs and cherry-picked this one, although they were all interesting in their own way.
3
11
189
@davidchalmers42
AFAIK the closest might be "The Unreasonable Effectiveness of Data" from Halevy, Norvig and Pereira (Google) in 2009. In hindsight it was remarkably prescient: "For many tasks, words... provide all the representational machinery we need to learn from text"
2
13
184
Is this the new "Will it run Doom?"
I've sucefully runned LLaMA 7B model on my 4GB RAM Raspberry Pi 4. It's super slow about 10sec/token. But it looks we can run powerful cognitive pipelines on a cheap hardware.
116
644
5K
3
19
178
Given the current state of AI Twitter let's hope it doesn't start colliding with panpsychist Twitter
11
12
171
Officially done with my PhD, and excited to share that I've accepted a postdoctoral fellowship
@Columbia
to work on spatial self-representation in philosophy & neuroscience! More to follow after I get started in July...
15
2
165
Extremely impressive – LLaMa 65B running on a laptop CPU in pure C++ with 4-bit quantization! Between this and
@togethercompute
OpenChatKit announced earlier today, ChatGPT-level chatbots running locally on consumer hardware might be within reach much sooner that I thought...
4
14
154
I've seen some questions about how I could produce the texts I shared earlier by prompting GPT-3, and whether GPT-3 is capable of producing such a convincing output at all, so here's a thread to clarify a few points.
1
49
145
It's become increasingly clear over the past few weeks that the Overton window for "scaling maximalism" – the claim that scaling existing approaches is all we need to build AGI – has shifted, at least in the industry. Some thoughts on this 🧵 1/
8
21
136
I've been following this project with great interest since it started. The latest: RNNs up to 14B params are competitive with Transformers. Very curious to see whether we'll find similar emergent capabilities with further scaling, perhaps knocking self-attention off its pedestal.
#RWKV
is One Dev's Journey to Dethrone GPT Transformers. The largest RNN ever (up to 14B). Parallelizable. Faster inference & training. Supports INT8/4. No KV cache. 3 years of hard work. DEMO: Computation sponsored by
@StabilityAI
@AiEleuther
@EMostaque
46
348
2K
2
19
135
For a start I'm not sure the melodramatic tone serves the argument: "machine learning will degrade our science and debase our ethics", and "we can only laugh or cry at [LLM's] popularity"! I know op-eds are often editorialized for dramatic effect, but maybe this is a bit much? 2/
2
2
129
I wish there were more nuance in public discussions of LLMs (whether they're dismissed or praised). There are many important ethical concerns to be addressed, but sweeping statements such as "ChatGPT exhibits the banality of evil" aren't very helpful to get at those. 17/17
13
6
126
I've seen a lot of discussion about the notion of artificial general intelligence (AGI) lately – what it means, if anything. Like many debates regarding AI, this is a topic that invites verbal disputes if people don't have the same definitions in mind. 1/18
7
21
122
Contrary to what Chomsky claims, we could learn a lot about language acquisition from in silico experiments with model learners, if the learning parameters and environment are set up to allow meaningful inferences. 5/
7
9
123
A final amusing detail - GPT-4 gets a score of 86% on the theory and knowledge part of the Certified Sommelier Examination. Question for a philosophy class: Can one be a wine expert without having ever tasted wine? 8/8
5
24
122
But we were surprised to see that
@GoogleAI
's new language model, PaLM, achieved excellent results on our task in the 5-shot learning regime, virtually matching the human avg. This shows a sophisticated ability to combine the word meanings in semantically plausible ways. 14/25
2
14
116
New paper (in open access): "Philosophy of cognitive science in the age of deep learning" – in which I argue that although progress in DL is largely driven by engineering goals, it is far from irrelevant to (the philosophy of) cog sci, and vice versa.
5
29
115
Golden Gate Claude's reductio ad pontem
(Context: )
5
18
115
Program now live!
June 29 – Why Compositionality Matters for AI
w/
@AllysonEttinger
,
@paul_smolensky
,
@GaryMarcus
& myself
June 30 – Can Language Models Handle Compositionality?
w/
@_dieuwke_
,
@tallinzen
,
@elliepavlick
,
@scychan_brains
&
@LakeBrenden
Excited to announce this two-day online workshop on compositionality and AI co-organized with
@GaryMarcus
with a stellar line-up of speakers! Program (TBC) & registration: 1/
7
63
236
4
28
110
PDF of the response here:
2
12
106
Strikingly, the actual examples of supposed failures of LLMs that are given in the article are either purely speculative or inconclusive. Let's look at the apple example. Here are ChatGPT,
@Anthropic
's Claude and Bing Chat giving fine answers containing explanations. 10/
5
2
104
Since this is blowing up and a lot of people are questioning whether it's real, I recorded my screen with another attempt showing the prompt and several outputs (bonus alternative version at the end).
I asked ChatGPT to rewrite Bohemian Rhapsody to be about the life of a postdoc, and the output was flawless:
155
2K
10K
2
13
97
Of course it's widely unlikely that Sora literally makes function calls to an external physics engine like UE5 during inference. Note that this has been done before with LLMs, see this Google paper where the model answers questions through simulations with a physics engine. 2/
1
4
94
Let's look at this other example. The authors assert that "the predictions of machine learning systems will always be superficial and dubious", and give once again an imaginary failure case that current chatbots easily avoid. 12/
6
5
92
The bottom line is that we need to go beyond behavioral evidence to settle this kind of debate, and we need to be more specific about what we mean by 'world simulation' beyond buzzwords. This is fertile ground for research at the crossroads between ML, cogsci and philosophy! /end
5
7
86
Convoluted prompt engineering strategies for SOTA one-upmanship on LLM benchmarks are getting as ridiculous as the names of open weights finetunes. Soon we'll have blog posts about mixtral-8x7B-capybara-platypus-uncensored beating SOTA on MMLU by 0.01% with Medprompt++
@128
.
5
11
81
There is much to say about each of these points, and much has been said already. LLMs don't learn language like children do, but it doesn't mean they can't induce syntactic structure and (more controversially) some aspects of meaning. 4/
2
3
84
For further discussion on this topic, join the upcoming workshop on compositionality and AI I'm organizing with
@GaryMarcus
in June – free registration here: 11/11
3
6
79
The substantive claims are all too familiar: LLMs learn from co-occurrence statistics without leveraging innate structure; they describe and predict instead of doing causal inference; and they can't balance original reasoning with epistemic and moral constraints. 3/
2
3
83
The Sora tech report is scant on details, but we know it's a diffusion model w/ a ViT backbone that processes frame patches as tokens. This architecture is likely expressive enough for sophisticated internal structure to emerge with scale and diverse training data. 8/
1
10
83
I don't think it's entirely fair to say that they "fail at compositionality". They exhibit both success and failure cases. Humans do too, but DL models fail far more often. Whether the remaining gap can be bridged without symbolic/hybrid architectures is an open question. 25/25
1
3
82
Schrödinger's cat () - science thought experiments are welcome too!
3
16
77
Conveying in simple prose the nuances of what generative models can and can't do, what they are and aren't, is a high-wire act even for experts.
@mpshanahan
offers a very balanced (if slightly deflationary) treatment of this question in this preprint: 6/
5
4
80
New paper out! 'Psychedelics, meditation and self-consciousness' -- an analysis of the similar and differential effects of psychedelic drugs and meditative practices on self-consciousness. Open access:
4
31
76
But that's not what most people are speculating about. Rather, the idea is that Sora would acquire an internal model of physics during training, and make use of this internal model to generate temporally and spatially coherent videos. 3/
1
1
77
See this new thread for more details on the methodology:
I've seen some questions about how I could produce the texts I shared earlier by prompting GPT-3, and whether GPT-3 is capable of producing such a convincing output at all, so here's a thread to clarify a few points.
1
49
145
3
15
77
The pace of development with LLaMA is vertiginous. If the finetuned 7B model is remotely in the same league as GPT-3.5 (text-davinci-003), can be reproduced for $100 and run on consumer laptops, this is really a turning point for LLMs (including implications for responsible use).
LLaMA has been fine-tuned by stanford,
"We performed a blind pairwise comparison between text-davinci-003 and Alpaca 7B, and we found that these two models have very similar performance: Alpaca wins 90 versus 89 comparisons against text-davinci-003."
51
376
3K
0
9
76
This one is pretty mind-blowing. I prompted GPT-3 to imagine a conversation between itself and
@keithfrankish
(again, lines in bold were written by me; everything else is GPT-3):
3
22
76
I absolutely get the imperative to resist anthropomorphism especially in public writing about generative models – and this kind of simple analogy may seem helpful for that. But I also worry about what
@sleepinyourhat
called the dangers of underclaiming: 5/
2
3
77
📄New preprint with Sam Musker, Alex Duchnowski & Ellie Pavlick
@Brown_NLP
! We investigate how humans subjects and LLMs perform on novel analogical reasoning tasks involving semantic structure-mapping. Our findings shed light on current LLMs' abilities and limitations. 1/
2
22
73
Foot's Trolley Problem () - this one was challenging and I couldn't quite get it right
2
7
74
The BabyLM challenge is a great example of the push towards more human-like artificial language learners. Whatever approach meets this challenge (e.g., new architectures or learning objectives), I would be shocked if it didn't involve deep learning! 7/
2
4
75
Here's the pre-conference debate on "Do Language Models Need Sensory Grounding for Meaning and Understanding?" ft.
@ylecun
, Ellie Pavlick
@Brown_NLP
,
@LakeBrenden
,
@davidchalmers42
,
@Jake_Browning00
&
@glupyan
5
15
72
But Sora doesn't generate videos by simulating a bunch of possible scenarios. For example, to make the video below, it doesn't run 100 internal simulations of the collision between the glass and the table. There's no 'intuitive physics engine' in the traditional cogsci sense. 6/
2
6
69
As I understand it, the hypothesis is generally intended to target a model of intuitive physics – something that uses approximate and probability simulations to make fast and flexible inferences about entities and their dynamics in natural scenes. 5/
1
5
69
To clarify: I used a slightly modified version of StyleCLIP to generate these (). The paper in preparation is not about a new method for GAN inversion or disentanglement, but a theoretical paper on audiovisual manipulations based on deep learning.
2
4
64
In sum: even if current LLMs don't have the right inductive biases to acquire language as efficiently as children do, it doesn't mean they can't improve; and regardless of efficiency, it doesn't mean they can't in principle learn language in a different, less efficient way. 9/
5
4
70
Fantastic paper that confirms my suspicion regarding the compositional shortcomings of CLIP-based vision-language models: standard contrastive learning on image-caption pairs doesn't force models to learn compositional structure!
Excited to share our new
#ICLR2023
oral (top 2%) paper!
We study why
#VisualLanguage
#AI
(eg CLIP) acts like bag-of-words + how to fix it.
tldr: contrastive learning issues ➡️ models don't know if "horse is eating grass" or "grass is eating horse" 🤯
🧵
2
70
324
3
10
69
On the other hand, they do remarkably well on many instances that require such parsing, and on the corresponding visual translation in the case of DALL-E 2. 21/25
1
4
69
This bit about interpolation strikes me as particularly misleading. Inference on generative models involves computations that are way more complex and structured than (say) nearest neighbor pixel interpolation in image decompression. 3/
4
2
68
I certainly can't fault Ted Chiang for failing to answer in a short general audience piece questions that experts still grapple with. But I think we should be careful not to take all-encompassing metaphors about generative models, from parrots to compression, too literally. 7/
3
6
68
If we look at simpler image diffusion models, we know linear probes can decode information about scene geometry, support relations, lighting, shadows, and depth from internal activations. 9/
1
10
68
We will host a pre-conference debate on Friday, March 24th on the question: "Do Language Models Need Sensory Grounding for Meaning and Understanding?"
The debate will feature
@Jake_Browning00
,
@davidchalmers42
,
@LakeBrenden
,
@ylecun
,
@glupyan
& Ellie Pavlick (
@BrownCSDept
).
6
13
64
How will AI transform art, if at all? This question can be approached from a few different angles. In this new piece for
@Wired
, I argue that AI art is expanding the multifaceted notion of curation. 1/8
3
20
66
