Raphaël Millière
@raphaelmilliere
Followers
11K
Following
8K
Media
495
Statuses
3K
Philosopher of Artificial Intelligence & Cog Science @Macquarie_Uni Past @Columbia @UniofOxford Also on other platforms Blog: https://t.co/2hJjfShFfr
Sydney
Joined May 2016
My article 'Constitutive Self-Consciousness' is now published online in the Australasian Journal of Philosophy. It argues (spoiler alert!) against the claim that self-consciousness is constitutive of consciousness.
2
10
34
I asked ChatGPT to rewrite Bohemian Rhapsody to be about the life of a postdoc, and the output was flawless:
145
2K
9K
I asked GPT-3 to write a response to the philosophical essays written about it by @DrZimmermann, @rinireg @ShannonVallor, @add_hawk, @AmandaAskell, @dioscuri, David Chalmers, Carlos Montemayor, and Justin Khoo published yesterday by @DailyNousEditor. It's quite remarkable!
84
1K
2K
Yann LeCun kicking off the debate with a bold prediction: nobody in their right mind will use autoregressive models 5 years from now #phildeeplearning
42
168
1K
Another day, another opinion essay about ChatGPT in the @nytimes. This time, Noam Chomsky and colleagues weigh in on the shortcomings of language models. Unfortunately, this is not the nuanced discussion one could have hoped for. 🧵 1/.
33
231
1K
The release of impressive new deep learning models in the past few weeks, notably #dalle2 from @OpenAI and #PaLM from @GoogleAI, has prompted a heated discussion of @GaryMarcus's claim that DL is "hitting a wall". Here are some thoughts on the controversy du jour. 🧵 1/25.
26
196
1K
There's a lot of speculation about whether OpenAI's video generation model Sora has a 'physics engine' (bolstered by OAI's own claims about 'world simulation'). Like the debate about world models in LLMs, this question is both genuinely interesting and somewhat ill-defined. 🧵1/
29
157
927
More experiments with semantic guidance of a disentangled generative adversarial network for a paper in preparation. Here we start from a GAN-inverted picture of Bertrand Russell, and modify it in various ways using text prompts and a #CLIP-based loss.
7
123
562
One was really sassy: "I'll admit that my ideas are largely untested. I haven't spent years in academia toiling away at some low-paying job that I don't really enjoy just so that I can eventually get a job doing something that I don't really want to be doing in the first place.".
6
40
493
Happy to share that the recordings of the #phildeeplearning conference are now available! Head to to find individual links in the program, or to for the whole Youtube playlist.
Very excited to share the final line-up and program of our upcoming conference on the Philosophy of Deep Learning!. Co-organized with @davidchalmers42 & @De_dicto and co-sponsored by @columbiacss's PSSN & @nyuconscious. Info, registration & full program:
9
119
432
Figure taken from a paper just published in @NatureComms. This is why deep learning researchers need to engage with cognitive science.
17
87
412
The recent update to the Midjourney beta with Stable Diffusion is pretty impressive, and arguably has the edge over DALL-E 2 for the generation of striking imagery - even though it's more stylistically opinionated.
11
55
399
I don't think lossy compression is a very helpful analogy to convey what (linguistic or multimodal) generative models do – at least if "blurry JPEGs" is the leading metaphor. It might work in a loose sense, but it doesn't tell the whole story. 1/.
22
68
397
Very excited to share the final line-up and program of our upcoming conference on the Philosophy of Deep Learning!. Co-organized with @davidchalmers42 & @De_dicto and co-sponsored by @columbiacss's PSSN & @nyuconscious. Info, registration & full program:
23
107
337
📝New preprint! What does it take for AI models to have grounded representations of lexical items? There is a lot of disagreement – some verbal, some substantive – about what grounding involves. Dimitri Mollo and I frame this old question in a new light 1/.
15
82
329
📄Now preprinted - Part I of a two-part philosophical introduction to language models co-authored with @cameronjbuckner! This first paper offers a primer on language models and an opinionated survey of their relevance to classic philosophical issues. 1/5
4
57
266
The recordings of our event "The Challenge of Compositionality for AI" are now available on .@GaryMarcus
3
63
251
Can you reliably get image generation models like DALL-E 2 to illustrate specific visual concepts using made-up words? In this new preprint, I show that you can, using new approaches for text-based adversarial attacks on image generation. 1/12.
9
59
243
Reminder: we invite high-quality submissions for the first issue of PHILOSOPHY AND THE MIND SCIENCES, a new peer-reviewed, fee-less, open-access journal dedicated to research at the interface of philosophy of mind, psychology and cognitive neuroscience. Please retweet! (thread)
8
150
222
In this new Vox piece written with @CRathkopf, we ask why people – experts included – are so polarized about the kinds of psychological capacities they ascribe to language models like ChatGPT, and how we can move beyond simple dichotomies in this debate.
12
39
236
PSA for LaTeX users: GPT-4 almost makes TikZ usable (almost). Behold, this TikZ diagram of a simple perceptron with three input nodes, generated zero-shot:
7
13
234
Excited to announce this two-day online workshop on compositionality and AI co-organized with @GaryMarcus with a stellar line-up of speakers! Program (TBC) & registration: 1/
7
62
230
Following @keithfrankish's tongue-in-cheek suggestion, I prompted GPT-3 to write a short philosophy paper. The result puts its strengths & weaknesses on display. The mimicry of argumentation is uncanny, if very flawed, and often unintentionally funny.
22
74
203
With all the recent brouhaha about DALL-E 2, I somehow missed the release of the VQGAN-CLIP paper by @RiversHaveWings, @BlancheMinerva, and others. This will probably go down as the open source algorithm that democratized artistic uses of computer vision:
3
29
201
OpenAI unveiled its video generation model Sora two weeks ago. The technical report emphatically suggests that video generation models like Sora are world simulators. Are they? What does that even mean? I'm taking a deep dive into these questions in a new blog post (link below).
9
52
205
Are large pre-trained models nothing more than stochastic parrots? Is scaling them all we need to bridge the gap between humans and machines? In this new opinion piece for @NautilusMag, I argue that the answer lies somewhere in between. 1/14.
12
53
191
📄Now preprinted - Part II of our philosophical introduction to language models! While Part I focused on continuity w/ classical debates, Part II is more forward-looking and cover new issues. 1/5.
📄Now preprinted - Part I of a two-part philosophical introduction to language models co-authored with @cameronjbuckner! This first paper offers a primer on language models and an opinionated survey of their relevance to classic philosophical issues. 1/5
5
31
195
The prompt contained the essays themselves, plus a blurb explaining that GPT-3 had to respond to them. Full disclosure: I produced a few outputs and cherry-picked this one, although they were all interesting in their own way.
3
11
177
@davidchalmers42 AFAIK the closest might be "The Unreasonable Effectiveness of Data" from Halevy, Norvig and Pereira (Google) in 2009. In hindsight it was remarkably prescient: "For many tasks, words. provide all the representational machinery we need to learn from text".
2
13
179
Is this the new "Will it run Doom?".
I've sucefully runned LLaMA 7B model on my 4GB RAM Raspberry Pi 4. It's super slow about 10sec/token. But it looks we can run powerful cognitive pipelines on a cheap hardware.
3
17
175
Given the current state of AI Twitter let's hope it doesn't start colliding with panpsychist Twitter.
11
12
163
Officially done with my PhD, and excited to share that I've accepted a postdoctoral fellowship @Columbia to work on spatial self-representation in philosophy & neuroscience! More to follow after I get started in July. .
13
2
152
Extremely impressive – LLaMa 65B running on a laptop CPU in pure C++ with 4-bit quantization! Between this and @togethercompute OpenChatKit announced earlier today, ChatGPT-level chatbots running locally on consumer hardware might be within reach much sooner that I thought. .
4
12
151
I've seen some questions about how I could produce the texts I shared earlier by prompting GPT-3, and whether GPT-3 is capable of producing such a convincing output at all, so here's a thread to clarify a few points.
1
49
138
It's become increasingly clear over the past few weeks that the Overton window for "scaling maximalism" – the claim that scaling existing approaches is all we need to build AGI – has shifted, at least in the industry. Some thoughts on this 🧵 1/.
8
20
128
I've been following this project with great interest since it started. The latest: RNNs up to 14B params are competitive with Transformers. Very curious to see whether we'll find similar emergent capabilities with further scaling, perhaps knocking self-attention off its pedestal.
#RWKV is One Dev's Journey to Dethrone GPT Transformers. The largest RNN ever (up to 14B). Parallelizable. Faster inference & training. Supports INT8/4. No KV cache. 3 years of hard work. DEMO: Computation sponsored by @StabilityAI @AiEleuther @EMostaque
2
19
131
For a start I'm not sure the melodramatic tone serves the argument: "machine learning will degrade our science and debase our ethics", and "we can only laugh or cry at [LLM's] popularity"! I know op-eds are often editorialized for dramatic effect, but maybe this is a bit much? 2/.
2
2
126
I wish there were more nuance in public discussions of LLMs (whether they're dismissed or praised). There are many important ethical concerns to be addressed, but sweeping statements such as "ChatGPT exhibits the banality of evil" aren't very helpful to get at those. 17/17.
12
5
120
I've seen a lot of discussion about the notion of artificial general intelligence (AGI) lately – what it means, if anything. Like many debates regarding AI, this is a topic that invites verbal disputes if people don't have the same definitions in mind. 1/18.
7
20
118
Contrary to what Chomsky claims, we could learn a lot about language acquisition from in silico experiments with model learners, if the learning parameters and environment are set up to allow meaningful inferences. 5/.
7
9
118
A final amusing detail - GPT-4 gets a score of 86% on the theory and knowledge part of the Certified Sommelier Examination. Question for a philosophy class: Can one be a wine expert without having ever tasted wine? 8/8
4
23
112
📄I finally preprinted this new chapter for the upcoming Oxford Handbook of the Philosophy of Linguistics, edited by the excellent Gabe Dupre, @ryan_nefdt & Kate Stanton. Before you get mad about the title – read on! 1/.
5
21
117
But we were surprised to see that @GoogleAI's new language model, PaLM, achieved excellent results on our task in the 5-shot learning regime, virtually matching the human avg. This shows a sophisticated ability to combine the word meanings in semantically plausible ways. 14/25
2
14
112
New paper (in open access): "Philosophy of cognitive science in the age of deep learning" – in which I argue that although progress in DL is largely driven by engineering goals, it is far from irrelevant to (the philosophy of) cog sci, and vice versa.
5
27
111
Golden Gate Claude's reductio ad pontem. (Context:
5
18
110
Program now live!. June 29 – Why Compositionality Matters for AI .w/ @AllysonEttinger, @paul_smolensky, @GaryMarcus & myself. June 30 – Can Language Models Handle Compositionality?.w/ @_dieuwke_, @tallinzen, @elliepavlick, @scychan_brains & @LakeBrenden.
Excited to announce this two-day online workshop on compositionality and AI co-organized with @GaryMarcus with a stellar line-up of speakers! Program (TBC) & registration: 1/
3
28
106
Strikingly, the actual examples of supposed failures of LLMs that are given in the article are either purely speculative or inconclusive. Let's look at the apple example. Here are ChatGPT, @Anthropic's Claude and Bing Chat giving fine answers containing explanations. 10/
4
2
100
Since this is blowing up and a lot of people are questioning whether it's real, I recorded my screen with another attempt showing the prompt and several outputs (bonus alternative version at the end).
I asked ChatGPT to rewrite Bohemian Rhapsody to be about the life of a postdoc, and the output was flawless:
1
13
93
Of course it's widely unlikely that Sora literally makes function calls to an external physics engine like UE5 during inference. Note that this has been done before with LLMs, see this Google paper where the model answers questions through simulations with a physics engine. 2/
1
4
90
Let's look at this other example. The authors assert that "the predictions of machine learning systems will always be superficial and dubious", and give once again an imaginary failure case that current chatbots easily avoid. 12/
6
5
90
The bottom line is that we need to go beyond behavioral evidence to settle this kind of debate, and we need to be more specific about what we mean by 'world simulation' beyond buzzwords. This is fertile ground for research at the crossroads between ML, cogsci and philosophy! /end.
4
7
83
Convoluted prompt engineering strategies for SOTA one-upmanship on LLM benchmarks are getting as ridiculous as the names of open weights finetunes. Soon we'll have blog posts about mixtral-8x7B-capybara-platypus-uncensored beating SOTA on MMLU by 0.01% with Medprompt++@128.
5
10
80
There is much to say about each of these points, and much has been said already. LLMs don't learn language like children do, but it doesn't mean they can't induce syntactic structure and (more controversially) some aspects of meaning. 4/.
2
3
81
For further discussion on this topic, join the upcoming workshop on compositionality and AI I'm organizing with @GaryMarcus in June – free registration here: 11/11
3
6
75
The substantive claims are all too familiar: LLMs learn from co-occurrence statistics without leveraging innate structure; they describe and predict instead of doing causal inference; and they can't balance original reasoning with epistemic and moral constraints. 3/.
2
3
79
The Sora tech report is scant on details, but we know it's a diffusion model w/ a ViT backbone that processes frame patches as tokens. This architecture is likely expressive enough for sophisticated internal structure to emerge with scale and diverse training data. 8/
1
10
80
I don't think it's entirely fair to say that they "fail at compositionality". They exhibit both success and failure cases. Humans do too, but DL models fail far more often. Whether the remaining gap can be bridged without symbolic/hybrid architectures is an open question. 25/25.
1
3
79
Schrödinger's cat ( - science thought experiments are welcome too!
3
16
77
Conveying in simple prose the nuances of what generative models can and can't do, what they are and aren't, is a high-wire act even for experts. @mpshanahan offers a very balanced (if slightly deflationary) treatment of this question in this preprint: 6/.
4
4
78
New paper out! 'Psychedelics, meditation and self-consciousness' -- an analysis of the similar and differential effects of psychedelic drugs and meditative practices on self-consciousness. Open access:
4
30
76
But that's not what most people are speculating about. Rather, the idea is that Sora would acquire an internal model of physics during training, and make use of this internal model to generate temporally and spatially coherent videos. 3/.
1
1
74
See this new thread for more details on the methodology: .
I've seen some questions about how I could produce the texts I shared earlier by prompting GPT-3, and whether GPT-3 is capable of producing such a convincing output at all, so here's a thread to clarify a few points.
3
14
74
The pace of development with LLaMA is vertiginous. If the finetuned 7B model is remotely in the same league as GPT-3.5 (text-davinci-003), can be reproduced for $100 and run on consumer laptops, this is really a turning point for LLMs (including implications for responsible use).
LLaMA has been fine-tuned by stanford, . "We performed a blind pairwise comparison between text-davinci-003 and Alpaca 7B, and we found that these two models have very similar performance: Alpaca wins 90 versus 89 comparisons against text-davinci-003."
0
9
73
This one is pretty mind-blowing. I prompted GPT-3 to imagine a conversation between itself and @keithfrankish (again, lines in bold were written by me; everything else is GPT-3):
3
22
73
I absolutely get the imperative to resist anthropomorphism especially in public writing about generative models – and this kind of simple analogy may seem helpful for that. But I also worry about what @sleepinyourhat called the dangers of underclaiming: 5/.
2
3
76
📄New preprint with Sam Musker, Alex Duchnowski & Ellie Pavlick @Brown_NLP! We investigate how humans subjects and LLMs perform on novel analogical reasoning tasks involving semantic structure-mapping. Our findings shed light on current LLMs' abilities and limitations. 1/.
2
21
69
Foot's Trolley Problem ( - this one was challenging and I couldn't quite get it right
1
7
71
The BabyLM challenge is a great example of the push towards more human-like artificial language learners. Whatever approach meets this challenge (e.g., new architectures or learning objectives), I would be shocked if it didn't involve deep learning! 7/.
2
4
72
Here's the pre-conference debate on "Do Language Models Need Sensory Grounding for Meaning and Understanding?" ft. @ylecun, Ellie Pavlick @Brown_NLP, @LakeBrenden, @davidchalmers42, @Jake_Browning00 & @glupyan
5
14
67
But Sora doesn't generate videos by simulating a bunch of possible scenarios. For example, to make the video below, it doesn't run 100 internal simulations of the collision between the glass and the table. There's no 'intuitive physics engine' in the traditional cogsci sense. 6/
2
6
67
As I understand it, the hypothesis is generally intended to target a model of intuitive physics – something that uses approximate and probability simulations to make fast and flexible inferences about entities and their dynamics in natural scenes. 5/.
1
5
66
To clarify: I used a slightly modified version of StyleCLIP to generate these (. The paper in preparation is not about a new method for GAN inversion or disentanglement, but a theoretical paper on audiovisual manipulations based on deep learning.
2
4
64
In sum: even if current LLMs don't have the right inductive biases to acquire language as efficiently as children do, it doesn't mean they can't improve; and regardless of efficiency, it doesn't mean they can't in principle learn language in a different, less efficient way. 9/.
3
4
68
Fantastic paper that confirms my suspicion regarding the compositional shortcomings of CLIP-based vision-language models: standard contrastive learning on image-caption pairs doesn't force models to learn compositional structure!.
Excited to share our new #ICLR2023 oral (top 2%) paper!. We study why #VisualLanguage #AI (eg CLIP) acts like bag-of-words + how to fix it. tldr: contrastive learning issues ➡️ models don't know if "horse is eating grass" or "grass is eating horse" 🤯.🧵
3
9
67
On the other hand, they do remarkably well on many instances that require such parsing, and on the corresponding visual translation in the case of DALL-E 2. 21/25
1
4
66
This bit about interpolation strikes me as particularly misleading. Inference on generative models involves computations that are way more complex and structured than (say) nearest neighbor pixel interpolation in image decompression. 3/
4
1
67
I certainly can't fault Ted Chiang for failing to answer in a short general audience piece questions that experts still grapple with. But I think we should be careful not to take all-encompassing metaphors about generative models, from parrots to compression, too literally. 7/.
3
6
67
If we look at simpler image diffusion models, we know linear probes can decode information about scene geometry, support relations, lighting, shadows, and depth from internal activations. 9/.
1
10
66
We will host a pre-conference debate on Friday, March 24th on the question: "Do Language Models Need Sensory Grounding for Meaning and Understanding?". The debate will feature @Jake_Browning00, @davidchalmers42, @LakeBrenden, @ylecun, @glupyan & Ellie Pavlick (@BrownCSDept).
6
12
62
How will AI transform art, if at all? This question can be approached from a few different angles. In this new piece for @Wired, I argue that AI art is expanding the multifaceted notion of curation. 1/8.
3
19
64
