I'm thrilled to announce that I'll join
@PrincetonCS
/
@PrincetonCITP
as an assistant professor in Spring 2025 — can't wait to join this amazing group of researchers.
I will continue to work on online platforms / the societal impact of AI. If you’d like to collaborate, reach out!
One of our key sources of human data is no longer fully “human"!
We estimate that 33-46% of crowd workers on MTurk used large language models (LLMs) in a text production task - which may increase as ChatGPT and the like become more popular and powerful.
Is the YouTube algorithm radicalizing us?
Our new PNAS paper says: probably not!
But why should you trust this paper rather than other papers or reports saying otherwise?
Because we came up with a way to disentangle the causal effect of the algorithm.
In our new pre-print, we find significant evidence of user radicalization on
#YouTube
. Tracking >79M comments across >300k videos, we show that those who comment on the so-called Intellectual Dark Web/Alt-lite consistently migrate to Alt-right channels.
🚨LLMs outdebate humans when given access to opponents' personal Information!🚨
In a pre-reg study (N=820), participants who debated ChatGPT had 81.7% (p<0.01) higher odds of agreeing with their opponents after the debate (compared to a human baseline)
📜
I’m hitting the job market looking mainly for tenure-track asst. professor positions (2024)! 🚨👨🔬
My core research informs how we should curate content on online platforms — and I’m also interested in how large language models will impact on the Web in the upcoming years.
How can we ensure our data is generated by humans?
Our new preprint shows that
>30% of crowd workers used LLMs on a text summarization task on Prolific
>This is similar to previous figures on less vetted MTurk
>Mitigation strategies can reduce use >50%
📜
I’m thrilled that our paper, with
@cervisiarius
and led by
@russogiusep
: “Stranger Danger! Cross-community interactions with fringe users increase the growth of fringe communities on Reddit” won the best paper award at
@icwsm
! 🎉🎉🎉
The most useful paper anyone working with Twitter data could read. 30m that could save you a lifetime of headaches!
Great analyses/discussion on the new (improved) API including:
- location bias
- time bias
- data decay
- content moderation bias
📰
Received ChatGPT-like reviews? They may have boosted your paper's odds of being accepted!
In a quasi-experimental study of a top AI conference, ICLR, we measured the effect of AI-assisted peer reviews on scores and acceptance rates. (Led by
@russogiusep
)
New preprint 🚨
We show that recommender system audits are of limited utility in determining the prevalence of phenomena like radicalization, rabbit holes, etc., if they do not model how users interact with algorithms (which no one really does!)
📰:
This paper (to appear at WWW, I guess) proposes a very interesting framework to measure the "exploration" vs. "exploitation" of recommender systems on TikTok.
A helpful way to think about rec. systems + it is great to see research using data donations.
I’m excited to share our (w/
@iDRAMALab
folks) newest pre-print, on a topic that does not get all the attention it deserves: ~The Manosphere~! Want to know more about it? Read what we found out in this 🧵!
Paper:
@oatila
@slpng_giants_pt
Ambas as coisas são possíveis, apesar de que, como a imaginação do
@oatila
sugeriu, o pós-fato é muito mais simples. A grande dificuldade é conseguir medir a disseminação dessa desinformação e fazer a ligação com dados de saúde pública! ... 1/
When toxic web communities get banned, they don’t disappear... Rather, they migrate to a new platform!
In that context, it is worth asking:
-> Should platforms ban these communities?
Our new pre-print tackles this question
📜:
1/
Can large language models (LLMs) recognize convincing arguments? Kind of. 😅
We propose evaluating LLMs' persuasiveness capabilities by measuring their ability to recognize whether an argument would convince a user with specific demographics.
📜
Why does this matter? Because
1. human data remains critical in the sciences
2. training models on synthetic data may perpetuate biases and ideologies
3. adoption will only increase as models become popular and better/multimodal
Our paper on the effects of automated content moderation on Facebook (w
@justincheng
&
@cervisiarius
) was accepted at The Web Conference 2023! Will make a detailed thread soon, but you can already read the pre-print!
See you in Texas yeehaw (?)
📰:
Honored to receive a Facebook Fellowship🎉, which I hope will help me to advance my research. My work aims to characterize troublesome online phenomena & assess the proposed solutions, things I believe are key to improve our information ecosystem!
We (me +
@cervisiarius
) just released the YouNiverse dataset ()! 🎉🚀
It contains metadata from over 136k channels and 73M videos published between May 2005 and Oct 2019, as well as channel-level time-series data with weekly subscriber and view counts.
🧵
This WebSci22 paper by
@TamarMitts
et al shows that banning anti-vax Facebook groups increased the production of similar content on Twitter.
Cool cross-platform study design: they contrast Twitter users who posted links toward banned/unbanned groups!
📰
New on PNAS NEXUS 🚨
*Deplatforming Parler following January 6th did not decrease the activity of Parler users in fringe social media in general* Users migrated to other platforms like Gab at a rate that compensated for the decrease in activity in Parler
📰
In our new pre-print (w/
@shagunjhaver
@cervisiarius
), we study how 165 deplatforming events impacted online attention toward influencers (n=101; using Google Trends & Wikipedia Pageviews).
TL;DR: deplatforming (even temporary) decreased online attention
Você faz graduação/mestrado no Brasil 🇧🇷 e quer fazer doutorado fora do país ✈️?
Eu e
@cottascience
estamos conectando alunas/os e mentoras/es em instituições de renome (eg, Purdue, EPFL, DeepMind, Stanford) para oferecer suporte e conselhos.
Detalhes:
In our latest
@icwsm
paper, we study how proactive moderation impacts groups on Facebook. We do so by studying a feature called "Post Approvals," popular among groups that seek to ensure a high standard of quality for all their posts
🧵
@oatila
@slpng_giants_pt
É difícil saber ao certo o que se passa no feed do facebook e nos grupos de Whatsapp de cada um (apesar de termos iniciativas de pesquisa muito bacanas pra tentar desvendar essas plataformas, ex: , do prof
@fbenevenuto
)... 2/
@oatila
@slpng_giants_pt
@fbenevenuto
Fazer o ligamento entre esses dados da internet e o mundo real é difícil, afinal o dado da internet é um arroba no twitter, ou um número no whatsapp, mas os dados de saúde pública muitas vezes são agregados por bairro, cidade ou estado. 3/
We validate our results using keystroke data, finding that
1. summaries written entirely in the MTurk textbox (unlikely to be synthetic) were all classified as real;
2. In pasted summaries, there is a clear distinction between the use of extractive summarization and LLMs.
Stranger Danger? Your parents were right!
Our new
@icwsm
2024 study shows that interactions with fringe community members across Reddit drive users to join these communities.
led by
@russogiusep
and with
@cervisiarius
b) YouTube has to be held accountable and take action. I believe our work presents significant evidence that a large scale radicalization happened on the website. This process has led to horrific acts of terrorism, and should be addressed accordingly.
Later today, during the
@icwsm
poster session, I will present our work (w/
@VminVsky
and
@cervisiarius
) on why recommender systems audits are kinda finicky.
Paper:
Demo:
Three papers accepted to ICWSM'23 🏖️!
- The Amplification Paradox in Rec Systems (w
@VminVsky
/
@cervisiarius
)
- Quotatives Indicate Decline in Objectivity in US Political News (led by
@tiancheng_hu
)
- Spillover of Antisocial Behavior from Fringe Platforms
(led by
@russogiusep
)
Do you like Reddit? And also computational social science? You should def. check out , I think the format there is more adequate for meaningful discussions about science than Twitter
@oatila
@slpng_giants_pt
@fbenevenuto
... e eu me pergunto se também isso não seria uma ocasião para usar esse grande desejo de tantos de fazer o bem e combater a desinformação para gerar dados que possam ser utilizados cientificamente! 8/
When toxic web communities get banned, they don’t disappear... Rather, they migrate to a new platform!
In that context, it is worth asking:
-> Should platforms ban these communities?
Our new pre-print tackles this question
📜:
1/
This
@acm_chi
paper found that LLM-powered conversational search systems are more susceptible to selective exposure and opinion polarization.
The effect becomes even stronger when LLMs are steered to reflect user opinions.
It's that time of the year once again!
With
@cottascience
and 20+ other amazing mentors, we will have set up a simple "mentorship" program to help you get your Ph.D. application in shape.
This
@acm_chi
paper found people perceive content labeled as human and AI-generated as equally original, trustworthy, and informative.
Perceptions are bound to change as Generative AI becomes more used. Can someone please do some meta-analysis on this!?
Using answers from the original study and data synthetically generated with ChatGPT, we train a bespoke synthetic-real classifier that works remarkably well. We use this classifier to estimate the prevalence of synthetic answers in the task we reran.
This
@icwsm
paper comprehensively studies "conflict" in Facebook communities. Users involved in conflict are more likely to be male, engage in other negative online activities, and are less connected to the group where conflict occurs.
@oatila
@slpng_giants_pt
@fbenevenuto
Será que conseguiríamos criar modelos de colaboração entre pessoas engajadas com causas importantes (como o
@sleeping_giants_pt
) e projetos científicos? Existe esse conceito muito bonito de “Citizen Science” (), onde a ciência seria mais participativa… 7/
My new YouTube addiction is Sabine Hossenfelder (
@skdh
). Her skeptic take on physics (which I know very little about to be quite honest) and, more broadly, in what she calls "beauty-driven" science is very refreshing
New ICWSM paper led by
@russogiusep
! We show that antisocial behavior from fringe platforms can spill over into mainstream social media through “coactive users,” those who use both platforms.
This
@acm_chi
paper presents a systematic study of platforms' community guidelines. They develop a taxonomy and annotate the policies of 43 large online platforms. Cool resource that could be extended to study temporal changes to community guidelines.
@oatila
@slpng_giants_pt
@fbenevenuto
Pesquisadores poderiam usar métodos (matching, variáveis instrumentais) para estimar o impacto indireto da desinformação na saúde. Isso são técnicas mais avançadas mas já estabelecidas (a wikipedia tem um exemplo legal ) 5/
We re-run a human intelligence task from Horta Ribeiro et al. 2019 (yours truly): to summarize abstracts from the New England Journal of Medicine. This is a laborious task that is within reach of current LLMs, like ChatGPT, Claude, Bard, etc.
In late 2017, I got surprised by the presence of fringe ideologies (e.g. White Supremacy) on YouTube. In 2018 I got redirected to one of such channels from an "Anti-SJW"-style video. Later that year, when I read
@zeynep
and
@beccalew
pieces on the alleged phenomena of...
...we collected and annotated a YouTube dataset of unprecedented size. 360 channels, more than 79 million comments, more than 2 million recommendations. We broadly categorized channels into Alt-right, Alt-lite, and Intellectual Dark Web...
🎉 New preprint fresh out of the oven!
We find that anti-social behavior from fringe websites (eg incels-dot-is) spillovers into mainstream social media (eg Reddit) through users who participate in communities on both fringe and mainstream platforms.
📰
@oatila
@slpng_giants_pt
@fbenevenuto
Uma direção que eu acho promissora é tentar estudar isso um pouco mais indiretamente. Isso é se a gente tem a relação causal apoio ao governo -> desinformação sobre cloroquina/falta de isolamento social, e existirem dados geolocalizados de apoio político e de saúde pública … 4/
@oatila
@slpng_giants_pt
@fbenevenuto
A médio prazo, contudo, seria muito bacana tentar envolver pessoas não diretamente ligados à pesquisa científica mas que querem ajudar a resolver problemas na sociedade! Devolvo uma pergunta não só pro
@oatila
mas para todos... 6/
New Twitter paper finds that a soft intervention during the creation of an offensive tweet make people:
1 - edit their tweets to be less offensive
2 - less likely to compose offensive tweets in the future
3 - receive fewer offensive replies
This
@icwsm
paper longitudinally studies bot "accusations" on Twitter. They find evidence of "concept drift" after 2017, i.e., people use the term bot pejoratively to dehumanize those they dislike.
> Alt-lite channels are easily discovered from I.D.W. channels, and that Alt-right channels may be reached from the two other communities (!).
I believe our research has implications for content creators and for YouTube:
This pre-print makes a compelling argument for including students in the quest to replicate scientific findings!
They describe a (quite successful) replication project done by students in a Data Science course at EPFL.
...user radicalization on YouTube it all just clicked in my head. I still thought that the evidence indicating this phenomenon was happening was very anecdotal. Spent months devising a careful data collection method with
@raphaottoni
. By the end of this period...
More personalization at Spotify:
1) increased podcast streams
2) decreased the individual-level diversity of podcast streams
3) increased the aggregate diversity of podcast
streams
This project came from restless weekends and late nights working (or days, weird timezones) working with
@VminVsky
and Bob. I’m particularly happy Venia (a master's student who will be applying to Ph.D. positions soon!) had so much willpower within him to push the final sprint :)
As many high-profile people adopted the "Red Pill" metaphor, it is my pleasure to announce that we have just published a much improved version of our (with
@iDRAMALab
) manuscript on the first large-scale multi-platform analysis of the Manosphere!
📜
1/
This
@acm_chi
paper asks: what if HCI researchers saw the policy as integral to HCI?
I was happy to see the call for more observational empirical work on system-people-policy interactions and to read an overview of the existing work on policy + HCI.
a) I believe that popular content creators like those in the Intellectual Dark Web, should be aware that this process is happening I genuinely believe this is not what they want, and they are in a good position to try to weaken fringe ideologies like white supremacy.
Nature paper on online toxicity & participation
Cool analysis, and it is nice that Nature is publishing this kind of work. I wish the "toxicity does not discourage participation" was better tied to a DAG... Thinking about self-selection makes me dizzy
I am looking fwd to
@IC2S2
this week — if you are going and want to connect, let me know!
Also, considering the weather forecast, this might be the first time I enjoy American AC
3) We also find that the intersection is not only growing due to new users but that there is significant user migration among the communities being studied. Users that initially comment only on content from the I.D.W. or the Alt-lite throughout the years consistently start to...
4) Lastly, we take a look at the impact of YouTube's recommendation algorithms, running simulations on recommendation graphs. Our analyses show that, particularly through the channel recommender system,...
My MSc Thesis (w/
@wagnermeirajr
&
@virgilioalmeida
) won third place in a Brazil-wide award. Very happy about that, but also humbled by the quality of the work that was presented by others. It truly amazes me how much great stuff is being done in Brazil.
> on Alt-right content. These users are a significant fraction of the Alt-right commenting user base. Interestingly, although control channels share, on a yearly basis, a significant number of users with Alt-right channels, we cannot observe significant user migration from them.
1) We looked at the growth of the I.D.W., the Alt-lite and the Alt-right throughout the last decade in terms of videos, likes, and views, finding a steep rise in activity and engagement in the communities of interest when compared with the control channels;
This
@acm_chi
paper examines how LLM's "writing style" impacts inclusion, control, and ownership over the work when co-writing with LLMs. People who feel like the LLMs "sound like them" are likelier to feel ownership over texts co-written with LLMs.
...These communities were broadly defined by non-profits, by the media, and by the own content creators. We also collected control channels from large media outlets. Our findings blew my mind a little bit:
It is my pleasure to announce that reproducibility code + data for our forthcoming paper "The Evolution of the Manosphere Across the Web". Is now available!
🧑💻Code:
>
🗄️ Data:
>
📹 Video:
>
We call the new approach counterfactual bots.
It needs two ingredients:
1: real user traces (here YT histories)
2: bots—computer programs simulating user behavior
For each YT history, we instantiate various bots. One is a "control bot" that watches the same videos as the user.
I wrote a blog post about the promise and perils of doing YouTube-related research. Secretly, the goal is to promote our new dataset, YouNiverse🚀. Check it out!