Mark Dredze
@mdredze
Followers
4,449
Following
791
Media
412
Statuses
2,079
John C Malone Professor at @JohnsHopkins @JHUCompSci @jhuclsp @jhumceh ; Part time @techatbloomberg (tweets my own) Mastodon @mdredze @sigmoid .social
Baltimore, MD
Joined August 2012
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Flamengo
• 268609 Tweets
Beyoncé
• 244460 Tweets
sabrina
• 131046 Tweets
Cruzeiro
• 112134 Tweets
#DNCConvention2024
• 102708 Tweets
#DNC2024CHICAGO
• 96376 Tweets
京都国際
• 89582 Tweets
Román
• 81569 Tweets
Rossi
• 67548 Tweets
Bolívar
• 61355 Tweets
Sudamericana
• 52151 Tweets
Adam Kinzinger
• 45400 Tweets
Advincula
• 44113 Tweets
#ShortNSweet
• 43068 Tweets
Yachty
• 40847 Tweets
Riquelme
• 37557 Tweets
甲子園決勝
• 37287 Tweets
関東第一
• 37159 Tweets
#KamalaHarris
• 33594 Tweets
Romero
• 33212 Tweets
Merentiel
• 30073 Tweets
ラストマイル
• 28846 Tweets
Boca Juniors
• 23557 Tweets
タイブレーク
• 23047 Tweets
Kemp
• 20171 Tweets
Figal
• 19879 Tweets
Ayrton Lucas
• 19617 Tweets
Vignolo
• 17275 Tweets
Carlinhos
• 16682 Tweets
Cassio
• 16279 Tweets
Gerson
• 14558 Tweets
Maya Harris
• 14155 Tweets
Leon Panetta
• 12798 Tweets
関東一高
• 12372 Tweets
교토국제고
• 11646 Tweets
韓国語の校歌
• 11466 Tweets
Bruno Henrique
• 10992 Tweets
New study! We compared ChatGPT responses to people's medical questions with those of doctors. Healthcare professionals preferred ChatGPT 79% of the time; as more empathetic and higher quality.
I'm excited to figure out how to use LLMs to help doctors!
23
134
571
NLP research over the years:
1990s: specialized features for NER
2000s: non-parametric prior for unsupervised parsing
2010s: embeddings + a forward/backward, inside/outside RNN
Today: When you say “Super pretty please” to the LLM, performance goes up
7
95
559
I'm very honored to announce that the
@JohnsHopkins
President and Board of Trustees have approved my promotion to Full Professor in
@JHUCompSci
@HopkinsEngineer
.
49
10
374
This slide is from my lecture on data for ML/AI systems. I happen to think it's clever.
Like when you see it.
15
34
339
I am thrilled to announce that I will be the Director of Research (Foundations of AI) for the
@JohnsHopkins
AI-X Foundry, the new university-wide org for Artificial Intelligence.
@HopkinsEngineer
19
15
286
Idea for a killer app:
ChatGPT: Modify my ACL paper, which is currently over the limit by 4 lines, so that it fits into the 8 page limit.
12
11
258
Putting together a slide deck on clinical LLMs. My student's proposed title slide is too good not to share.
7
18
223
Industry PhD summer interns talking to their labmates at school.
0
6
211
#BloombergGPT
Chronicles are now live!
@TechAtBloomberg
released a detailed narrative of our experience training a 50B parameter
#LLM
. We hope this helps other groups building large
#AI
/
#NLProc
models. Let’s look at some of the highlights. 🧵
A few weeks ago, we published our
#BloombergGPT
paper on arXiv. It's now been updated to include the training logs (Chronicles) detailing our team's experience in training this 50B parameter
#LargeLanguageModel
. We hope it benefits others.
#AI
#NLProc
#LLM
0
40
182
5
47
195
Students often ask: How do you review PhD applications? What do you look for? Tell me the secrets behind the process!
I made a video last year explaining how it all works.
To all PhD applicants: good luck with your graduate school search this year!
1
45
189
Step 1: Promotion.
Step 2: Take a take "Full Professor" picture.
Step 3: ???
29
2
186
@JohnsHopkins
Engineering
@HopkinsEngineer
is making a massive investment in AI. The plan is so ambitious and unique that it’s worth breaking down the announcement in detail.
This will transform the landscape of engineering research and education. 🧵
4
40
179
🚨Summer AI Research Internships🚨
The
@TechAtBloomberg
office of the CTO is looking for summer AI research interns! ML, NLP, Vision, LLMs
Spread the word.
Interested candidates should apply here:
2
37
161
We (
@jhuclsp
) are hiring faculty in Computer Science! Spread the word.
All areas/ranks, and especially candidates currently serving in faculty or industry research positions.
We also have an "early action" option. It's a unique ad, so it's worth a 🧵
3
45
146
Negative vspace.
@nlpnoah
@anmarasovic
@mdredze
after years of negative vspace crimes, i no longer have a moral compass
1
4
51
2
12
147
1) you should not be working on research problems that will be solved by the next large language model release.
2) it is difficult to know what those problems are.
5
13
136
I’m delighted to have been part of this amazing team that spanned
@Bloomberg
’s
#AI
Engineering &
#ML
Product and Research group in CTO. This was a thrilling and exhausting experience, and we’re excited to share with the larger research community what we learned.
Meet
#BloombergGPT
👋🏻
This 50-billion parameter
#LargeLanguageModel
was purpose-built from scratch for
#finance
using a unique mix of
@Bloomberg
's
#data
and public datasets to support financial
#NLProc
tasks.
#AI
#ArtificialIntelligence
#LLMs
#ML
#GPT
17
245
561
6
18
140
I'm looking for a postdoc! Topics: LLMs, text generation, QA, medical NLP.
Join two amazing postdocs in my group:
@hanjie_chen
(phd
@CS_UVA
, incoming prof
@RiceCompSci
) and
@sharonlevy21
(phd
@ucsbNLP
, incoming prof
@RutgersU
)
Apply:
🙏retweet
Questions?
5
56
132
The
@aclmeeting
policy on AI Writing Assistants is excellent. It is detailed, thoughtful, and informative.
Well done!
1
36
125
A PhD student watching a full professor try to write code to run on a GPU.
0
6
121
Author feedback: A weakness of this review was that it was generated by (what appears to be) chatGPT (gpt-3.5-turbo.) We suggest the reviewer use newer models, like GPT-4o or Calude 3 Opus, and evaluate the efficacy of multiple models for review generation.
3
10
114
In the spirit of First Day of School pictures.
I can't believe this is really semester
#30
at
@jhucomp
@HopkinsEngineer
. Despite having done this for a while, I'm really excited for this school year.
1
4
114
Folks: I still teach SVMs in my ML class. I don’t plan on stopping anytime soon.
9
4
107
Really interesting. When we used RNNs (LSTMs), a persistent issue was that they would lose information in the middle of the sequence. Tokens at the start and end had a large effect, while tokens in the middle were lost. It's interesting to find Transformers having the same issue.
Lost in the Middle: How Language Models Use Long Contexts
Finds that performance of LMs is often highest when relevant info occurs at the beginning or end of the input context, and significantly degrades otherwise
18
130
687
4
16
104
This one stings.
3
6
102
Co-advised PhD student.
Run in opposite directions to see who your dog loves more.. 😅
949
14K
121K
1
1
101
Congratulations
@partha_p_t
! Well deserved.
We went to grad school together at
@CIS_Penn
, shared an adviser (
@earnmyturns
) and office. We've published (and failed to publish!) together. Partha has been a fantastic collaborator, colleague and friend.
4
0
99
Hi DALL-E.
Create a photograph of a PhD student working on a single paper for an upcoming conference deadline.
2 papers
10 papers
100 papers
5
32
91
Not subtweeting a specific paper, but it feels like many papers.
2
6
85
Applying for a PhD in
#NLProc
(CS/ECE (STEM))?
What goes into my research statement?
Publications before I apply?
Email professors in advance?
Admissions are stressful! You have questions! For answers, watch my PhD Admissions AMA.
#admissions
1
14
80
Advisor in the audience when PhD student steps up to give a conference talk.
0
7
79
I just realized that I defended my PhD thesis 15 years ago today. Happy doctorversary(?) to me.
2
1
79
Hi folks at
#ACL2023
Do you know someone interested in PhD programs in
#nlproc
? We'd love to have them apply to Johns Hopkins! Our growing
@jhuclsp
faculty (almost 30!) now include
@DanielKhashabi
and
@anjalie_f
3
24
78
Good news for AI in Baltimore!
"Baltimore region selected as federal tech hub for artificial intelligence and biotechnology"
Adding >1000 people to our Engineering school over the next few years should help 😄
4
15
75
Current state of AI: a friend was using a customer service chat and wasn't getting anywhere.
Friend: Can I speak to a live agent?
Chat: I am a live agent.
My takeaways:
1) General public believes AI is this advanced
2) LLMs will be integrated everywhere
3
13
70
Faculty (me) can be really bad at time estimates.
Me: Train the model on this data. Should take a few hours of work.
Student: 😐That will take 2 weeks.
Me: The budget documents are going to take me hours to complete!
Admin: 😑It's literally 5 minutes of work. Please do it.
3
2
70
Me: How is the model fine-tuning going so far?
My student replied with this:
5
5
64
This is a lovely and unexpected surprise! Our article on using AI Chatbots to answer patient questions was the most viewed paper of 2023 at
@JAMAInternalMed
!
Almost 250k views!
1
11
65
When someone in your lab really knows how to work with PyTorch and Huggingface.
4
2
63
I am honored to receive an Optum Labs Research Award for the use of large language models for generating synthetic clinical records to reduce bias. We are very excited to start the project and thrilled about the recognition.
4
4
64
Reviewers when we state experimental limitations related to GPU availability and cost.
2
8
59
Help review more ARR submissions if you can! Otherwise, this is the reality we face.
ARR needs your help! We received 5800+ submissions for the June cycle, but with our current capacity, we can handle only half of these submissions. We can't start the review process without significant help. (1/2)
12
48
89
2
14
59
ARR resubmission choice.
1. New reviewers
2. Same reviewers
3. Only keep the two positive reviewers
2
1
57
Excited for my speaking event today!
Note: it’s not in the building behind me, but across the street 😁
2
1
57
JHU is
#1
in research spending for 44 years in a row!
I can only claim credit for the past 15 years. The 29 years before that were all achieved without my help.
🤣🤣🤣
TLDR:
@JohnsHopkins
is a fantastic place for research.
1
9
53
If you write a paper review and provide a list of weaknesses, and the author's response addresses those weaknesses, raise your score. Otherwise, you need to better explain your concerns.
4
7
54
Me in grad school: If we use the biggest machine, and are careful with our code, I think we can use a model with 1 million parameters.
Me now: let's keep it simple and use the 70b parameter model.
1
4
53
Great thread! Building training sets for LLMs is challenging and time consuming. It was a major part of BloombergGPT.
This will become increasingly important. The key differential between LLMs will increasingly become data.
Just decided to make a thread to show how many decisions and pre-processing steps you need to train large language models (LLMs) such as LLaMA. This is mainly based on CommonCrawl (CC) dataset and the pipeline used in LLaMA to generate its dataset. 1/n
6
99
509
1
13
51
Beautifully said.
A major focus of our PhD admissions process is to find those who have not just been successful so far, but who will be most successful in the future. We want those who have made the most of whatever opportunities they have had so far.
💎It’s PhD admission season!💎How do you make admission decisions?
Instead of the typical top pubs criterion, why not look for and lift up “diamonds in the rough”? The reward is so much bigger when you uncover diamonds who start shining *because of you*.
2
26
195
1
2
50
I'm honored to give an invited talk today about BloombergGPT from
@TechAtBloomberg
at the Symposium on Large Language Models for Financial Services (FinLLM )
@IJCAIconf
#ijcai2023
The talk is online, sorry I can't join in person!
1
7
50
An absolute privilege to speak today to the
@WHFellows
about AI risks and opportunities with
@munmun10
,
@lrvarshney
, and moderator Cara LaPointe.
@Surgeon_General
spoke about technology, mental health, and adolescence, a great intro to our topic!
1
4
48
Great news!
@JohnsHopkins
is funding our proposal "Artificial Intelligence (AI) for Health Equity Research (AIHE)"
This is a truly cross-disciplinary effort that includes
@JHUCarey
@HopkinsEngineer
@bermaninstitute
@HopkinsMedicine
3
10
48
@mayhewsw
I remember thinking that we couldn’t train our own BERT model. It’s too expensive.
And now…
2
0
47
End of the semester group lunch! Thanks for a great semester everyone, have a good summer, and good luck with the upcoming
@emnlpmeeting
deadline 😁
0
2
47
When the reviewer says that critical relevant work wasn’t cited but won’t tell you what you are missing.
4
3
47
Fantastic research group meeting today at
@JHUBloombergCtr
. We then went to see the Cherry Blossoms. We also saw Stumpy!
2
11
44
My lab often communicates using gifs. Here's
@YLiiiYLiii
telling everyone we need to clear some disk space. Love it!
1
5
45
Congratulations to my PhD student Keith Harrigian on defending his
@JHUCompSci
thesis "Towards Robust Natural Language Processing to Promote Health Equity".
Thank you to committee members
@anqi_liu33
and
@2plus2make5
0
2
45
Two sides of being a CS professor:
At work: advise on the development and application of transformational technologies.
At home: can you help me fix my printer?
2
3
45
Congratulations Dr. Elliot Schumacher.
@elliotschu
Me: what have you been up to in the few months since you defended?
Elliot: I posted three new papers to arxiv.
I’ve got reading to do!!!
0
5
45
Hi
#neurips2023
folks 👋
@HopkinsEngineer
is growing dramatically in AI/ML/data science. 80 new faculty, 30 chairs, and many other opportunities. May be the largest engineering hiring initiative in history!
We're hiring in all areas and all ranks. Join us!
@JohnsHopkins
Engineering
@HopkinsEngineer
is making a massive investment in AI. The plan is so ambitious and unique that it’s worth breaking down the announcement in detail.
This will transform the landscape of engineering research and education. 🧵
4
40
179
1
17
43
CSStipendRankings: PhD Stipend Rankings
Happy to see that
@JHUCompSci
ranks
#4
overall on "After Fees & Living". Ensuring our graduate students are well supported is a priority in the department. Baltimore is also by far the biggest city in the top 5.
3
3
44
Reviewer: why didn’t you describe your data?
Me: it’s in the paper, see section called “data”
2
1
44
🚨 Postdocs at
@jhuclsp
!
We're interested in several topics, including:
- Explainable AI/NLP for medicine
- Clinical NLP
- Evaluation of LLMs
Apply:
Spread the word!
#ACL2024NLP
#ACL2024
@aclmeeting
0
18
44
Today I am working from the beautiful new
@JohnsHopkins
Washington DC building 555 Penn, home to
@JHUCarey
and
@SAISHopkins
.
It's amazing! Take a look at our neighbors down the block. Seems like a good place to be if you care about shaping US national policy about AI. 🙂🇺🇸
0
3
43
Me after a start of the semester research meeting brainstorming ideas.
0
1
43
I read papers that build and test a dataset with an LLM. Even when it's justified and the paper does this well, there is still no grounding with a human eval.
We're creating a literature of benchmarks and papers with the same underlying bias.
My worry? House of cards.
1
1
42
Immensely satisfying today to see results from a baseline method beat chatGPT. Brings back some sanity to research.
"Wow, it's really amazing that chatGPT does so well! I mean, maybe it's trained on our test set, and it is worse than our baseline method. But still!"
2
0
42
✨Congratulations✨ to my PhD student
@KaiserWhoLearns
who received an Honorable Mention at
#CoNLL2023
.
🎉🎉🎉
6
5
65
3
6
42
Congratulations to my colleagues on this outstanding paper award. The best part? This paper doesn't mention GPT, LLMs, BERT, etc!
There are many fantastic and creative ideas out there. A diversity of research topics is great for the field.
The paper won the outstanding paper award at EACL🎉! Thanks again, my amazing collaborators! (Will Gantt, Weiwei Gu,
@ctongfei
,
@aaronsteven
, and
@ben_vandurme
)
#EACL2023
#eacl
@jhuclsp
9
10
85
0
5
41
I was sad to learn about the passing of Dr. Eugene Charniak. He was a giant in the field and recipient of the ACL lifetime achievement award, among many other honors. "Charniak parser" was a ubiquitous term when I was in graduate school. 🧵
2
7
41
I will NOT be at
@emnlpmeeting
, but you should say hi to my PhD students! 👋 They are fantastic, and happy to answer questions about my group,
@jhuclsp
and (most importantly) their excellent work. Several are seeking internships.
@Alexir563
@dam_nlp
@isabelcachola
@amuuueller
1
4
40
@andriy_mulyar
@sleepinyourhat
@srush_nlp
@chrmanning
@ChrisGPotts
There are a lot of good comments already. I will try not to repeat anyone else.
In general, my advice is always to work on a topic that:
1) You don't think someone else will solve.
2) You are uniquely positioned to contribute.
1
4
40
Hot take for reviewers: if you write “repeat experiments with newer LLMs from the past few months” ask yourself: why?
“Newer” isn’t a good reason. What changes about the papers hypothesis?
Also: if your paper would be invalidated by a new model, write a different paper.
1
1
40
Good luck to everyone working on your ACL submission!
May your jobs finish on time, may the turkers produce the desired results, and may ChatGPT help you write a great paper (consistent with the ACL policy on AI writing aids).
To my students: I'll now go back to paper editing.
0
2
39
Johns Hopkins has a fantastic new PhD Fellowship: Vivien Thomas Scholars Initiative (VTSI). Fully funds a STEM PhD at JHU for the exceptional diverse talent that exists at HBCUs and minority serving institutions.
@JHUCompSci
@HopkinsEngineer
1
16
39
A bonus to speaking at
@JHUBloombergCtr
is that you get awesome pictures!
I just wrapped up my talk about the myths of AI at this event. I had a fantastic time with a great audience, and a beautiful backdrop.
0
1
26
0
3
39