Tom Carpenter, PhD (@tcarpenter.bsky.social)
@tcarpenter216
Followers
9K
Following
36K
Media
1K
Statuses
21K
Data Scientist, Survey Scientist, People Scientist | #rstats, python, ai/ml, psychology
Seattle, WA
Joined January 2012
What most activates a sense of purpose within you in your career, work, etc.? For me, it's . * working with others.* who have something to teach me.* while I use people/data/survey-science skills.* to help others. How about you?.
6
1
52
The lack of fundamental probability knowledge in society is painful. If my doctor said this, I’d be elated.
356
2K
77K
@TableSlamStan And it should. But it’s reasonable to inquire into the possibility that the chance of success here is better. And certainly you should not assume that a failure is “overdue”.
9
14
9K
Whatever your stance on journals, it’s *BIZARRE* that Elsevier (with a profit margin rivaling Apple’s) won’t pay for your peer review labor, gets content for free, and sells it back to your school for profit. Serious question—how can we do this differently? Alternatives?.
83
381
2K
I’m 6 years post PhD, a tenured associate prof, and happy. I still have recurring dreams that I did not finish a class in high school, and so all of my education is a fraud, and I will lose my job if anyone finds out. Does anyone else have this? Is this just imposter syndrome?.
228
72
2K
Academic tweeps: do you find that you essentially have to write and rewrite and rewrite (and rewrite) in order to get a really clear, strong manuscript? Or is it just me.
200
160
2K
Was asked by a student this week: “but don’t you always *want* to reject the null hypothesis?” . I responded: “not if it’s true!” and it was a revelation for the class. They need someone to tell them the goal is *truth,* not p < .05.
29
473
2K
@HeadSnapss Unclear what’s untrustworthy in this example, but the meme seems to imply that the doctor is “overdue” for a bad outcome (which is false; that’s not how probability works).
5
8
2K
This is a great book! Do you analyze survey data? Do you lack the stats training to address sampling biases or to use stratified designs? Were you trained in social science rather than stats? Do you want to up your quant game? Get this book. You won’t regret it! #scilearn
12
165
1K
@WilliamBLong21 We call that the gamblers fallacy. There is no “due” across independent events.
8
5
1K
Friend reminder that it’s all regression.
17
163
802
In Rstudio, writing R code, and started typing "import tidyverse as . " and then stopped and cried.
28
64
789
@rustykitty_ Start with the chair or dean. > you did not use AI.> there is no proof that you did.> the tool in question is questionable (can you see the evidence it supposedly has?) and besides the score was low!.> the faculty admits they don’t care . ~ a former prof + AI / ML practitioner.
3
11
756
This book is great and has more value than you’d think. But the best value is chapter 8 on “career advice” … this has to be the best DS career chapter I’ve seen. Red flags in a job? Why data teams succeed if fail? Why you love (or hate) your data job and what to do? It’s here
9
75
760
Not working 17-hour days is quiet quitting?.
11
87
690
Academics, may I introduce you to my friend, professional project management? I feel like this relationship should have begun sooner.
33
38
641
Does anyone go back and forth between thinking (a) their paper is amazing and a great contribution and (b) their paper is obvious and makes not contribution? How do you keep the faith alive?.
25
15
555
Please RT! Psych Tweeps: Started a google doc to curate an annotated list of R books, tutorials, and resources for folks wishing to "make the jump." Think: colleagues, grad students, and undergrads. Fully editable! Let's keep this going? Edit/share.
14
311
535
Does anyone else do this?. Me: ooh, look at this great resource I see on twitter. Let me email it to myself for later. Also me: why did I send myself so many emails!?.
34
7
515
@ColIegeStudent Serious take: Major is different from career path. The major is about building a foundation. There are MANY more career paths than majors. Most people build satisfying careers and most use skills from their major, but not directly.
3
73
454
Friends--I am thrilled to announce a major transition: I'm walking away from tenure and pursuing an impactful career around analytics/data science. It's missional work to me and I am thrilled for the next chapter. I start in January at Microsoft! Details here :)
37
14
404
Let’s talk about reproducible analyses! . Before I left academia, I started making reproducible R notebooks for every study. Walks user through every step, explains logic & matches paper. Folks—it’s as much work as the actual paper and the only way i see to make this work . 1/n.
A plea from Steve Lindsay to psychology professional societies that publish journals: Assess computational reproducibility
4
61
425
STOP talking trash about different statistical techniques. Multilevel models are VERSATILE. t-tests are ICONIC. Regression is MAGNIFICENT. Factor analysis is USEFUL. ANOVA. Mixture models are CUTTING EDGE.
7
63
403
Psych folks who know #rstats but are intimidated by SQL databases (i.e., industry work)--you already know the core concepts. Let me help you translate. 🧵. 1. The "database" is analogue to your R environment. It can store many tables. (1/n).
8
62
391
After a year at Microsoft (and out of academia), I’m amazed at how much more organized my personal life is. Planning. Notes. Not just doing the thing that’s on fire. Such a good life change!.
5
7
395
Psych folk--consider this for your department. We added a "data analytics" track in our psych major. Students can learn advanced R, databases/SQL, and beginning programming. Add to stats/methods and all the people insights and you are positioned for industry with your BA
16
55
380
Just drafted 25-page guide to scientific writing for undergraduates in psychology (focused on students in a Research Methods). Broke good science writing into four principles and three skills. Looking for feedback! Also might there be value in submitting?
17
56
379
Analyzing longitudinal data? This collection of tutorials is *AMAZING* . beautiful markdown tutorials for every kind of analysis with code, data, everything
4
114
383
@g_mppa @TableSlamStan Also “regression to the prior.” Yes, it’s possible that this doctor’s average is worse than 100%. But that does not imply that the recent successes make the one “due” for a failure as the meme suggests (by my reading).
3
0
384
Most psych grads won't go into academia or even "psychology." They get jobs. To use stats skills, most workplaces emphasize excel, not R (against my own wishes, but it's reality). We need some place in the psych major where they can get minimally competent in excel.
23
28
328
Do you simulate data in R? What tools do you use? What can you share? I'm building a resource list (and will share here when done)! Share code snippets, blog posts, R packages, etc. For example, just found 'truncnorm' that generates normal data within a given range. What else?.
44
97
324
The Lumley book on surveys is so good! I hope the {survey} package has a longevity plan because I’m all in
1
44
336
New meta analysis shows almost a zero correlation between student course evaluations and learning
11
253
300
The average is the maximum. Got it. …Wait. ….
19
17
296
Want to up your regression game? Nominal outcomes? Ordered outcomes? Switch between R and python? I am finding this book by @dr_keithmcnulty to be quite delightful (free online!)! Great, fast read! Has a nice section on power analysis #scilearn.
3
66
307
Linguistic analysis of language from police body cameras during traffic stops. Analysis reveals a considerable difference between how white and nonwhite individuals are treated. White folk cautioned about safety; black folk told to keep hands on wheel. h/t @MikeShor for posting
6
115
271
Do you have a background in social science and want to use stats to solve real world problems? Did your grad training stop at multiple regression variants? Do you find advanced stats books inaccessible? . This book was written for you. Read it! #scilearn
3
35
287
Ex-academic in tech, end of week 1. I never realized how much I felt “perpetually behind” in academia and how much of an emotional toll that took. We habituate to it but it still takes a toll.
5
20
270
I’m struggling to find words, so I will just share some personal news: I have been promoted to Associate Professor, starting in the fall! I can’t help but think that psych twitter has played a major community support role. Thank you!.
15
1
267
In case you wondered why the “database icon” looks like a cylinder.
@tcarpenter216 @adamjnafa The original hard drives, you used to be able to add more platters to expand storage. Thus bad boy probably weighed 200lbs and stored 10MB
2
75
280
Ex-academic in tech, month 4. The best thing I got from my PhD was the confidence that I could learn anything to do a project. Learning happens at (or over) the edge of comfort zones. So new projects will be uncomfortable but growth WILL come. Embrace it. You’re a learn-it-all.
1
18
255
Data science truly is an amazing job for a methods/stats person. Training cool models + running cool analyses—much cooler methods than i used in academia. Building with the coolest toys. Solving big problems. Really impactful work. Living the dream!.
4
14
274
If you analyze data and care about stats, read this book. It’s accessible and will convince you that much of what you know is wrong. Many statistical techniques are much more fragile (non-robust) than you were taught.
5
52
267
🧵Want a great industry job doing behavioral science, surveys, data analysis, data science, or other "psych research" skills? You might be surprised that technical skills are NOT the thing you should focus on (1/n).
3
40
256
Note to self: when randomizing items in Qualtrics, be sure to randomize the order of the *items* and not the order of the *likert scale points* . dodged a disaster there!
15
14
249
It's official. My students vastly prefer R over SPSS in an intro stats course for psychology majors #rstats #spss #psychology #teaching.
6
60
236
For the first time, I had a grant accepted today. Many failures to get here, but thanks to psych twitter for your encouragement and the conversations that inspired the project.
10
1
231
Since adopting #OpenScience practices, I’ve fallen in love with research all over again. It feels more real—a quest to find truth, not just a publication game. Just another perk. Thanks open science folk!.
2
44
229
Love this intuitive explanation for R^2 from @dr_keithmcnulty. How large are your residuals using the mean as a predictor? How large are they under your regression? The R^2 reflects the proportion of the (squared, summed) residuals that you have reduced.
5
27
232
Once again I find myself searching for "hide legend ggplot".
20
9
220
Curious for thoughts. Why is it *SO HARD* for people to admit they're wrong? What if we created a culture that normalized being wrong--and then improving? Seems integral to everything: science, relationships, personal growth. Have to know I'm wrong today to be more right tomorrow.
20
36
228
Leaving academia? It can be a big leap. Remember this, PhDs -- your whole game is that you learn whatever you need in order to get the job done. Don't know databases? You will learn. Don't know what a blob storage service is? You can figure it out. You're a "learn-it-all".
2
12
219
It took me several years to realize this, but academic twitter is largely the same debates repeated on a roughly annual rotation.
11
11
212
I get that this analysis would not be publishable and doesn’t necessarily give an interpretable signal, but the personal attacks, tearing others down, and criticism with nothing constructive to say … makes me not want to be part of the stats community.
There are some confounders here, but the inflation thing is actually statistically significant! Each additional $100 of inflation in a state since January 2021 predicts a further 1.6 swing against Harris in our polling average vs. the Biden-Trump margin in 2020.
19
7
235
It’s also not a knife or a spoon or a comb. Or many other things it doesn’t try to be. #Rstats is excellent at functional programming with data. It’s not useful for much else. And that’s ok. Anyone complaining that it’s not “more than that” doesn’t understand R.
14
15
221
In the past 24 hours, I have been called unskilled, less educated than a HS student, and held up as the example of all that is wrong with the PhD system. I’ve taken down my post and will not be tweeting about data science any time soon. It’s not worth it.
48
2
223
Hot take: the postdoc (and with it, academic funding) system is fundamentally broken.
22
12
210
dplyr's pivot_wider() and pivot_longer() are some of the best APIs I've seen for quick data reshaping.
14
8
218
Kids, don’t plagiarize, cheat, or fake your work. We—your professors—know that the only thing that can come of it is a speaking circuit with big fees, a prestigious position at Duke, and a tv show with a character based on you. Not worth it.
I'm totally flabbergasted: At what point will there be some consequences for this type of behavior from Duke University, academic journals, or other institutions that ostensibly care about academic integrity? . What is the appropriate outcome in a situation like this?.
5
23
205
Data visualization in psychology is . lacking. One of the best books on #dataviz is this one by @ClausWilke -- and turns out it's free online! Read it, preach it, teach it. Reviewer 2 often told me to remove figures. still worth a shot!
4
50
208
More on this book (basics of modern stats, data science, ML). I’ve rarely seen a book THIS GOOD at teaching. Every sentence is worded to avoid confusion, lay foundation, or build deep understanding. Makes me miss teaching stats. Worth your time. #scilearn.
2
14
214
Ok but to be fair sometimes that works.
5
15
197
I don’t know any R user who thinks other languages should go away. R is specialized for data analysis and, with its libraries, does things you can’t do in other languages. Typically I see R users defending their R usage against criticism from people who don’t analyze data.
13
11
204
My wife and i had a whole conversation about which direction they rotate before we realized they ARE NOT MOVING AT ALL.
7
61
204
Research methods instructors: I give you a gift for the first day of class.
I. I have no words. Non-randomized, n=63, . Control: staring at desk. Active intervention: caring for plant. Outcome: tiny difference in proxy of mental health. Headlines:
6
43
187
Heck of a time to leave academia for data science 👀.
9
7
195
In my analytics consulting career, I discovered that this is how many organizations do inference today. Fun fact—a client once thought I was LESS skilled than another analyst bidding because I claimed needed a larger sample size to draw a conclusion.
Reading a paper from 1905 which reports results as means & then says "Just take a look, the means are different". No analysis? And this is how I learned the t-test wasn't invented until 1908. People were just like, looking at numbers & going "yeah, looks different" before then.
4
14
192
What if universities self published academic journals for half the price and the money went into reducing tuition?.
14
20
182
I’ve spent my life assuming there are infinite moments. My PhD taught me that life comes “later” (bye 20s). Tenure took my early 30s. The things that mattered were when I showed up in love and care for others. Advocacy. Marriage. Fostering. Parenting. Worth thinking about.
6
11
186
I realized a couple years ago I stopped caring whether my hypotheses were supported. and started treating contrary evidence as a sign that we need to revise theory (vs some failure of mine). It improved my science and sense of well-being. Feeling grateful today for #openscience.
1
30
164
It took a few *years* for this to get written and accepted … but it’s a great paper and you should read and recommend it!.
“Underpowered to detect what?” should be the first question to anyone who says a “study” is underpowered
2
26
185
Ex-academic in tech, Day 2: I love my job, love my team, and love my role. I did good work today. Then, at 5 pm, I put it away. It's weird putting work away and not feeling like it's a compromise somehow. Looking forward to this.
5
3
177
Can someone please explain to me why it is *so hard* to turn what seems like a clear paper idea into a clear paper?.
15
6
165
Ex-academic in tech, day 8: all the stuff I used to do that felt like “not real work” (but I still had to do) was real work. If there aren’t enough hours in the week to do that stuff *and* teach *and* write then the problem is the work plan, not the execution.
4
10
166
Causal inference comes down to 5 rules familiar to “mediation” crowd:. 1. Control for confounders.2. Don’t control for colliders.3. Controlling for mediators makes estimate from X a direct (vs total) effect.4. Magically know the DAG to decide ^^^.5. random assign when possible.
6
22
171
Is there an “extraordinary least squares”?.
21
22
171
You get to recommend ONE book to social science phds who want to build probability and stats skills. What is it?.
34
36
165
Stats, psych, data twitter -- need your help! Industry is full of smart and ambitious people doing analytics who haven't had stats/methods training. Have regular need to direct people to the equivalent of intro stats, intro research methods, advanced iterations, etc. Where send?.
32
24
167
I have good news: my committee has voted to recommend me for tenure!.
6
0
148
@ling_sprenger Let’s normalize this. I do this occasionally. Negative feedback is commonly shared and positive feedback rarely is. Let’s share the good!.
1
4
151
Decided that p-values make much more sense to students when phrased as "% of the time." E.g., "If H0 were true, results this strong would still occur 34% of the time" . Wondering how much confusion we'd avoid if we taught it that way more consistently.
11
18
158
Thread. Totally reworked my advanced stats course this year. Rebuilt from bottom up. Now 100% analyzing open datasets. Big insights (1/n).
6
36
155
Has anyone seen something like a causal inference boot camp or online course? Eg an online course covering modern techniques for those who didn’t get much in grad school beyond regression and RCTs? I’m self-taught and want to improve my skills this year.
25
22
164
Moved advanced stats class to be R focused (instead of SPSS). Realizing I have to go slower but that’s because students are learning deeper. It was too easy in SPSS to give a 30,000 foot view with no depth. Teaching with R forces understanding.
6
19
152
Academics: not sure who needs to hear this, but responding to student emails, scheduling, and all the other things that take up your day are "work" and if you spent a day doing them, you actually did "get work done.".
2
10
134
Understanding p-values.
3
13
151
Am I the only one who finds this data visualization incredibly hard to read?.
26
5
138
Most 'data analysis' resources are technical =(e.g., coding in R or python, data engineering, etc). Wanting to help others up level into the THINKING process that analysts/scientists do naturally (problem --> helpful question --> test --> insight). What can I recommend to ppl?.
19
9
146
Thesis: the magic of Twitter was never the platform. It was that it made it effortless to find a community of people you didn’t already know. I have yet to find that on any other platform.
5
17
129
I just submitted my file for promotion to associate professor. Hands still trembling! Wish me luck!.
2
1
127
Let’s STOP using STATISTICS and start using STATISTICS.
As traditional political polling methods appear to fall short again, some tech firms say that AI holds promise via @WSJ.
5
6
134
One of the kindest things my advisor told me in grad school was that she would support my career goals, even if I wanted to decorate cakes. She’d teach me how to design experiments to make the perfect cake. Heard that in 2012 and I still find it inspiring.
The academic path. Undergrad --> PhD --> Postdoc --> PI --> bakery shop owner.
3
4
133
We need to move to a publishing/hiring/training model that emphasizes quality over quantity, teams over PIs, accuracy over prestige, integrity over speed, and importance over flash. For science to reach its potential, these should be nonnegotiable. 6/n.
2
21
130