Excited to share our work on using generative AI tutors in the classroom - we ran a RCT on 1000+ students at a large high school in collaboration with math teachers. We find that students use GenAI as a "crutch," harming their learning:
Our paper on targeted COVID-19 border testing in Greece via reinforcement learning is out in
@Nature
! We caught 2x as many infected travelers as random testing & highlight serious deficiencies in publicly reported metrics for measuring traveler risk
🚨 New paper alert 🚨: we describe the large-scale, data-driven bandit algorithm we deployed to perform dynamic and targeted COVID-19 testing of international visitors (~40k per day) at the Greek border last summer.
Consistent w/ prev lit, access to GenAI tutors improves student performance on practice problems (relative to tools like textbooks). But, students that accessed GenAI tutors perform 17% *worse* on a subsequent exam after access is taken away. In other words, GenAI harms learning!
Delighted that
@kanxu526
&
@AndererArielle
will be starting tenure-track faculty positions this fall at
@WPCareySchool
&
@Cornell
respectively!! Was such a privilege to be part of their journeys, and I can't wait to see them succeed in their new roles 😍
Sex trafficking is the 3rd largest criminal enterprise in the world, preying on millions of vulnerable victims every year. We provide a new, large-scale view into these illicit supply chains w/ massive deep web data & machine learning.
#AnalyticsAtWharton
I respectfully disagree with the tone of this. Women married to fellow faculty members (incl myself) have a harder time getting jobs (2-body problem) & are more likely to get their contributions dismissed (e.g. this thread). I am proud that
@Penn
supports academic couples
Success of women married to fellow faculty members, with their husbands among top co-authors and active collaborators on grants, teaching the same classes, doesn’t indicate change.
These women are talented and amazing but their experience is far from that of independent women.
Students seem to use GenAI as a "crutch" & simply copy answers instead of engaging substantively -- a concern prev raised w/ Khanmigo (). Despite this, students' perceptions are overly optimistic about their learning w/ GenAI & subsequent exam performance
I worked with
@Meta
last year on using multi-armed bandits to dynamically improve their content moderation. Excited that the short paper is finally up!
Was fun working on this with
@vas1089
,
@obastani
, Deeksha, Parikshit, Caner & many others at Meta CDS
These harms are largely mitigated by incorporating appropriate safeguards (incl teacher-derived inputs) into the prompt, encouraging more substantive interactions. Still, more work is needed for GenAI to create its promised positive impact of democratizing access to education
Very excited about the rockstar lineup we’ve put together for
@CODEConference
this year!
Since the conference is in two weeks, I figured I’d briefly let people know what is on the program in case you’re on the fence about attending (hint: you should attend).
More info below 👇
My department at
@Wharton
@Penn
is hiring for faculty positions this year in operations management, operations research, or analytics applied to operations. Come be my colleague!
The exploration-exploitation tradeoff in
#RL
raises concerns about exploration -- who does it impact and how much? In a recent
#AISTATS
paper, we show how to effectively "spread out" exploration across episodes (individuals) w/ only a small cost to regret:
The unrelenting pressure of conference deadlines is why I didn't pursue a career in computer science. It's sad that some people won't take the pedal off even during a devastating pandemic.
As I struggled to deal with the impact of COVID on my family members in India, I got delayed by a day for submitting my reviews for a conference & I got a message from a senior reviewer with the blurb below. My humble request to everyone - pls don't say this to anyone ever! [1/n]
Utterly shocked... Nagesh was one of the kindest souls I've met. As a PhD student, his encouragement meant the world to me. We were chatting about his new socially responsible operations course at USC just 3 days ago & he was so excited. This is an immense loss. RIP, dear Nagesh.
Shocked to learn of Nagesh Gavirneni's passing. He was a great colleague and friend. My last conversation with him was after a casual conference practice talk event: he had the nicest words to say about how much hearing colleagues present research enriched his life.
We're looking to hire an early career data scientist to work with
@dean_c_knox
,
@MarimanRachel
and me on high-impact applications of AI/OR for social good, incl improving livelihoods in developing countries, policing, mitigating bias in algorithmic recruiting & media polarization
This is important bc of AI's "jagged frontier" () -- outputs can often be wrong depending on the task & need to be checked by human experts. Failing to educate the next gen can lead to lack of expertise to check outputs & blunders ()
Thrilled to co-direct the new Wharton Healthcare Analytics Lab under AAW with
@marissadking
&
@LauraZarrow
! We're working on exciting translational research w/ govt agencies, health systems,
@PennCHIBE
& more - stay tuned! Grateful to
@ebradlow
&
@iyengar_raghu
for their guidance
As data algorithms expand and management practices continue to evolve, the need for high-quality healthcare remains. That’s why we're launching the Wharton Healthcare Analytics Lab.
Learn how
@WhartonAnlytcs
is shaping the future of healthcare practice.
Congrats to my PhD student
@KanXu08497413
for his very first paper "Group-Sparse Matrix Factorization for Transfer Learning of Word Embeddings," accepted in
#ICML2021
! Big thanks to co-conspirators Xuanyi Zhao &
@obastani
Attn CS/OR/ECON PhD students interested in
#EAAMMO
(Equity & Access in Algorithms, Mechanisms & Optimization): apply to the inaugural doctoral consortium to meet like-minded students/mentors & learn abt career paths! Co-organized w/
@brwilder
&
@JubaZiani
Really enjoyed working with Kimon, Vishal & Jon to help build the ML platform behind Greece's targeted COVID-19 testing effort for a safe tourist season.
#MachineLearning
#AI
#COVID19
Taking a page from
@thserra
's playbook, I'll attempt to live-tweet snippets from our Workshop On AI and Analytics for Social Good today with
@mvilborg2
@JuiRamaprasad
🧵...
Spilled breast milk all over my Macbook while attempting to feed my infant at 4am. Switched to an older Macbook, which she promptly kicked & splashed milk on that one too 🤦🏾♀️. Guess I will not be responding to emails after all...
Excited for the future of data-centric AI conference! Our work on counter-human trafficking was made possible by support from
@TellFinder
&
@WhartonAnlytcs
Learn how
@hamsabastani
and team are leveraging massive deep web data in tandem with a novel machine learning framework to unmask human trafficking at 👈
Gave a zoom seminar yesterday with only 1/2 my mouth (other 1/2 was numbed from an emergency dental procedure earlier that day). Feel like I've unlocked some sort of academic achievement 🏆?
+100, I read
@robothika
's article every year. Academia is a privilege -- we get to choose fun & meaningful projects w/ incredible people. Ignoring tenure helps me enjoy that while maintaining work-life balance + Netflix queue, instead of worrying abt how others perceive my work.
If you're on the tenure track or planning to be or hoping to be or even just thinking about being an academic or WHATEVER and you haven't read this, read it now
The 2022 Early-Career Sustainable OM Workshop will be held
@Wharton
this Feb 25-27! The workshop will follow a hybrid format, & we'd be thrilled to have you attend in-person or virtually :)
🚨Dataset Alert🚨: we manually digitized Kaplan-Meier curves on surrogate & true outcomes from 1,865 metastatic breast cancer clinical trials. Useful for studying bandits or adaptive clinical trial designs w/ surrogate outcomes, delayed feedback, etc.
Horrifying. How can we comfortably encourage female students to network at these conferences when behavior like this goes unchecked despite reporting to organizers?
Walter Lasecki assaulted me. He pinned me against a bar, he put his hand up my skirt and his fingers in my underwear. He grabbed and twisted. He leaned in and insisted that I sleep with him. “It could be innocent,” he said.
Check out the awesome work
@AngelTHChung
has been doing to improve health supply chains in Sierra Leone! Recent deployment improved patient access to essential meds by ~20%!
Learn how
@Wharton
PhD student,
@AngelTHChung
, leveraged machine learning to help the government of Sierra Leone distribute limited medical aid to the communities who need it most, as we continue to highlight Wharton Women in Analytics.
@femonomics
My husband (also TT) accompanied me to every Dr visit, did all the housework when I was pregnant/postpartum, took all the night shifts with baby while I pumped, etc. Giving unequal benefits feels like it would be punishing good partners - shouldn't we just set better norms?
Agree & thank you to
@nathankallus
for broaching this issue publicly. Requiring pregnant women to choose between attending our primary conference & safeguarding their health/rights is a
#DEI
issue that
@INFORMS
should strongly consider
I am boycotting
@informs2022
. We cannot ask our woman colleagues to travel to a state where their health is put at danger and their choices limited about their own bodies. I am calling on
@INFORMS
to move venues from Indiana or change to virtual-only in light of the new law.
My husband is (successfully!) rocking my daughter to sleep on one arm WHILE writing a paper on Overleaf on the other. Meanwhile I try to respond to ONE email while feeding her and destroy two laptops... 🤦🏾♀️
Spilled breast milk all over my Macbook while attempting to feed my infant at 4am. Switched to an older Macbook, which she promptly kicked & splashed milk on that one too 🤦🏾♀️. Guess I will not be responding to emails after all...
INSEAD (
@INSEAD
) and Wharton (
@Wharton
) are conducting a study on AI and algorithmic advice in chess. The study aims to provide insights into how AI and algorithmic advice and their effect on sequential human decision-making. The research, grounded in chess, will offer valuable
We are very honored to have been named the winner of the
#INFORMS2021
Doing Good with Good OR competition!
All finalists had inspiring work: vaccine development+distribution, citizens assemblies, COVID supply distribution.
🎙 Come to our live talks Tues 4:30pm PT in room 211A!
The EC Workshop on the Design of Online Platforms will be held on July 23. Greg Lewis (MSR) and Vivek Farias (MIT) will give keynotes. Please share your work on theoretical, empirical, and experimental issues in design of online platforms (due May 21)!
If you're considering a PhD / academic career, please apply to IDDEAS
@Wharton
: a 2-day program, April 20-21,2023, introducing outstanding undergrads with a strong interest in math/stats/econ/psych/engineering to business research & academia.
Enjoyed discussing work with
@HaosenGe
&
@obastani
on rethinking algorithmic fairness for real-world human-AI collaboration
@mit_ide
-- their article () contextualizes these challenges & outlines how we can address them w/ better algorithms
@WhartonAnlytcs
Submit your work to the Workshop on AI & Analytics for Social Good (due 3/15), to be held @ UMD on 5/5! Learn about privacy, policing, & hiring bias from experts
@ssnstudy
,
@radcummings
, Nick Freeman,
@dean_c_knox
, & Danielle Li!
⭐️Considering a PhD & an academic career? Love math/stat/econ/psych? Come learn about business school doctoral programs thru the annual IDDEAS program
@Wharton
(held in Philly, 4/18-19)!⭐️
Apply by 1/31:
Why does changing poopy diapers for infants have such a bad rep? Their poop doesn't even smell bad & it's a great time for interaction (huge smiles if you're lucky). Feeding and putting them to sleep seems like the exhausting part 🥱
As this year comes to a close, we look back on our enriching student programming, exciting in-person events, impactful research, and more!
Thank you for your continued support and engagement. We wish you a happy and healthy 2023!
How can the same LLM pass AP math and also claim 9.10 > 9.9?
Our
#ICML2024
paper: LLMs are wrong in ways that people can't predict. Benchmarks can be misleading because people decide how LLMs are used.
Solution: measure LLM alignment with the "human generalization function" 🧵
Amazing advice, never thought about it this way but I've been following a similar strategy after seeing ppl fall asleep in my talks one too many times 😅
Random presentation tip for econ/finance PhD students, esp. for technical/theory ish stuff:
When I first started presenting, I assumed a distribution of audience expertise that looked like this. Roughly normally distributed, most people know something about your topic matter
awesome end to the world chess championships!!
ding liren has only a minute left to play, everyone predicting he'll take the safe draw
he goes for the win instead
absolutely fearless! why we still watch humans play a game computers beat them at.
📢 📢 We're thrilled to announce the inaugural ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '21)!
This conference grew out of the MD4SG workshop series and will build on the same themes.
Pronounced emo :)
"And always be grateful but never forget that the system is rigged so find a way to acknowledge that but also always be grateful." -- loving the
@AmericaFerrera
monologue in
@TheBarbieMovie
Shout-out to
@oziadias
for extremely helpful comments on earlier drafts as well as this kind editorial on our work, anonymous review team, &
@obastani
for helpful feedback! It really does take a village 😊
We're hiring! If you have project-management skills and are interested in social impact and computational social science, I'd love to chat. Joint position working with
@hamsabastani
and me. Reach out to either of us or apply directly at
Same! I always tell my female students that work-life balance isn't bad when you have an incredible partner that steps up everyday, thankful for
@obastani
My husband makes me dinner every night, packs my lunch every day, and does the cleaning and laundry. Just putting this here for all the women, especially those in surgical residencies - men who support women do exist.
Work made possible by incredible co-authors Kimon Drakopoulos, Vishal Gupta,
@jonromero
& Greek COVID-19 taskforce, as well as the support of the Greek PM, border control agents, doctors, nurses, lab personnel, etc
📢 Our fourth mentoring workshop will be held on Oct 26-27, 2022, in collaboration with FODSI. Registration () is open and free for all. See the schedule at .
#MLTheory
[1/4]
Next up is
@ssnstudy
talking about "Who benefits from the data economy?" -- he starts off with the privacy paradox: even though 95% of people say they care about privacy, 95% will also give their DNA sample for a big Mac
The 2022 Sustainable Ops Workshop will be held virtually Feb 26-27. We have an exciting line-up of 11 full talks + 8 flash talks; see agenda @
Register here:
After giving a talk at a conference recently, I received an email from an unfamiliar senior professor in the audience remarking that my “look has extremely improved over the years” from “nerdy lab mouse” to “a touch of upper class.” 1/4
@NikhGarg
@myopicagent
@NickArnosti
+1, thanks
@NickArnosti
! I also love the
@NCFDD
's "Monday Motivator" emails -- they touch on many of these issues (like aligning your time with the end goal and avoiding perfectionism) that many of us struggle with.
@jetnew_sg
@emollick
&
@lilach_m
have work on prompt constructions to encourage engagement. We built on this & found we also needed to add a ton of problem-specific teacher input (solutions, hints to give, etc); see examples below. Designing these is effective but hugely labor intensive
Classic "conservative" algorithms may concentrate exploration on a small # of individuals; alternatively, "shielding" may fail to learn the optimal policy. We introduce a new adaptive meta-episodic strategy to efficiently explore w/o excessive exploration cost to any individual.
It has been gut-wrenching to study these atrocities (>50% of victims in recent prosecutions are children). I truly hope modern ML tools can help our heroic but resource-constrained counter-trafficking units better protect our vulnerable populations.
Kicking off the next session is my colleague
@dean_c_knox
on "Evaluating bias in policing" -- this is a hard causal problem & we often don't have the ideal data to evaluate counterfactuals due to limited visibility (e.g. did the officer have probable cause for an encounter?)
@roydanroy
@Penn
Ofc I agree & I don't know any female faculty who are not pushing hard to hire/promote talented junior women. I don't think it's fair to imply that women in academic couples have it "easier" because they might collaborate or co-teach w/ their spouses
Instead, can we get nonparametric sharp bounds w/ the messy data we do have? Transparent assumptions + characterize best & worst case scenarios.
@guilhermejd1
&
@dean_c_knox
show any discrete causal inference --> tractable polynomial program. Automatic causal inference!
Closing up privacy discussion is
@radcummings
talking about "Improving technical communication with end users about differential privacy" -- she argues that the probabilistic guarantees given by diff privacy on inference by adversaries is entirely uninterpretable to consumers
@rad_niazadeh
In ML conferences, reviewers are not *required* to read the supplement (proofs) whereas we are supposed to check correctness for journals. It's harder to get the time to sit down and understand a long proof