Jason Wu
@jasonwuishere
Followers
2K
Following
4K
Statuses
414
When you ask an LLM to generate a UI, how do you know if it's good? It is often easy to recognize a good design but difficult to explain why and reason about/evaluate tradeoffs. UIClip is a data-driven model that assigns numerical scores to represent quality of UI screenshots
3
9
84
RT @hankhplee: 🚨Trapped by infinite scroll & autoplay on social media? Our TOCHI paper introduces a system that reduces distraction by 4x b…
0
6
0
RT @YuxuanL_: 🚨 Advanced LLMs may sound unbiased—but it's a ruse. We show that demographically-informed language agents reveal stark, impli…
0
23
0
RT @FariaHuqOaishi: [1/6] 🤔 Ever wondered if you could collaborate with an agent on web tasks? We present CowPilot 🐮, a framework for hu…
0
20
0
RT @michigan_AI: As we start the new year, we're excited to share the @UMichCSE Annual Report!🌟 Featuring groundbreaking work from Profs. @…
0
3
0
RT @lukas_m_ziegler: 🚨 Apple joins the robotics race! Researchers from @CarnegieMellon and @Apple have developed a robot collision avoidan…
0
183
0
RT @HongyiWang10: I have three Ph.D. student openings in my research group at @RutgersCS starting in Fall 2025. If you are interested in…
0
118
0
1
0
1
Try out our new demo where you can use UIClip to score and compare the design quality of screenshots!
When you ask an LLM to generate a UI, how do you know if it's good? It is often easy to recognize a good design but difficult to explain why and reason about/evaluate tradeoffs. UIClip is a data-driven model that assigns numerical scores to represent quality of UI screenshots
0
6
30
RT @AndreaHCI: Dear HCI and @ACMUIST community, are you interested in volunteering to be an Associate Chair (AC) for #UIST2025? Consider vo…
0
17
0
@oulasvirta @jeffbigham update on this thread!
When you ask an LLM to generate a UI, how do you know if it's good? It is often easy to recognize a good design but difficult to explain why and reason about/evaluate tradeoffs. UIClip is a data-driven model that assigns numerical scores to represent quality of UI screenshots
0
0
6
[alt text: A screenshot of our UI generation example application, which is a chat interface that returns 5 ranked UIs. And a screenshot of UI design feedback app which predicts likely "jitters" in an uploaded screenshot]
0
0
0
[alt text: A bar chart showing model performance on design choice prediction. Baselines have accuracies in the 50 percent range, which is close to random chance. UIClip variants can achieve up to 63.2 percent for the huam ratings and up to 87.1 percent accuracy for jitters]
0
0
0
[alt text: an original web page screenshot is jittered to create sub optimal variations. these are used to train UIClip, which is based on the CLIP dual-encoder transformer architecture.]
0
0
0
[alt text: A figure showing an incorrect design assessment and reasoning from GPT-4V. Two UI screenshots are arranged horizontally, where the one on the left has large text that overflows and causes readability problems. The model says the screenshot on the left is better]
0
0
0
this work was done with amazing collaborators at @cmuhcii and @Apple : @yolohao @AmandaL79938988 @a_swearngin @jeffbigham @jwnichls
1
0
3