Vivian Liu
@viv_lavida
Followers
1,020
Following
228
Media
33
Statuses
77
student researcher at @GoogleDeepmind CS PhD student @Columbia . previously @AdobeResearch @ADSKResearch 🌷
NYC
Joined May 2020
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
McDonald
• 1205536 Tweets
الهلال
• 689651 Tweets
سالم
• 152360 Tweets
Liz Cheney
• 92265 Tweets
Neymar
• 92110 Tweets
علي العين
• 87268 Tweets
AFIP
• 54843 Tweets
نيمار
• 42163 Tweets
البليهي
• 35891 Tweets
#الاهلي_الريان
• 31392 Tweets
Paul Di'Anno
• 28411 Tweets
curitiba
• 27282 Tweets
زعيم اسيا
• 26988 Tweets
Iron Maiden
• 26100 Tweets
Chris Kaba
• 24466 Tweets
كوليبالي
• 24293 Tweets
جيسوس
• 22753 Tweets
سفيان رحيمي
• 15588 Tweets
Samantha Irvin
• 15465 Tweets
Simone
• 13830 Tweets
سافيتش
• 12708 Tweets
Yulia
• 12192 Tweets
علي الفوز
• 11857 Tweets
الشوط الاول
• 11394 Tweets
الاتحاد الاسيوي
• 10246 Tweets
ever since i was little, people in my family have relied on me for english help. last week my dad suggested my mom use ChatGPT when i didn't reply fast enough. because it worked so well now one of my jobs as a daughter has been automated away :')
#ChatGPT
7
12
365
📰 Back in April, I wrote a paper w/ Han Qiao and prof. Lydia Chilton combining GPT3 & text-to-image AI (VQGAN+CLIP) in one user interface to help news illustrators create graphics for news articles.
Paper:
9
50
279
there was a time in my PhD when I thought I would only make chairs. this is a user interface dust bunny from that era (from my first year, when I was exploring connections between language and 3D) 🌬️🐰
magic color math function by
@hybridherbst
:)
10
20
277
Delighted to present 3DALL-E: Integrating Text-to-Image-AI in at DIS’23 at CMU tomorrow in the Creating with AI session! This work was done with
@jozilla
, George Fitzmaurice,
@JustinMatejka
at Autodesk Research.
5
12
91
Will be back in SF / the Bay for
#UIST2023
. 🌷🐕 This was my first time being on an organizing committee as a web chair, and I got to work on the logo with
@jeremywrnr
. Since it involved generative tools (my research area), I thought I would share a snippet of the process.
3
7
78
I’m open sourcing Generative Disco a text-to-video music visualization UI: Huggingface Spaces: . Code: thank you
@_nateraw
+ HF for the GPU grant that makes this accessible and free 🤗.
3
17
58
🪩 Generative Disco is one of the first to explore human-computer interaction problems surrounding text-to-video systems and assist music visualization through generative AI.
Here are examples: 🎵 - Clair de Lune
Arxiv: Youtube:
5
9
45
@_akhaliq
💃 🪩 🕺🏻 introducing generative disco, a text-to-video system for music visualization.
thank you to collaborators 🌷
@taolongg
@_nateraw
@LydiaChilton
arxiv: youtube:
1
8
35
the best part is when your mom explains that she discovered prompt engineering (ChatGPT "please edit: ..."), and you try to tell her that that's what you've been researching for the past two years
1
3
25
alternatively- the gummy bear and chicken nugget era of my PhD 🍒🧸🍡
I was interested in neurosymbolic approaches at the time (parameterized, semantic axes)
0
0
20
I will be presenting this work in November at UIST2022
@ACMUIST
. So stay tuned😊 On another note I'm usually so Twitter shy it's so funny to me that I wrote like a dozen tweets 👁️👄👁️
0
0
18
I finished dancing for the Columbia Dance Team this year. We performed halftime at football and basketball games. I really cherished the opportunity to perform while doing the PhD❣️
1
0
17
In this paper, we had prompt suggestion pipelines. From news article text, GPT3 generated related keywords, tones, styles, and icons. Prompts were parameterized by subject / styles, and automatically generated for the user
1
1
10
Takeaway 1️⃣ for text-to-image UI: Scaffold + suggest prompts for users. Leaving them with just a text box just leads to random trial and error; it doesn't help users get a mental model of AI's abilities.
1
0
9
Takeaway 2️⃣ is that showing users a stream of diverse generations helps. Simply seeing more helps navigate the stochastic nature of gen. AI + its lack of control. Users can feel control + find more usable material when they choose from more generations
1
0
8
I'm also very humbled and grateful that
@erichorvitz
reached out and featured 3DALL-E in his Tanner Lecture at UMichigan.🙂
0
1
7
What is kind of funny is that the minimal/ablated baseline that we compared it against is really similar to
#dalle2
's interface--a version with just a textbox. 👀
1
0
6
This is still a work in progress for sure. The way Disco is designed isn’t really the best for Huggingface spaces (many users to edit the same state 🫠), so if you really want to use it in a dedicated way, it’s probably best to duplicate or clone for persistent work.
1
0
6
Open sourcing is a learning process for me, and this took a lot of effort as I was bouncing around places for the PhD. Thank you everyone I had a discussion about open source. I welcome any and all improvements. (Also, I love Docker! 🐳)
0
0
6
Other wonderful things in the paper: we measured prompt complexity (how complex should prompts be in a UI?), proposed prompt bibliographies as a form of tracking human-AI design history, and potential impacts on product/ industrial design, mech engineering, etc.
1
0
5
🪩 Generated Disco helped express meaning through transitions in color, subject, style, time. Video could hold around certain images to encourage consistency + visual emphasis.
🎵 - Empire State of NY - Alicia Keys
Arxiv: Youtube:
0
1
6
Takeaway 3️⃣: Incorporating GPT-3 into our system performed well + significantly mitigated human effort. Making large models (
#gpt3
,
#vqganclip
) interplay is about finding their common conceptual spaces or ways the output of one → input of another
1
1
5
✨ fun fact: TechCrunch picked it up a few days later after I put the preprint on arxiv "New research out of Columbia University..."
1
0
5
🪩 Professionals mentioned that Disco could be a source of stock footage, video loops, stylized animation. Here a participant used Disco's output as the bg of a lyric video.
🎵 - Lucky Ones by Lana Del Rey
Arxiv: Youtube:
2
0
4
There was definitely still a ton of trial and error / exhaustive search. But there was also still a graphic designer that was paid at the end of the day. (wasn’t me though, lol)
0
0
3
@michinebeling
@hybridherbst
thank you for the motivating comment and stopping by my circus of shapes! 🎪 it was very nice to meet you at UIST, glad we got to be a part of the organizing committee together :D
0
0
1
Dancing this season also meant a lot because when I started the PhD during the pandemic, the only dancing I could do was on Zoom and in my room. Three years later — I’ve danced to full stadiums, basketball courts, and ESPN+ 🥳
0
0
2
@plopesresearch
@hybridherbst
@RanaHanocka
thank you so much!! 🌺 I checked out 3DL and I am a big fan of all the work the lab’s doing with language, 3d, and stylization (+point cloud reconstruction, literally everything I wished for in 2020) —plus
@allthatissolids
is there!
0
1
2
Videos are created by warping between start and end generations to the frequency of music. Disco builds on
@_nateraw
's brilliant stable-diffusion-videos repo.
🎵 - One More Time - Daft Punk
Arxiv: Youtube:
1
1
2
p.s. we are looking to recruit people to use a text-to-image user interface we've been developing too so feel free to DM if you're interested!
0
0
2
@imanolooo
@hybridherbst
it’s a minimap over the latent space! 🗺️ I did dimensionality reduction over the embeddings and down projected into 2D space with t-SNE. :) and if you “cache” it, you can have real time exploration
1
0
1
Art project, made this during early pandemic with StyleGAN transfer learned on a nebula / NASA dataset 🎆
0
0
1
0
0
1
🪩 Generative Disco lets users make music visualizations in intervals, which are parameterized by start and end text prompts. Prompts are brainstormed with GPT-4.
🎵 - Beauty and Beat - JB and Nicki Minaj
Arxiv: Youtube:
1
0
1