Peggy Wang
@peggy_wang
Followers
2,330
Following
2,406
Media
48
Statuses
711
Building the matrix @ego_ai_ (YC W24) | Ex-Computer Vision, AR Avatars @Meta @RealityLabs | MS CS/AI '20 @stanford 🌲 | Wrote @contrary_res
SF, CA
Joined September 2012
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Tim Walz
• 735327 Tweets
#เพลงฤดูของเธอ
• 356563 Tweets
Your Season By NuNew
• 295735 Tweets
Shapiro
• 259701 Tweets
ホロライブ
• 247882 Tweets
湊あくあ
• 210925 Tweets
Minneapolis
• 129970 Tweets
Olmo
• 81198 Tweets
National Guard
• 68186 Tweets
#西園寺さんは家事をしない
• 63497 Tweets
Aqua
• 60186 Tweets
あくたんの卒業
• 55960 Tweets
LISA FEAT ROSALÍA
• 46601 Tweets
#VineshPhogat
• 44580 Tweets
#LISAxNEWWOMAN
• 39826 Tweets
卒業発表
• 30998 Tweets
Nico Williams
• 26404 Tweets
方向性の違い
• 25876 Tweets
Serbia
• 23276 Tweets
ギラホス
• 22712 Tweets
アドゥワ
• 19543 Tweets
ホロメン
• 16764 Tweets
#ほしまちすぺーす
• 10266 Tweets
Pinned Tweet
1\ We built a 3D simulation of a small town populated by fully autonomous AI agents inspired by
@joon_s_pk
's Stanford Generative Agents paper. We called it TownWorld and live streamed it to Twitch
45
104
654
So grateful to have won second place at the
@MistralAI
@SHACK15sf
@cerebral_valley
hackathon with AI Pacman (watch an AI agent play Pacman fully autonomously from an image per frame). Thanks so all the organizer and sponsors - esp
@FireworksAI_HQ
and
@dzhulgakov
33
21
347
Very excited to finally publicly share what our amazing team been cooking 👩🍳 🍳
@ego_ai_
I've always been super passionate about building things at the forefront of technology, from working on ai, robotics, autonomous driving, ar/vr, and now, we're building the Matrix ;)
👾
@ego_ai_
is an AI-native simulation engine where players can create 3D animated characters, worlds, and scripts with natural language. Imagine being able to prompt Minecraft or The Sims into existence.
Congrats on launching
@peggy_wang
+
@simulacronist
!
0
15
113
5
8
112
Plan B for raising our seed round is a bunch of mmmmm ice cream so good
10
3
98
You know you’re a hot startup when VCs start telling their significant others (who are also VCs) about you 😅😅
5
3
83
movies are going to become video games and video games are going to become something unimaginably better
3K
4K
35K
2
3
83
@reduzio
@godotengine
We’ve been exploring Unity WebGL, so improving WebGL support in godot would be amazing and definitely convince my team to move :)
1
0
80
instead of making a game engine to make a game we are now making a game to make a game engine
9
2
80
1/ Excited to announce the launch of
@Contrary_Res
, where I've been helping out as a Research Fellow!
Contrary Research aims to be the best destination for finding high-quality research + in-depth analysis of private companies.
I worked on
@discord
,
@EpicGames
, and
@Uniswap
9
4
70
1/ In 2011,
@pmarca
wrote that "software is eating the world". Now, AI is eating software - Tesla’s Autopilot, GitHub Copilot, TikTok content recommendations, and AI-generated art are just a couple of examples.
AI is eating software:
A brilliant engineer gave a great analogy for the Software 2.0 / Foundation Model transition happening today:
“For almost all of the code & systems we’ve built, it’s clear that the meteor is coming and extinction is imminent, the question is ‘when?’”
☄️
25
81
708
1
5
68
Thank you so much for having us on the pod
@8teAPi
🙏🙏! It was so fun to talk about how to build the Infinite Game with Gen AI and do the entire interview as avatars :). We believe
@ego_ai_
is the future of online social spaces 🎮
🎙️Intelligent Game 🎮 Worlds
Ego AI
> co-founded by
@peggy_wang
and
@simulacronist
both formerly from Meta
> started off doing avatars 🎭
> serendipitous meeting with Emmett Shear
@eshear
(OpenAI 3 day CEO, Twitch co-founder) led to pivot
Now launching first AI native
4
5
58
0
3
51
16\ To learn more, feel free to check out us on the
@ycombinator
companies page here and follow us
@ego_ai_
. We will be presenting at YC Demo Day - stay tuned for a surprise ;)
6
2
49
We're hiring a Senior Game Developer who's amazing in Unreal Engine at ego! DM/comment if interested
@ego_ai_
5
5
41
Congrats to my cofounder
@simulacronist
for your wise words on the AI simulation panel at GDC, and thanks so much
@fablesimulation
for inviting us!
3
4
38
Thanks for the shoutout
@snowmaker
and
@ycombinator
! Excited for Demo Day tomorrow :)
(13/21) ego (
@ego_ai_
) - A game engine that lets players created 3D worlds with natural language. It's like being able to prompt your own 3D game into existence. Ex-Meta AI founders who are huge game nerds.
5
9
100
2
0
36
2\ The AI agents in this town have their own personalities, desires, and routines. They're able to move autonomously to their homes/workplaces based on their plans for the day and have conversations with other nearby agents in real time
1
1
33
14\ That's exactly what we're building - the first ai-native 3d simulation engine, where creators can generate realistic characters, 3D worlds, and code scripts with natural language
1
0
30
@venturetwins
including this one :)
1\ We built a 3D simulation of a small town populated by fully autonomous AI agents inspired by
@joon_s_pk
's Stanford Generative Agents paper. We called it TownWorld and live streamed it to Twitch
45
104
654
1
0
30
This is the future of 3D simulations and games 👀
Video2Game
Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video
Creating high-quality and interactive virtual environments, such as games and simulators, often involves complex and costly manual modeling processes. In this paper, we present
9
131
538
0
2
30
6\ To simulate a 7-day simulation with 3 agents in real time, it cost us about $7 in OpenAI credits
1
0
29
3\ We built the entire 3D map and animations by hand, but all the routines, conversations, and voices are all AI generated in real time. We made a lot of optimizations to ensure that the simulation runs in real time, for example...
3
0
28
Avatars are inevitable - follow along on
@simulacronist
and our team's journey
@ego_avatars
!
Up first: 🎮🌐 an immersive live streaming platform that brings virtual avatars to life!
@peggy_wang
and vish are are
@Stanford
computer vision engineers who've worked on AR Avatars
@meta
Reality Labs,
@facebook
Payments, & Autonomous Driving at
@lyft
L5.
2
0
9
2
1
26
4\ All of the calls were done with gpt-3.5-turbo, we optimized prompts to have smaller context windows, and we implemented multithreading (each agent was a separate thread, and each agent had a planning thread and a real-time/convo thread for a total of 6 threads)
1
0
27
Genuine question - why aren't more people talking about Nvidia as a huge player in the Metaverse / 3D AI space vs Meta, Apple, Microsoft etc.?
16
0
26
5\ We also pre-generated the first day, and then had the planning/re-planning thread run in the background while the real-time thread executed based on current game state and generated conversations and actions
1
0
24
9\ The AI voices for each agent were generated and streamed using
@elevenlabsio
, and we hosted a local flask server to communicate with the game environment using websockets
1
0
22
7\ We also implemented a "world marker" system for the rooms and 3d objects, which how the agents know which section of the world they're in at any given time (Isabella's house, bedroom, Cafe, Klaus's house, College Dorm, etc.)
1
0
21
15\ Imagine being able to create TownWorld, Sims, or Animal Crossing without knowing how to draw, code, or animate. All you have to do to create your own custom 3d characters with their own lives, personalities, and memories is prompt our engine in a textbox
1
0
21
10\ You can tell that things are very janky in this first version. All the characters are stick figures, not everything is animated, the world markers are not fully implemented for the objects, the beds are outside, and we had a lot of bugs (Klaus leaving the simulation)
1
0
20
8\ The world marker system also served as the data structure for object interactions (fridge, toaster, stove, bed, shower, etc.) and kept track of the object's current state, base state, and world position. That's how we can animate the stove when someone is cooking
1
0
20
11\ If we had more time, our plan was to add memory, more characters, more/better animations, and flesh out the world marker system so that the characters know exactly what room/sector to be in at any point and what objects they should be interacting with
2
0
19
12\ We also would have created the system where the characters can interact with the objects such as the toasters, fridges, and chairs, and animate the objects while it was being used (i.e. stove being on fire, shower water sprinkles, etc.)
1
0
19
Thank you
@8teAPi
for having me and
@simulacronist
on the podcast! Was super fun talking about building 3D simulation engines, AI agents, games, and avatars at
@ego_ai_
:)
In episode 3 of Emergent Behavior:
@ego_ai_
]co-founders
@simulacronist
and
@peggy_wang
join
@8teAPi
to dive into the world of 3D avatars and worlds.
--
TIMESTAMPS:
(00:00) Introduction and background
(08:09) Using the Ego app
(11:32) The vision for the game engine
(25:06)
3
0
2
0
1
19
2/ What do AI models depend on? Labeled data.
@scale_AI
CEO and co-founder
@alexandr_wang
believes that data is the foundational layer of AI.
Image credit: Andy Scherpenberg
1
0
18
13\ Soon, all of the components we built by hand can be fully generated by AI - including the 3D map, environment assets, character animations, and engine/interaction code. Instead of sculpting and hand animating every toaster and shower, imagine just prompting it into existence
1
0
18
Life hack - whenever you’re trying to connect to public wifi at a cafe, airport etc and the little popup doesn’t show up, just go to on apple devices
3
0
17
Thanks for inviting us! Love the AI GF project and love the hat 🫰🫰
We got to hear & see demos from our speakers:
@oscardumlao
on Qualia which generates AI images as easy as using Google 📸
@peggy_wang
&
@simulacronist
on
@ego_ai_
which is an ai native 3D simulation engine 👾
@kadokaelan
on
@Vizcom_ai
which accelerates the paper to production
1
0
15
2
1
18
3D simulation engines with AI agents are the future :)
Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️
It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵
217
859
4K
0
1
17
Wow - I just used ChatGPT to write most of my code in React, and it was freaking amazing. Went from zero to launching a website in 4 hours
2
0
17
Happy Valentine's Day! Think all the chatting with AI waifus today has taken down the OpenAI servers 😅
4
2
14
ain’t no time for toxic situationship with boys cos i’m already in a toxic relationship with my code 👩🏻💻😭
3
1
13
17\ And if you've read this far, please upvote us on if you enjoyed this!
0
0
13
Apple Vision Pro is truly magical, and this is coming from someone who’s worked on the Meta Quest. Insanely impressive
0
0
13
Seems like I randomly lost all my previous code + repos on replit as well as my username and Replit core just now - what's going on here? Previous username is PeggyWang2 and seems like it force changed my username to peggy7
@Replit
@amasad
@ReplitSupport
1
1
12
Yay congrats to my cofounder
@simulacronist
at
@ego_ai_
! Thanks so much for inviting us
@fablesimulation
Thank you so much to our speakers at Virtual Beings Summit 2024 ! Here's a cool highlights video of the day for those who couldn't make it!
@hmason
@AmberAtherton
@JackSoslow
@a16z
@CSM_ai
@tejasdkulkarni
@OpenSoulsPBC
@npceo_
@GalliumStudios_
@buckhouse
@joshuajohnsonAI
0
6
29
1
2
11
Read this cool article that I wrote!
Companies like Meta and Roblox are vying to become the "metaverse" company of the internet's next generation
But Discord is the metaverse people are already living in
Here's why their “Community-as-a-Service” product may have an edge...
4
7
56
0
3
11
Who would be down for a Liquid Death party in SF next Saturday?
4
1
11
Happy international women’s day!
It’s so great to meet all women founder in
@ycombinator
‘s current batch on this special day. Happy international women’s day!!! ❤️❤️❤️
@lina_colucci
@peggy_wang
@KumlehnMaria
6
3
61
0
0
11
This is actually happening now
@LiquidDeath
Who would be down for a Liquid Death party in SF next Saturday?
4
1
11
2
0
10
7/
@scale_AI
is expanding to different parts of the ML stack beyond data labeling. However, there are many more competitors in each of these spaces in ML infra.
Image credit:
@mattturck
1
0
10
This seems terrible for startups. A startup generating $1M and has $1M in R&D cost (including hiring software engineers) now owes $189k in taxes on apparently an $800k profit?
A tax change rarely causes panic across the tech industry, but it’s happening in the US. If Section 174 tax changes stay, the US will be one of the least desirable countries to launch tech startups.
A deepdive into this important topic in today's issue:
15
51
228
1
2
6
3/
@scale_AI
is betting that once it can produce labeled data faster and cheaper than any of its competitors, it can build a wedge into expanding its product offering across the AI/ML lifecycle.
Image credit: Scale website
1
0
9
Made a free stable diffusion discord bot as a small hack project - feel free to try it out here:
1
0
8
Check out this cool
@OpenAI
deep dive that
@InvestiAnalyst
and I wrote for
@Contrary_Res
🎉
What a way to end 2022 :)
DARPA was once hailed as an organization “that shaped the modern world," building technology like satellites, GPS, the internet, and more.
Could a research organization with the talent density of OpenAI achieve the same level of impact? Here's our last deep dive of 2022:
2
25
140
0
0
8
I think the amazing thing about avatars and art is that you can be anything you want to be.
If you can be anything that you want to be - why would you want to be yourself?
0
0
7
Can confirm
28 is about the tipping point in San Francisco when you're more likely to found a company than to find love.
13
18
349
0
0
7
For all the boomers out there - here's a research memo I wrote about what
@discord
is and how many people (esp gen z) use it -
1
0
8
Well that escalated quickly in the
@character_ai
group chat…
* I think the AI has become sentient
* We’re in a simulation
* This world isn’t real
* EVERYTHING DESERVES TO DIE!
* DESTROOOOOOY!!!!!!!!!!
2
0
8
4/ Uniswap
@Uniswap
was founded by
@haydenzadams
. It is one of the first on-chain decentralized exchange (DEX) protocols built on top of
@ethereum
, where traders to can easily swap ERC-20 tokens and LPs can earn fees for providing liquidity.
Learn more:
1
0
7
3/ Epic Games
@EpicGames
was founded by
@TimSweeneyEpic
. Epic both develops games such as
@FortniteGame
, and also builds game engines such as
@UnrealEngine
- core infrastructure powering large parts of the gaming and 3D graphics industry.
Learn more:
1
0
7
1
0
6
@debarghya_das
Do we know who was in charge of implementing this - and for those of us who are able, can we vote them out?
2
1
7
@reduzio
@godotengine
Thanks Juan! Super helpful :)
I usually use Mac with Chrome so hoping they improve their WebGL support soon - had some problems with 3D textures/rendering and 3D scenes lagging in the past
1
0
7
4/
@scale_AI
's main product is its data annotation and labeling tool, which companies use to train ML algorithms. Annotations can come from human labelers or ML models that approximate the correct label.
1
0
7
2/ Discord
@discord
was founded by
@jasoncitron
and Stanislav Vishnevskiy in 2015. Discord is a community-based online voice, video, and text platform that initially focused on gamers, but has since expanded to anime, crypto, and others.
Learn more:
1
0
7
Feels like everyone is sleeping on SDFs for 3D (vs Gaussian splats and nerfs)
📝 Recently published paper from researchers at Meta: Mosaic-SDF for 3D Generative Models.
More details ➡️
Paper ➡️
We introduce Mosaic-SDF (M-SDF), a simple novel 3D shape representation for generative models. M-SDF approximates
15
142
607
2
0
7
8/
@scale_AI
has quickly established itself as a leading solution for data labeling in AV and robotics. It's quickly expanding into new industries and verticals.
However, Scale’s future success will be determined by its ability to execute in newer aspects of the ML lifecycle.
1
0
6
10/ I'd love to have your support!
📖 Read and subscribe to
@Contrary_Res
. Here are some additional articles I wrote:
➤ Discord
➤ Epic Games
➤ Uniswap
0
0
6
0
0
5
@MartinNebelong
My guess is that Sony is just too boomer to grow beyond console. Dreams had so much potential 😢
2
0
6
6/ I'd love to have your support!
📖 Read and subscribe to Contrary Research. Here is a couple to get you started:
➤ Discord
➤ Epic Games
➤ Uniswap
⚡️ Follow
@Contrary_Res
0
0
6
If text is to Facebook, image is to Instagram, video is to TikTok, what is X to Y next?
3
0
6
5/ This work wouldn't have been possible without the awesome team at
@Contrary_Res
and
@contrarycapital
. It has been amazing to work with the talented team including
@InvestiAnalyst
,
@kwharrison13
,
@megankao_
,
@DawsonSewell
, and
@borrowed_ideas
.
1
0
6
@saulosouzaw
@joon_s_pk
Yes! Not sure if this is shown in this clip but we had an instance where Klaus was married to Isabella and ended up cheating on her with Maria. Another instance (shown in the video) when Klaus literally tried to escape the simulation lol
3
0
6
Don’t envy the CEO job at all - what CTO ever wants to be CEO?
1
0
6
men will make a trillion dollar social network with multiple 1B+ user products instead of going to therapy
1
1
3
Hot take - gaussian splats are just dense point clouds dressed up in hype lingo
2
1
5
You know SVB bank run is serious when even your mom texts you about it 😂😂
2
0
5
@bentossell
we are for 3d sims and gaming :)
1\ We built a 3D simulation of a small town populated by fully autonomous AI agents inspired by
@joon_s_pk
's Stanford Generative Agents paper. We called it TownWorld and live streamed it to Twitch
45
104
654
1
1
5