Sumit Gupta
@SumitGup
Followers
3,161
Following
864
Media
969
Statuses
7,228
Lead Apple Cloud Infra products, ex-VP/GM Google, IBM, NVIDIA for AI & GPUs, #DataCenter
Sunnyvale, CA
Joined July 2011
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
North Carolina
• 927844 Tweets
Hurricane Helene
• 703802 Tweets
#FreeMorara
• 363975 Tweets
FEMA
• 353514 Tweets
Dikembe Mutombo
• 144249 Tweets
Mets
• 92424 Tweets
Mets
• 92424 Tweets
Game 2
• 87140 Tweets
Diaz
• 87109 Tweets
Mimi
• 74593 Tweets
Verizon
• 74450 Tweets
#النصر_الريان
• 53264 Tweets
#الاهلي_الوصل
• 44024 Tweets
#KYSvBJK
• 33349 Tweets
Kemp
• 31067 Tweets
#KızılGoncalar
• 23944 Tweets
Kayseri
• 22956 Tweets
Onana
• 22571 Tweets
Semih
• 18761 Tweets
الجيش اللبناني
• 16546 Tweets
رونالدو
• 14880 Tweets
Lindor
• 13352 Tweets
Gavin Creel
• 11750 Tweets
Bournemouth
• 11293 Tweets
Southampton
• 10945 Tweets
Ozzie
• 10555 Tweets
Pinned Tweet
Super excited to share that my daughters (high school seniors) published this cookbook for teens (and adults!). Kudos to their mom on motivating them that cooking is a life skill. Please do support them by buying a copy (great gift!) & leaving a review
2
1
17
Proud to say that this was architecture proposal was my biggest contribution at Google. Much cheaper at scale and scales much better. Needs excellent scaling software to take advantage of this system architecture. But ideal for very large LLMs.
Google has moved away from big chips to swarms of smaller, cheaper chips with huge interconnect & datacenter connect (Jupiter) to scale to millions in a run
Trillium has 32GB VRAM per chip but pods are 256 so 8.2 TB per pod (great for long context), likely each chip 1/3 H100
11
46
341
8
18
343
IBM is exiting the face recognition business
2
100
312
You know you live in the Geek valley aka Silicon Valley, when the local
@SCCgov
Santa Clara county government puts up signs like this.
#COVID19
#WearAMask
5
69
227
Fun and informative video about TPUs in the data center, with some really great shots of our TPU supercomputers in the data center
4
28
166
The optical switches connecting our TPU Pods give Google a massive scaling advantage for TPU Pods, that enables us to train large language models (LLMs) like Bard / LamDa at a much lower cost than GPUs.
2
26
142
The
#AI
chips are falling. First Mythic laid off 50% of their staff in June, now Graphcore is in trouble. The cliff to build a competitive AI chip is very high, whether its training of inference
#MachineLearning
7
45
135
@melissasavenko
@Donn1nger
I agree. Agents provide a valuable service. But that service is worth $10-25K per agent. Not a percentage of the home price, which in my area is >$3M. Also I think a new system will develop soon that will work just fine, but realtors will no longer make a big payout
8
1
124
.
@Google
’s TPU v4 ML supercomputers set performance records on five benchmarks, with an average speedup of 1.42x over the next fastest non-Google submission, and 1.5x vs our MLPerf 1.0 submission.
#ml
#google
#deeplearning
2
26
88
Amazing story about how Thailand prevented
#coronavirus
from gaining a foothold. 95% of people wore masks and if you didn't, you were shamed by people around you.
#Mask4All
#COVID19
@jeremyphoward
5
28
64
CS is completely dominating mindshare among engineering programs in high schooler's minds - I see the anecdotal information as my own kids and their peers consider college. Its almost like the death of Moore's law is followed by the death of EE too!
10
24
55
First time in history you can save humanity by doing nothing and lying in front of the TV....
DON'T SCREW IT UP..🥃
1
14
49
Interesting project: "ZLUDA is a drop-in replacement for CUDA on Intel GPU. ZLUDA allows to run unmodified CUDA applications using Intel GPUs with near-native performance."
Now, Intel just has to produce a decent GPU .....
1
7
44
The interesting thing to see is that VCs spent over $2 to $3 Billion on new
#AI
#DeepLearning
chips (besides big investments from Intel, Qualcomm, others), and after 4+ years, none of them have a production chip in the market that is competitive. Really cuts through AI chip hype
A summary of some of the huge investments in AI chips from
@Google
, but not including Google's investment in its own TPUs.
#AIHardwareSummit2020
3
4
23
9
17
38
One key premise for the long term future of
@Tesla
is based on leading in AI, given that electric cars are now a commodity. And now Tesla is way behind on AI and so Musk wants everyone else to slow down 🤣🤣
1
7
37
Google Cloud announces our next generation TPU: Cloud TPU v5e
TPU v5e delivers up to 2x higher training performance per dollar and up to 2.5x inference performance per dollar for LLMs and gen AI models compared to Cloud TPU v4.
0
6
36
I have to say, I feel guilty of doing this too -- I too, am Anakin, assistant to many people
3
10
35
Love it when analysts make up random large numbers for a market. $3 trillion was taken by some other market, so they went with $8 trillion
Morgan Stanley: The metaverse is an $8 trillion market that will replace the mobile internet
204
976
5K
2
7
35
Today, we
@IBMPowerSystems
@OpenPOWERorg
Foundation announced that we are open-sourcing the
#IBM
POWER ISA. Its the high-performance counterpart of
@Risc_V
from
@SiFive
@Calista_Redmond
1
14
34
Yesterday, we announced the IBM-H2O partnership to bring automated machine learning to the enterprise, and today, we announced the largest & fastest AI supercomputer "Summit" at Oak Ridge National Labs
@ORNL
@IBMPowerSystems
@H2oai
#MachineLearning
#AI
1
14
33
Just realized that next time POTUS has a technology council meeting with the CEOs of IBM, Microsoft, Google, they might want to order Indian food ... ;)
Current CEOs of notable tech companies:
Alphabet - Sundar Pichai
Microsoft - Satya Nadella
Adobe - Shantanu Narayen
MasterCard - Ajaypal Singh Banga
Vimeo - Anjali Sud
And now IBM - Arvind Krishna
Common link?
All their parents are still disappointed that they’re not doctors
14
107
544
4
3
33
Corporations using ChatGPT are giving their internal secrets to
@OpenAI
, as Samsung engineers did when they used ChatGPT to write code.
You will need to use Generative AI that comes with guarantees that the provider does not read your data.
3
2
32
.
@IBM
is donating a "mini-Summit" Supercomputer to
@MIT
Computing to advance
#AI
research, based on
@IBMPowerSystems
AC922 Power systems with
@NVIDIADC
#GPUs
John Kelly: the IBM Summit supercomputer is the most powerful in the world and is breaking
#AI
records. We're donating 5 racks to
#MITComputing
to power research across the institution.
#MITIBM
0
26
57
1
11
29
There is no business model for AI generated search results - for Perplexity and for Google.
And the internet starts dying as publishers / content developers stop developing content because they get no traffic.
PERPLEXITY CEO: “.. In a world where everyone gets answers and doesn't have to click on links, the biggest loser is Google.”
@axios
$GOOGL
5
6
35
5
6
28
and IBM build a Strategic Partnership to bring AI innovation to the market together
@h2oai
sees large speed ups with
@IBMPowerSystems
and
@NVIDIA
GPUs
1
21
28
#DeepLearning
training times are going up.
@IBMPowerSystems
with NVLink between
@NVIDIADC
GPUs and Power9 CPUs, enables large models to train ~4x faster than Intel-based
#GPU
systems, and with Distributed deep learning, you can use 100s of GPUs
Heh - funny how much things have changed since May 2017!
3
42
181
2
13
24
Looking forward to this discussion on Sunday.
Tech Life Skills—my
#webinar
+live Q&A series to help
#students
,
#professionals
,
#developers
,everyone, learn from
#tech
#leaders
.
This week's special guest:
@SumitGup
, the
#DeepLearning
Guru! We'll talk
#HPC
,
#QuantumComputing
&
#COVID
ー19. May 10 @ 12 PM EST:
1
12
41
1
6
26
Its ironic that China's zero Covid policy will push manufacturing out to countries like India and Vietnam. Add to this the increasing US-China isolation / conflict, we will see an acceleration of the manufacturing move.
There are some violet protests going on in Foxconn Zhengzhou, China as workers attempt to flee the factories they are locked in.
This is the largest Apple product factory complex in the world with >200k people working there at some parts of the year
$AAPL
4
29
81
1
2
10
Even Keenu Reeves can see through the bullshit of
#NFT
Keanu Reeves' reaction to this question about NFTs is priceless.
137
2K
13K
2
6
24
There was a decision made yesterday that I believe will be a historical decision. Unfortunately I can't tell anyone for a few years. Nevertheless recording the date on Twitter
3
3
23
The hype of Generative AI is finally starting to dampen as everyone realizes that after 1.5 years, there is no commercially viable enterprise use case. The GenAI chatbots have had little impact due to inaccuracies and costs of serving.
AWS, Google and other generative AI providers are quietly tempering expectations. Learn more.
14
43
211
0
6
23
Early prototype of the Terminator. Judgement day is coming. Right after World War Z finishes.
2
9
22
Took us just 50 minutes to train ImageNet-1K using ResNet-50 - beat Facebook on performance & Microsoft on accuracy…
0
13
20
My guess-estimates on the logic behind why
@Apple
built the
@ARM
based
#M1Chip
. Anyone got
#MacBook
sales data that can improve my analysis?
@anshelsag
@PatrickMoorhead
@rwang0
@jonmasters
10
5
22
I am hiring Product Managers for machine learning infrastructure at
@Google
. Looking for folks with super strong product management expertise and experience. ML / analytics / similar experience very, very nice to have. Send me a message if you are interested.
#AI
#MachineLearning
1
4
20
Terrific report from
@Usefyi
- 90% of remote workers plan on remote work for rest of their careers
- 96% recommend remote work to friends
- 91% of remote workers say remote work is good fit for them
#COVID19
accelerating trend
#futureofwork
#worktrends
3
9
20
This was the quotable quote from Dr. Jackson's interview at
@IBM
last week.
“Diversity is a fact, inclusion is a choice” - Dr. Shirley Jackson
2
4
21
1
4
19
I believe this is a good thing. Celebrities can't just endorse a Ponzi scheme like Bitcoin / Crypto and not expect consequences.
JUST IN: Tom Brady, Steph Curry, Golden State Warriors, and other high-profile celebrities sued for endorsing FTX.
3K
4K
31K
1
4
20
Barclay's analyst also thinks it's too hard to get ROI enough to justify the current GenAI infra build. Same analysis has Sequoia.
3
3
18
This is such a critical problem in so many large tech companies. its so attractive to bring in more "senior" experienced people, who don't understand the company culture and at the same time, you piss off internal star performers
When we hire from the outside, we pay more to get less.
Despite costing more, external hires perform worse and quit more. Promoting from within fuels individual growth and collective success.
If there are never strong internal candidates, we've failed at leadership development.
69
741
4K
2
3
19
Automated
#SignLanguage
recognition using
@IBMPowerSystems
#AutoAI
#DeepLearning
based vision software called PowerAI Vision
#AI
#MachineLearning
Sneak peak at our latest project in the
@IBMPowerSystems
AI team. We’ve trained a DL model to recognise American Sign Language with PowerAI Vision. Here’s the model at work, transcribing my best attempt at the alphabet!
#ibm
#deeplearning
#poweraivision
#DeafAwarenessWeek
1
16
25
0
12
17
This license plate says it all. Grades matter to a point. And success is not directly correlated to grades
5
0
19
.
@Forbes
says 3.5x speedup for
#MachineLearning
training time is just as important as claiming the performance leadership for overall benchmark using
@IBMPowerSystems
with
@NVIDIADC
GPUs
#AI
#ArtificialIntelligence
1
14
18
As new CEO of
@Starbucks
, the first thing Laxman Narasimhan will do rename Chai tea to Chai (which means tea).
3
5
18
of course, the code is broken, because there is no way to reset the variable StillAlive. It needs a If Positive = GetTested(), then .... anyway
5
0
17
Big news:
@AMD
in advanced talks to buy
@Xilinx
. Massive consolidation happening in
#semiconductor
space, as companies use their stock valuations to buy others. Signs of a rapidly maturing (and slowing down) market
2
6
17
Looks like first
#AI
Startup with silicon for
#deeplearning
inference with remarkable performance results. 15,000 ResNet-50 images/second with 1.3ms latency, batch size 10 at 100W.
#MachineLearning
0
6
17
We are finally revealing some of the details behind our currently deployed TPU infrastructure, the TPU v4.
Detailed blog with link to ISCA paper here:
@CNET
write up here
1
4
17
This is indeed big news. Yet another specialized accelerator for HPC (NEC SX vector processors) bite the dust.
The commercial engine of ML and graphics accelerators (TPUs, GPUs) are so big, that they can innovate faster and with the latest memory / interconnect IPs.
Buried in GTC/GTP excitement, NEC announces ceasing R&D of their next gen accelerator aiming to be 10x improvement in power efficiency, citing that commercial accelerators will match their objectives. This could be the final nail in the coffin for their SX vector processors
4
31
68
1
1
16
Results from researcher Sinead Griffin (below) at Lawrence Berkeley national labs (LBNL) confirms LK-99 as a superconductor at room temperature and then mic drops and takes a long deserved nap.
1
4
17
#GPU
acceleration of Logistic Regression; most widely used
#MachineLearning
algo per
@Kaggle
data scientist survey. 46x faster using
@NVIDIADC
GPUs in
@IBMPowerSystems
than
@GoogleCloud
work. NVLink between Power9-GPU enables training with sparse data
0
11
17
You can carry an AR15 around but not a journalist mic / gear
NEW - Journalists will face arrest, charges if caught reporting while on Uvalde Consolidated Independent School District property
940
4K
7K
0
3
17
Top500 list of supercomputers is out.
Top10 systems are more flops than bottom 490
Top 5 are 85% of the flops of the Top 10
This means, we have 5 really massive supercomputers, vs 100s of small ones. Large hyperscalars like Google don't enter Top500
#HPC
#supercomputing
#SC22
0
7
17
Delighted to see this new series of ARM-based server chips from Google finally out in the market!
BOOM! The worst kept semi secret is out.
@Google
announces Axion, an
@Arm
-based general purpose processor. Already deployed internally on services like
@YouTube
, in preview for enterprise customers soon. More details later from
@MattKimball_MIS
.
1
13
90
0
1
16
Great talk from PGS about they built a 53K node GKE system in
@GoogleCloud
with 800K+ vCPUs making it the 24th largest super computer in the world. They moved from a 260,000-core Cray supercomputer.
#HPC
infrastructure
@rseroter
1
5
17
Example of how *not* to do a marketing chart. Instead of saying 106% faster, please say 2x Faster
“Polite Schadenfreude” - It's hard to overstate how badly
@AMD
is shaming
@Intel
in the data center lately:
@arstechnica
#HPC
#AI
#Milan
@AMDServer
1
2
26
5
1
15
Thank god we had twins and not triplets, otherwise this would have been me
1
2
15
0
4
16
1
6
16
PowerAI
#DeepLearning
Meetups organized by
@IBMPowerSystems
European team have hit 7000 members. Join these meetups to learn more about
#AI
#MachineLearning
in the Enterprise
0
10
16
The OEMs and hyperscalars have gone from supporting just Intel CPUs to AMD and now ARM. There are two dynamics at play:
- AMD's strong roadmap
- Poor gen-to-gen perf improvements from CPUs (death of Moore's Law), prompting looking at alternate architectures (ARM, RISC-V)
0
4
15
Beautiful day and terrific view from the
@ibm
Munich office
@IBMWatsonIoT
center. Almost 200 people at the
#PowerAI
Meetup by
@IBMPowerSystems
0
5
14
Amazing white hat
#hacker
work by
@IBMSecurity
team on a new type of enterprise attack called "Warshipping". Beware of the $100 hacking device in our corporate gifts like Teddy Bears that can sniff your Wifi password
1
10
15
Learn about fastest
#Supercomputer
for US Science being built
@ORNL
Oak Ridge.
@Uber
ran their Horovod
#DeepLearning
scaling software with PowerAI on 2000 V100 GPUs using
@IBMPowerSystems
Power9 systems : Time lapse video at
#HPC
0
7
15
I have seen this video several times over the years & it is still a lesson in instruction. As technologists, we first gravitate to talking about speeds & feeds, rather than client value, our core value prop, our core product vision / what problem we are trying to solve.
A masterclass in marketing from one of the greatest ever, Steve Jobs. Marketing is about values.
90
4K
13K
0
1
15
0
4
15
So much fun tracking the
#StarWars
Millenium Falcon using
#Tensorflow
#deeplearning
w/
@IBMPowerSystems
PowerAI
0
14
15
It’s been a fun two weeks hanging out with
@srisatish
founder / CEO of
@h2oai
at
@IBM
#Think2018
and
@NVIDIA
#GTC2018
as we
@IBMPowerSystems
partner on
#MachineLearning
and
#DeepLearning
with PowerAI
0
10
15
Its chip week: - Google announced their custom ARM SoC, Axion & availability of TPU v5p:
- Intel announced Gaudi3 AI accelerator for training:
- Meta announced v2 of their training / inference AI chip, MTIA:
4
0
14
Benchmarks for AMD gpus showing up. Looks like great scaling and their investment in a much larger HBM memory leads to a dramatic reduction in the number of nodes required.
#HPC
what's really cool though is just how much more we can do with
@OLCFGOV
Frontier.
The same code runs on the AMD GPUs there (now using HIP instead of CUDA) and just works. The larger GPU memory means we can run on fewer nodes too! Here's strong scaling for this XRB run.
2
15
32
0
3
14
#AutoAI
for building
#DeepLearning
visual models to recognize
#SignLanguage
using
@IBMPowerSystems
PowerAI Vision with the power of
@NvidiaAI
#CUDA
#GPUs
#AI
@ylecun
@sallyeaves
0
4
14
Building self-driving cars is super hard. I advised a startup in the
#ADAS
space; realized its not about
#AI
/
#DeepLearning
. about building a complex real-time control system doing complex sensor fusion with diverse sensors. Argo AI is first of many IMHO
1
2
13
Silicon Valley has healed. At Local Starbucks at 4:30 pm on Friday. Every table is two people clearly having a business / startup ideation meeting. Many with laptops open
2
0
14
IBM Invents Jet Engine for
#DeepLearning
;
#PowerAI
scales TensorFlow & Caffe to 256 GPUs on IBM Power Systems…
0
18
14
IBM Invents Jet Engine for
#DeepLearning
;
#PowerAI
scales TensorFlow & Caffe to 256 GPUs on IBM Power Systems
0
9
13
Boggles my mind all the areas that
@IBMResearch
works on. In this work, IBM researchers working with researchers from Singapore develop new macromolecule that could counter antibiotic resistance
@TechCrunch
0
7
14