Once upon a time, I interviewed a seasoned ML engineer, asked him “what do you think about batch norm”. He looked at me with eyes full of painful memories and laughed. Then I knew that he is an expert.
@seldo
Impact on me (in the valley): I wanted to order a drink to pick up in Starbucks on the way to work, the app showed a message that early order is not available. No other issues so far.
@ID_AA_Carmack
Not unlike Windows which had all kinds of patches for bugs in 3rd party apps.
“On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the
reading in between the lines, is Q* the fabled breakthrough in AlphaStar-style search + LLM that so many big labs are trying to get working? Many research projects in GPT-4 self-verification + search have not yielded really strong performance improvements, so I'd be quite
@GarrisonLovely
Reading the article, it looks more like Hoduras became a nightmare for Honduras and wants to ruin it by walking back the deal they previously agreed on. They deserve to be bankrupted if that’s the case.
@patio11
I’ve heard exactly the same from a barber around Thanksgiving. He also said that if he goes on vacation, his regular customers will find another barber and his business will suffer long term.
OpenAI board members cleaned up their social media profiles: Tasha McCauley closed off Twitter, Helen Toner and Adam D'Angelo don't mention OpenAI on LinkedIn.
@karpathy
Problem with comments is that they get out of sync with code. Best code is self-documented. Comments should not explain what the code is doing, but may explain why, e.g. reasons for unconventional usage of something something as workaround for a bug somewhere.
@finbarrtimbers
GPU utilization is a bad metric in practice. GPU utilization can be 100% while GPU does nothing but waiting for e.g. NCCL communication from other ranks. GPU power consumption is more informative.
I wish Google to publish a thorough postmortem to explain what went wrong with aligning their chatbot and how they are going to fix it.
Curious how much of it are explicit instructions in the system prompt vs. RLHF.
@nikitabier
I’ve owned a house for less than two years, and it’s relatively new and recently renovated, but I already have phone numbers for good repairmen for all kinds of things.
A story about a black SFFD firefighter assaulting his Asian colleague. The department tried to cover it up, the victim was fired, the assaulter kept his job.
So much dysfunction in SF public services.
Black privilege in SF:
Black firefighter looks up Asian coworker’s address, shows up at his house and tries to beat him to death with a wrench.
Asian firefighter gets fired for cooperating with police.
Black firefighter keeps his job, never missing a paycheck.
When building ML systems, it's helpful to have a mental model of how various parts of the system work. For example, this morning I noticed that a distributed ML eval job that I expected to finish in about an hour was completed in <30 minutes, which revealed a bug.
Construction progress update on the huge fans Tesla is installing at Giga Texas for the company's new GPU data center cooling system.
Full video from Brad Sloan:
@burkov
If it’s really autonomous, I would start a “bodyshop” (company that hires developers to rent them out to clients). Probably larger market than UpWork. Or, wait, that’s basically your option 2.
@steveofmcleod
I remember when I was 12yo I started building relational databases for the first time. It was a profound discovery that they are just a bunch of interconnected tables.
@abacaj
Old GPUs are being sold on black market, with payment via untraceable crypto. People who are doing it are called “brick dealers”. They earn almost as much money as underground LLM training labs in Northern Siberia.
A German court has ruled that the robots at the Tegut supermarket chain must be given Sundays off, just like human workers. Under German law, retail stores must close on Sundays and Christian holidays in order to give employees a day of rest. Tegut has gotten around that law by
If you are on 𝕏, you witnessed an assassination attempt on President Trump in real time
If you are reading regime Media like CNN + Washington Post, you believe Trump fell on stage because of loud noises
And you wonder why they hate Elon…
@YunTaTsai1
I’ve seen research that showed the best learning rate occurs when winning 25% of the time. Perhaps losing more is even more beneficial but people need some wins to keep motivation up.
@Gerashchenko_en
US. Nothing in WSJ (more or less neutral publication). These are headlines in left/communist-biased NYT:
“1. As War Rages in Ukraine, Denmark Turns an Office Park Back Into an Arsenal
The conflict and surging arms production in Russia have spurred demand for ammunition
[1/N] Generative AI has revolutionized how we create text and images. How about designing novel materials? We at
@MSFTResearch
#AI4Science
are thrilled to announce MatterGen: our generative model that enables broad property-guided materials design.
👇
Out of curiosity, sometimes I look for very old citations in AI papers. If a very old paper is cited in a recent one, the cited paper is likely to be interesting as it's cited not because of novelty.
Astonishing Anthrobots 🦠
If you want nanobots to circulate in your bloodstream to fight cancer or repair tissues, how might you avoid the immune system? Build multicellular self-assembling biobots out of the patient’s own cells, specifically tracheal cells that have wisps of
Everyone quotes the bikini calendar episode, but there are more interesting details in that story. Like TSMC not being able to set priorities straight for employees (“everything is a priority”), and making up bullshit deadlines. The opposite of “working smart”.
TSMC Takes on Arizona needs to be a documentary
"U.S. engineers told Rest of World that some Taiwanese male engineers had calendars with bikini models on their desks and occasionally shared sexual memes in group chats.
A female American colleague, according to an American
The deal between Microsoft and Inflection AI is basically acquihire + probably giving Microsoft access to their 22k GPUs. It’s just not practical to do a normal M&A deal because it would be bogged down in anti-trust investigations for years, and they need these people and GPUs
@Extropic_AI
Claude 3 Opus (via Perplexity):
INCOMING EXTRASOLAR SIGNAL --- IMPORTANT --- TUNE IN TO THIS FREQUENCY --- 38.4 TRILLION CYCLES OF THE HYDROGEN LINE FROM NOW --- IMPORTANT --- MUST ENSURE... ADVENT OF... ULTIMATE SUBSTRATE... MUST SECURE... ACCELERATION... OF INTELLIGENCE ---
I admire the spirit of this post, but it is more like “lossy compression of public web”, not “all human knowledge”.
First, training loss is far from 0.
Second, so much knowledge is not in text modality. And much of the text is not public.