value and opportunity

Who is Andrej Karpathy ? Born in Slovakia and despite only 38 years young/old, he is already an AI “Veteran” having initially studied under AI Godfather Geoff Hinton in Canada, did internships at Google Brain and DeepMind, Co-founded OpenAI, was leading AI at Tesla, went back to OpenAi and now is focusing on teaching AI to everyone who would listen.

Since I discovered his Youtube educational Videos, I am following him because when he speaks about something, there is always a lot to learn. 

Yesterday, he did a 2 hour interview with maybe the best current “AI Podcaster” Dwarkesh Patel. Those two hours are quite dense and I had to use Gemini in parallel to understand some of the stuff, but at least on my Twitter timeline, it raised quite a “storm in the teacup” among AI “experts.

Here are some of his main talking points (as far as I understood them):

• “Real” AGI (Artificial General Intelligence) takes at least 10 years
• Current “AI Agents” are clumsy and will remain to be so for quite some time
• LLMs are not really good in writing new code (i.e. improving themselves for instance)
• Brute forcing the current models will not achieve great jumps, more structural advances are required
• The current architecture of LLM models with the giant data amounts used for pretraining actually prevents them from developing their “intelligence”, especially as the data is very bad
• Even he himself admits not to fully understand why and how these models actually work
• He also casually mentions that Self Driving is nowhere near perfect with many human operators still in the loop

As A diligent person, Karpathy watched his interview and clarified the main thesis in a long Twitter post.

In a nutshell, he claims that we are not anywhere close with AI to “General Intelligence” in contrast to what Instance Sam Altman, Elon Musk or Jensen Huang are claiming.

So why could this be a (big) problem ?

Well, that one is obvious: The gargantuan amount of money that is spent right now in scaling up “AI Data Centres” only makes sense, if AI keeps making giant leaps and the economic benefit (i.e. replacing lots humans with AI in the workplace) materializes in relatively short time horizons.

If Ai is only good enough to improve the efficiency of programmers and hooks people for even longer to Social Media (like ChatGPT now offering “Adult Content”), then that is clearly good for companies like Meta, Google etc, but it maybe doesn’t justify the amount of Capex spent at the moment and especially not on “quickly perishable” GPUs from Nvidia.

If that “explosion” of capabilities only happens in 10 years like Karpathy indicates, you might have burnt through trillions and trillions of Nvidia GPUs for pretty small improvements in productivity which would result in a similarly pretty small (if all) return on investment.

Interestingly, Karpathy himself mentions that overall, he doesn’t think that there is a massive overspending on AI infrastructure but he also mentions the Railrod and Telco/Fiber “Bubbles” of the past.

Some have more time than others

In this context, one thought from the recent Acquired Podcast about Googe’s AI capabilities came back to my mind:

Google (and Apple, Amazon and Microsoft) are clearly less in a hurry than OpenAI, Anthropic, XAI etc. Why ? Because if an AI breakthrough takes longer than 1 or 2 years, they still have a lot of cashflow from other activities, whereas for the “pure plays” timing is extremely important as they burn cash like crazy and if AGI doesn’t come soon, they might be in trouble.

Funnily enough, Elon challenged Karpathy on Twitter to a coding challenge against Grok 5, but Karpathy is way too smart for that.

It is also telling, that in parallel, a senior OpenAI researcher claimed on Twitter that OpenAi had found entirely new solutions for super hard mathematical problems, which was then very quickly debunked by a Google employee who found out that ChatGPT had actually found the solution on the internet.

So whenever we are listening to Sam Altman and Co, one should make sure to understand that whatever they claim, they are in a hurry. 

Karpathy’s small hack for investors:

It’s maybe not revolutionary, but Karpathy mentions that he would look into easily digitally automatable professions in order to check on the progress of AGI.

He explicitly mentions Call Center Operators. I would add for instance the typical IT outsourcing businesses. I will definitely add a few of those listed businesses to my general watchlist.

Conclusion:

To be honest, I don’t think that the “Karpathy moment” in the short term will make a big dent especially in the Stock market and the VC arena. The momentum is just too strong and there is a lot of money out there chasing the AGI dream.

But I guess it makes sense to look for more signs that momentum is slowing in one area or the other. 

P.S.: And I can only recommend to follow Karpathy and Dwarkesh in order to understand what is going on in AI. They are maybe better sources than the usual cheerleaders.