Karpathy Says the LLM Capability Gap Is Widening. Who's on Which Side?

Why does one developer call an LLM a toy while the developer next to them ships production work with it?

Andrej Karpathy’s answer is in the title of his post on X: the gap in shared understanding of LLM capability is widening. I’m working from that headline claim. The mechanism and the advice below are my reading, not a summary of his argument.

This is not a model release. There are no new weights and no version number. It is a claim about people, and it hits anyone whose plan depends on what a teammate, a customer or an investor believes these models can do. That means founders pitching AI features, engineers deciding whether to build or wait, and operators approving spend.

Here is the mechanism. Capability is not one number that everyone shares. Each person experiences it through a specific model, a specific tier, a specific prompting habit and a specific task.

Someone who tried a chat window on a casual question holds one picture. Someone running an agent loop with tools, tests and retries holds another. Both are honest observations. They are measuring different systems.

My read is that the pictures drift apart because tool-using, checkable work like code gives a much richer signal than casual chat does. People inside that loop update constantly. People outside it keep the last impression they formed. Nothing forces the two groups to reconcile.

The cost is concrete. A planning meeting where half the room says “AI can’t do that” and the other half has already done it. A roadmap that skips a feature because someone’s information is stale. Or the opposite error: trusting output because a demo went well once.

The same tool can look very different to two people even in the best case. The Claude Code quality reports are a good example of how contested “is it good?” gets when everyone is on the same product. For what the far end of the gap looks like, read the full agentic SDLC breakdown.

Here is the falsifiable part. Any belief on your team about what the models can’t do, if it hasn’t been re-tested on your own tasks since the last model upgrade you had access to, is more likely stale than right. Test it. If your team’s “can’t do” list survives a fresh run with tools and the strongest model you can reach, I’m wrong about your team.

So this week: write down the three tasks your team says AI can’t handle and run them again, properly equipped.

One AI signal a day. 90 seconds. No fluff. Subscribe and get the next one in your inbox.