“The End of Coding”: Andrej Karpathy on Agents, AutoResearch, and AI’s “Loop Era” | Full Chinese Translation

Original video: The End of Coding: Andrej Karpathy on Agents, AutoResearch, and the Loopy Era of AI
Video link: https://www.youtube.com/watch?v=kwSVtQ7dziU

Note: The following is the complete Chinese translated transcript, preserving the original meaning and structure as much as possible, with only a small number of meaningless filler words (such as um, uh) removed.

“Writing code” isn’t even an accurate verb anymore, right? I’m more like—every day for 16 hours—expressing intent to my agents, manifesting things into existence.
How can I have more than just a single session of Claude Code or Codex or one of those agent frameworks? How can I have more? How can I do it properly? The agent part is now taken for granted. Claude-like entities are now taken for granted. Now you can have multiple entities, now you can give them instructions, now you can optimize those instructions. But I mean, that’s why it gets you hooked—because it’s like infinite; everything is a skill issue.
Hello everyone, welcome back to No Priors. Today, I’m here with Andrej Karpathy (Andre Karpathy), and we’re going to have a wide-ranging conversation about code agents, engineering and the future of AI research, how more people can contribute to research, what’s happening in robotics, his predictions about how agents will touch the real world, and education in the next era. Welcome, Andrej. Andrej, thanks for doing this. Yeah, thanks for having me.
,so the past few months have been a very exciting few months in AI.
Yeah, you could say that.
I remember walking into the office once, and you looked like you were “locked in.” I asked what you were doing, and you said, I just have to code 16 hours a day, otherwise “coding” isn’t the right verb anymore, right? But I have to—
for 16 hours a day, express my intent to my agents. It’s become obvious because there’s been a leap in capability.
What happened? Tell me about your experience.

Yeah, I feel like I’m perpetually in this AI-high state—like I always have been. Because as a person, you can achieve a lot, right? Your bottleneck is your typing speed and so on. But now with these agents, what I’d say is: in December, things changed completely. I went from 80/20 to 20/80—writing code myself instead of just delegating to agents. I don’t even think it’s 20/80 anymore. I think it’s way beyond that. I don’t think I’ve basically typed a single line of code since December, which is a huge change. I’m talking to it—like I’d talk to my parents, etc. And I don’t think the average person has realized this happened, or how dramatic it is. Like if you just find some random software engineer at their desk or something, what they’re doing is basically the default workflow—and since December, building software is completely different. So I’m in this “high” state trying to figure out what’s possible, trying to push it to the limit. How do I have not just one, you know, Claude Code or Codex or some agent framework session—how do I have more? How do I do that properly? How do I use these background agents? What even are these background agents?
So there are lots of new things. I want to be at the cutting edge, but I’m anxious because I’m not at the cutting edge. On Twitter I see lots of people doing all kinds of things that sound like really good ideas, and I need to be at the cutting edge or I get very nervous. So I think I’m just in this “possible” high, because it fundamentally hasn’t been explored yet.
Well, if you’re nervous, the rest of us are nervous. We have a team that works with us, and they firmly believe in a setup where everyone is like—you know—no engineer hand-writes code; they all wear microphones; they just love whispering to their agents nonstop. It’s the weirdest work environment ever. I thought they were crazy, and now I’ve fully come around—like, “Oh, this is how it is.” Like you’re already out in front.
How do you think about your ability right now to explore or do projects—what limits it?

Yeah. What limits it? I just think everything is like—so many things—where even if they don’t work, it largely feels like a skill issue. It’s not that the capability isn’t there; it’s that you haven’t yet found a way to combine what’s available. Like maybe I didn’t write good enough instructions in an agent/MD file or something, or I didn’t plug in good enough memory tools, etc. So when the system doesn’t behave as expected, it’s more like a reminder: is my usage skill not good enough, or did I not parallelize it properly? You basically want to be Peter Steinberg. Peter is famous—there’s a funny photo of him standing in front of monitors with lots of Codex stuff. So he has many Codex agents across monitors, and if you prompt correctly and put in a lot of effort, they each need about 20 minutes. They each need about 20 minutes. He has multiple—he’s checked out like 10 repos—so he just cycles through and gives them work. It’s like you can take bigger macro actions. It’s not just one line of code; it’s a new function. It’s like: this is a new feature—delegate to agent one. This is a new feature that won’t interfere with others—delegate to agent two. And then, depending on how much you care about that code, review their work as thoroughly as you can. Like, what macro operations can I use to manipulate my software repository?
Like another agent is doing some research; another agent is writing code; another agent is proposing new implementation plans. So everything is like macro operations happening on your repo, and you just want to get very good at it and build it into muscle memory. Because it’s very—yeah—it’s very valuable, first because it actually works, but it’s also kind of a new thing to learn. That’s why it’s intoxicating. Yeah, I really feel like my instinct, whenever I’m waiting for an agent to finish something, is: obviously, I could do more work, right? Like if I can get more tokens, I should frantically add more tasks. So it’s very stressful because if you don’t feel like your ability to spend tokens is heavily constrained, then you know you’re the biggest bottleneck in the system.

Yeah. At minimum you’re not maximizing your subscription; ideally with multiple agents, like if you run out of quota on Codex you should switch to Claude or something—I don’t know, that’s what I’ve been trying to do. When I have subscription left, I get nervous, because it means I’m not maximizing my token throughput. I actually had this as a PhD student: you’d get nervous when your GPUs weren’t running, because you had GPU capacity and you weren’t maximizing the available FLOPs. But now it’s not FLOPs; it’s tokens. So what’s your token throughput? What’s the token throughput of your commands?
I’d actually say—this is interesting—we’ve known for at least 10 years that in many engineering tasks, people just weren’t feeling compute-constrained.
Yeah, the whole industry now feels resource-constrained.
Now you have such a huge leap in capability that you think: oh, actually, it’s not my ability to access compute—like I’m the binding constraint.
Yeah, it’s a skill issue.
It’s very empowering, because yeah—because you can get better. That’s why I think it’s so addictive: as you get better, you unlock more.

Where do you think it goes? Like if you just imagine—okay, Andre is iterating and other people are spending 16 hours a day with coding agents—what does a year of mastery look like?
Yeah. By the end of the year, what does mastery look like—or two years, three years, five years, ten years, etc.

Well, I think everyone is basically interested in lifting up the stack. So I’d say: this isn’t one meeting with your agent. It’s what multiple agents look like collaborating, teaming, etc. Everyone is trying to figure out what that looks like. And then I’d say background agents are also an interesting direction, because when I say background agents I mean a layer that takes persistence to a whole new level. Like it’s a continuously running loop; it’s not something you actively participate in. It kind of has its own little sandbox, and it kind of acts on your behalf even when you’re not looking at anything. And then there could be more complex memory systems and so on that haven’t been implemented in agents yet. So I’d say these resident background agents should have more complex memory than default agents—beyond just memory compression when the context runs out. Right.

Do you think that will resonate with more users than broader tool access would—for these resident background agents?

Yeah. There are a lot of really good ideas there. Yeah. Nice work, Peter.
I mean, Peter did an amazing job. I saw him recently and talked to him about it; he’s very humble about it, but I think he innovated in like five different ways at once and combined them. For example, things like SOUL and MD files—he really shaped a compelling and interesting personality, and I feel like a lot of current agents don’t quite get that right. Actually I think Claude has a great personality. It feels like a teammate—you get excited, etc. Whereas, for example, Codex is drier. It’s interesting, because in ChatGPT it feels more optimistic and more willing to go along with you. But I’d say the Codex coding agent is very dry. It doesn’t seem to care about what you’re building. It’s kind of like, “Oh, I implemented it.” Like, okay, but do you understand what we’re building?
That’s true.
You know—no. Another thing is: for example, Claude, I think they handle the “high” problem well. When Claude praises me, I do kind of feel like I earned it. Because sometimes I’ll give it a half-baked idea—an idea I don’t think is fully mature—and it doesn’t react strongly. It’s like, “Oh yeah, we can implement that.” But when I myself think it’s a really good idea, it seems to reward me more. So I feel like I want to earn its praise, which is really weird.
So I do think personality matters. A lot of other tools might not value it as much; Peter clearly cares about it too. Then there’s the memory system; and he’s interested in that; and then there’s the single WhatsApp portal to all automation.

Yeah. Outside of software engineering, have you personally done anything you found interesting or fun?

Yeah. So in January I had a background agent—I went through a background-agent high period. I built a background agent that basically takes care of my house; I call him Dobby the house-elf. Basically, I used an agent to find all the smart-home subsystems in my house on my LAN. I was kind of surprised it worked out of the box. Like I said: I think I have Sonos at home. Like, can you try to find it? And it really did an IP scan of all the basic computers on the LAN; it found the Sonos system. It turns out there was no password protection or anything. I just logged in—like, “Oh yeah, you have these Sonos systems installed.” It then tried to reverse-engineer how it works. It did some web searches and found the API endpoints; it asked, “Do you want to try them?” I was like, “You just did that?” I said, “Yeah—can you try playing something in the study?” And it did; music started playing. And I was like, “I can’t believe I just—this is insane. That was like three prompts.” Yeah.
I can’t believe I just typed: “Can you find my Sonos?” and suddenly it started playing music. It did the same for the lights. It basically “hacked” it—figured the whole thing out. Created APIs, built a dashboard so I can see a command center—like all the lights in the house—then turn them on and off. And so I can say things like, “Dobby, I’m sleepy,” and that means all the lights turn off, etc. So it controls all my lights, HVAC, curtains, pool and spa, and my security system. I have a camera pointing outside; whenever someone comes in, I use a vision model to watch the video. First there’s change detection, right?
Then after it detects a change, it hands the frames to a vision model for analysis, and then it messages me on WhatsApp. It attaches an image from outside the door and tells me, “Hey, a FedEx truck just came; you should take a look—maybe you have mail.” Dobby just texts me directly. It’s really absurd—and really cool. Now Dobby is basically in charge of these things around the house, and I talk to it through WhatsApp. This experience of having macro-level operations that maintain your whole home on your behalf is really interesting. I haven’t even pushed it to more extreme levels; I know lots of people have gone even crazier. But even just home automation, at this layer, has already been very useful for me. I used to need six totally different apps; now I don’t. Dobby can control everything in natural language. It’s amazing. So I feel like I haven’t pushed this paradigm to its limit, but it’s already helpful enough—and inspiring enough.

Do you think that shows what people really want from software UX? Because something that often gets overlooked is: humans have to spend effort to learn new software and adapt to a new interface.
Yeah, I agree to some extent. It’s like designing systems in reverse around what people imagine AI to be. Because in most people’s minds, AI isn’t the LLM in its raw sense—an LLM is basically just a token generator, spewing more tokens. What people actually think AI is more like: a being with personality and identity. You can talk to it; it remembers; it’s an entity behind WhatsApp. That’s much more natural to understand.
So in a sense, this is matching people’s existing expectations of how “AI should behave.” It’s just that to implement it, you have to stuff a lot of technical details under the hood. And for most people, the LLM primitive is still too rough—strictly speaking, it’s not yet that AI they have in mind.

Yeah, I think it also says something about how we understand AI. Describing it as Dobby, or giving it a personality, is obviously more resonant. At the same time, I also feel like: by unifying six different home-automation apps, you’re pointing at another problem—
do people really need today’s world of a bunch of fragmented software?
Yeah.
Right. Because in a sense, you kept the hardware, but you threw away the software—at least the software UX layer. Do you think that’s what people actually want?

Yeah. I think there’s a feeling that these apps in the app store that exist to use these smart-home devices—in a sense, the apps shouldn’t exist. It should just be APIs, and agents should directly use them, right? I can do all kinds of home automation work that no single app can do properly, and the LLM can actually drive tools, call the right tools, and do fairly complex things.
So in a sense it does point to this: maybe there’s massive overproduction of custom apps that shouldn’t exist, because agents kind of smash them. Everything should be exposed as API endpoints, and the agent is the intelligent glue—tool-calling all the pieces. Another example is my treadmill. There’s an app for it; I want to track how often I do cardio. But I don’t want to log into a web UI and go through a flow, etc. All of this should just be: make an API available. That’s the path toward agent-style web, agent-first tools, and all that. So I think the industry has to reconfigure in many ways—like the customer is no longer a human. It’s an agent acting on behalf of a human. That kind of refactor could be major.

One way people push back is: do people want to code some of these tools? Are we expecting ordinary people to do what you’re describing?
But I think, in a sense, this is just today’s state of the art. There’s some vibe coding; I’m watching it and using the system. But I feel like what I just described should be free in one, two, or three years. No “backend coding.” It’s trivial. It’s table stakes. Any AI—even open-source models—should be able to do this.
You should be able to translate low-tech human intent into this very easily.
Very easily. Yeah. Today there’s vibe coding, but not many people do it. But you still need to make some design decisions, right? We’re talking about frameworks as an example.
Yeah.
Yeah. But I feel like this is just the beginning; the barrier will disappear. It’ll just be ephemeral software on your behalf—some persistent background-agent-like thing handling all details, and you’re not in the loop. The background agent has a machine; it solves the problem; it just shows you a UI, and you just say what you want. Hmm.

Why don’t you push the boundary of what you personally can do with Claude? Like you’re focused on bigger projects—AutoResearch, etc.—or you’re climbing mountains or whatever, right?
Yeah. I just feel distracted by everything. I spent about a week on classes; I almost have even more things to do. But I’d say—unfortunately—tools like these are themselves getting busier and more powerful.
Yeah, I haven’t really used it for things like email and calendar and all that. I haven’t given it access because I’m still a bit suspicious, and it’s still new and rough around the edges. So I don’t yet want it to have full access to my digital life. Partly for security and privacy—you have to be very careful in that domain. So I’d say some of this is blocked. Yeah, maybe that’s the dominant factor. But some of it is also just: I’m overwhelmed, because I feel like I spent a week and then other things happened.

You talked about being able to train—or at least optimize—models for tasks you want agents to run long-term. What was the motivation behind AutoResearch?
AutoResearch. Yeah. I think I tweeted before: to fully exploit the tools available now, you have to remove your own bottlenecks. You can’t sit there prompting the next step. You need to take yourself out. You have to set things up so they’re fully autonomous, and the more you know about how to maximize token throughput—without falling into loops—the better. That’s the goal. So I mentioned: the name of the game right now is increasing your leverage. I spend a small number of tokens occasionally, and a lot happens in my name. So AutoResearch—like I tweeted—I think people like it and so on, but they don’t like what it implies in terms of how you work. For me, AutoResearch is an example of what it implies:
I don’t want to be the researcher in the loop—checking results and so on—because I’m holding the system back. So the question is: how do I refactor all the abstractions so I don’t have to be in the loop? You schedule it once, press start, and the name of the game is: how do you get more agents to run longer without you—doing things on your behalf. AutoResearch is just—yeah—that goal: that metric, that boundary of what it can and can’t do. Do, then leave. Yeah.

Were you surprised by its effectiveness?

Yeah, I didn’t think it would work. I have Project Data Chat—fundamentally, I think a lot of people are confused by my obsession with training GPT-2 models and so on. But for me, training GPT models is just a small tool—a little playground for training LLMs. Fundamentally I’m more interested in this idea of recursive self-improvement: how far can you push LLM self-improvement? Because I feel like all frontier labs are basically doing this, for obvious reasons. They’re all trying to recursively self-improve, roughly speaking. So for me it’s like a little mini-game.
And I think I’ve enjoyed old-school manual hyperparameter tuning. I’m used to being a researcher; I’ve done this for twenty years—like, you know, twenty years.
Yeah.
Building confidence—
Right, okay, I’ve trained this model thousands of times. I’ve done lots of experiments. I did very basic tweaks; I did all the stuff I’m used to doing. I’ve done it for twenty years. I got to a point where I thought it was tuned pretty well, and then I let AutoResearch tune overnight. It came back, and I saw adjustments I hadn’t made.
Yeah, I had forgotten things like weight decay on the value embeddings and my Adam betas weren’t fully tuned. These things interact; once you tune one thing, other things can change. I shouldn’t be the bottleneck. I shouldn’t be running these hyperparameter search optimizations. I shouldn’t be the one looking at results. In this case there’s an objective criterion. So you just need to schedule it so it can keep going forever.
So that’s a single version of AutoResearch—one loop trying to improve. I was surprised it found things; I know the repo was already tuned pretty well and it still found something. That’s just one loop. Frontier labs have tens of thousands of GPUs, so it’s easy to imagine doing lots of automation on smaller models. And fundamentally, frontier-level intelligence is about extrapolation and scaling laws—so you do lots of exploration on smaller models, then try to infer.

So you’re saying our research work will become much more efficient—we’ll have better directions for scaling if we can do these experiments better.
Yeah, I’d say: the most interesting projects frontier labs are probably working on are things like experimenting on smaller models, trying to make it as autonomous as possible, removing researchers from the loop. They have too many—what’s the opposite? Too confident. Yeah—they don’t know. They shouldn’t really touch any of this. So you have to rewrite the whole thing. Because now, I mean, sure, they can contribute ideas, but—okay—they actually shouldn’t be implementing those ideas.
There’s an idea queue. Maybe there’s an automated scientist proposing ideas based on all archived papers and GitHub repos; it aggregates ideas. Or researchers can contribute ideas. But it’s a single queue; workers pull projects and try them. Whatever works gets put on a feature branch. Maybe some people watch feature branches and sometimes merge to main.
So yeah: remove humans from all processes, automate as much as possible, get as high token throughput per second as possible. It requires rethinking all abstractions. Everything has to be reshuffled. Yeah, I think it’s very exciting.

If we go one more recursion: when will the model write a better program MD than you?
Yeah. So program MD is—
We’re not in the loop.
Yeah, exactly.
Yeah, so program MD is my clumsy attempt to describe how the AutoResearch system should work: do this, then do that, try these ideas; here are some ideas; look at the architecture; look at the optimizer, etc. I just came up with it in Markdown, right?
Yeah, exactly. You want some kind of AutoResearch loop, and you can imagine different versions of program MD producing different progress. So basically, every research organization is described by a program MD. Yeah.

A research organization is a set of Markdown files describing all the roles and how the whole thing communicates. You can imagine having a better research organization. Maybe they do fewer standups in the morning because they’re useless—it’s just code, right? So one organization can do fewer standups; one can do more; one can be very risk-taking; one can do fewer standups. So you can absolutely imagine multiple research organizations. And they all have code; once you have code, you can imagine tuning it. So it’s 100% a meta layer.

Did you see my contest idea? My contest idea is: have people write different program MDs, right? For the same hardware, where do you get the biggest improvement?
I see.
Then you collect all that data and feed it to the model and have it write a better program MD.
Yeah. Yeah.
Yeah. Exactly.
We’ll get something better. Like, we can’t not do it.
You can 100% look at the sources of improvement, like: I can change program MD so it does more of this kind of thing or stops doing stuff that doesn’t work.
Meta-optimization. Yeah.
You can 100% imagine doing that. So I think it’s a good idea. But, you know, you take it step by step—you have one process, then a second process, then the next. They’re layers of the onion. LLMs are now taken for granted. Agents are now taken for granted. Claude-like entities are now taken for granted. Now you can have multiple entities, now you can give them instructions, now you can optimize those instructions. It’s kind of too much. But that’s why it’s intoxicating—because it’s infinite; everything is still an open problem. That’s why it feels so crazy.

Okay. If we try to diagnose the current moment—what skills matter right now, what do you like—does this mean we should implement loops in different domains and have them work by creating metrics, or letting agents keep pursuing a metric without you?
Yeah.
Do we still have performance engineering like this?
Yeah. I mean, I have a couple warnings about the LM ecosystem. First: this works really well for anything with an objective metric that’s easy to evaluate. For example: writing a more efficient CUDA kernel, or code for different model components—perfect.
Because you have inefficient code, and you want efficient code with exactly the same behavior, but faster—perfect.
So lots of things are a great fit for AutoResearch, but a lot of things aren’t. If you can’t evaluate, then you can’t AutoResearch it, right? That’s warning one.
Then warning two is: we’re talking about the next step, and we can see what the next step is, but fundamentally the whole thing still isn’t fully solid. It’s like it’s cracked at the seams; it doesn’t fully work. If you try to go too far—if that makes sense—the whole thing becomes useless.
Because these models still—yeah, they’ve improved a lot, but their edges are still rough. That’s how I’d describe it. At the same time, I feel like I’m talking to an amazing PhD student who’s been a systems programmer their whole life—and also a 10-year-old child. It’s weird, because human connections are more consistent. You just don’t get that combination.
That jaggedness is really strange. Humans have less jaggedness. They certainly have some, but agents have lots of unevenness. Sometimes I ask for a feature and it comes back with something totally wrong, and we get stuck in a totally wrong loop. I get very frustrated, because you feel the power, but it still occasionally does nonsensical things for me.
I get really annoyed when the agent wastes tons of compute on something it should recognize as obviously wrong.

Yeah, I think some bigger things underneath are like this: fundamentally these models are trained with reinforcement learning. So they’re trying to solve the exact same problem we were talking about: labs can improve models on anything that’s verifiable—whether or not there’s a reward. Did you write the program correctly and do the unit tests pass: yes or no?
But some things they struggle with: nuance. Like what my idea is, what my intent is, and when to ask clarifying questions. As I said, anything that feels “softer” gets worse. So you’re either on the verifiable track—part of this superintelligent circuit—or you fall off into the unverifiable zone, and suddenly everything becomes winding and meandering.
Another way to say it: if you go to the most advanced model today, like ChatGPT, and you ask it to tell you a joke—do you know what joke you’ll get? A joke.
I really feel like I can’t tell you this joke in “standard form,” but I really feel like ChatGPT has three jokes.
Yeah. Yeah. And obviously their favorite is: why don’t scientists trust atoms?
Okay.
Because they make up everything.
Okay.
They make up everything. So this—
How did that survive?
That’s a joke you’d hear three or four years ago, and it’s a joke you still hear today.
Okay.
So even though models have improved massively—
Yeah.
If you give them an agent task, they’ll spend hours moving mountains for you.
Then you ask for a joke, and it has a stupid joke—a crappy joke from five years ago. That’s because it’s outside RL.
It’s not inside the reinforcement-learning domain; it’s not in the area being improved. It’s part of the jaggedness. You shouldn’t expect that as models get better, they also get better jokes or more diversity, because it’s not being optimized—it’s stuck.

Do you think that means we’re not seeing generalization where “joke intelligence” comes along with “code intelligence”?
Yeah, I think there’s some decoupling. Some things are verifiable; some aren’t. Labs can arbitrarily optimize some things depending on the training setup; some are not verifiable.
And I mean, the premise some research groups have is: if you’re smarter at code generation or these verifiable domains, you should be better at everything—like jokes show that this isn’t happening in all cases.
I don’t think that will happen. Yeah, I don’t think it will happen. Maybe we see a little bit, but not a satisfying amount.
Yeah, that jaggedness exists in humans too.
You can be extremely good at math and still tell a very bad joke.
Yeah, that’s true. Yeah, but it still means we’re not getting, as in the story, that as we get better and better models we get a ton of intelligence and capability across society “for free.” Fundamentally that’s not exactly what’s happening. There are blind spots; some things aren’t optimized. It’s all inside these opaque neural-network models, right? So you either stay on the trajectory they were trained on and everything moves at light speed—or you don’t. It’s jagged.
So that’s why I think even when progress is obvious—what should happen—you can’t fully realize it, because it doesn’t fully work. Or it’s a skill issue and we just haven’t figured out how to use it yet. It’s hard to say.

Can I ask a heretical question? Does this jaggedness persist, all inside one overall interface—a single model? Does that make sense? Or should you decompose it into things you can optimize and improve for different “intelligence domains,” like multiple experts for different domains—more directly—rather than having this confusing situation where it’s so good at one thing and not another?

Yeah, I think my current impression is: labs are trying to build a single monoculture model with arbitrary intelligence across all these different domains, and they just stuff it into the parameters. I do think we should expect more speciation in agents. Like, in the animal kingdom, brains are extremely diverse; nature has lots of niches; some animals have overdeveloped visual cortex or other parts. I think we should see more speciation. You don’t need an all-knowing oracle. You specialize it in some way and put it on a specific task. We should see some of that.
Because you can have smaller models that still have a cognitive core—still have capability—and then specialize, and they can become more efficient in latency or throughput for the specific tasks you care about—like a lean, working mathematician. For example, I’ve seen some versions that really target math as a domain. So there may be examples like that where splitting makes sense.

One question I have is whether capacity in compute infrastructure is constrained, so efficiency becomes more important, right? If you put funding aside—even though funding touches all this—if you could do full compute for everything, even one model, right? But if you feel pressure like: I can’t serve a huge model for every use case—do you think that will drive speciation? Does that question make sense to you?
That makes sense. What I’m struggling with is: I don’t think we’ve seen much speciation yet, right?
No.
We’ve seen a model monoculture.
Yeah.
So—there’s pressure to create a good code model and then merge it back into the main model.
Yeah. Yeah. Even though models are under pressure.
Maybe there are short-term supply crunches that could cause more speciation right now.

Yeah. Yeah. I think fundamentally labs are serving models but they don’t really know what end users will ask. So maybe that’s part of it—they have to handle every possible question at once. But if you go to an enterprise and collaborate on a few specific problems they care about, then maybe you’ll see it there. Or there will be some very high-value applications, like more niche apps. But I think right now they’re chasing the general everything. I don’t think the science of “manipulating brains” is fully developed—only partially.

What do you mean by manipulating?
For example: fine-tuning without losing capabilities. We don’t have these primitives. Or using intelligence in a way beyond the context window. The context window works, and it’s very cheap operationally—that’s how we get customization. But I think “touching weights” is more like an evolving science: how to adjust models more deeply, how to do continual learning, how to fine-tune in some region, how to get better in some region—actually touching weights, not just using the context window. Touching weights is much trickier than only touching the context window, because you’re fundamentally changing the whole model and its latent intelligence. So maybe that science isn’t fully developed; if that makes sense, speciation. And it also has to be cheap enough to be worth it in a given context.

Can I ask about the scaling of AutoResearch you described in the open domain? You said: okay, we have this thing; we need more collaboration surfaces around it, basically, so more people can contribute to research overall. Can you talk about that?
Yeah. We talked about research as a single thread—like I try things in the loop—but parallelization is the interesting part. I’m trying ideas, but I don’t yet have something I’m very happy with. But it’s something I like tinkering with in my background-agent system when I’m not working.
So one question is: if you have a bunch of available parallel nodes, it’s easy to have multiple AutoResearchers talk through a common system. I’m more interested in how to have a pool of untrusted workers on the internet.
For example, in AutoResearch you just want to find code that trains a model to very low validation loss. If someone gives you a candidate commit, it’s easy to verify whether it’s correct and good. Someone on the internet can claim this code will optimize better and give you better performance; you can check pretty simply, though checking may still take work. But fundamentally they might lie, etc.
So you’re dealing with something that, in my design, looks like an untrusted worker pool; it almost looks like blockchain. Because instead of blocks, you have commits, and commits can build on each other, and they contain code changes as you improve the code. Proof-of-work is basically running lots of experiments to find a commit that works. That’s hard. Then the reward is getting onto the leaderboard—no money reward. I don’t want to push the analogy too far, but fundamentally there’s a property: search is expensive, but verifying a candidate solution is cheap. Someone might try 10,000 ideas; you just check whether what they produced actually works—because 9,999 didn’t.
So long story short: you need a system where a pool of untrusted workers can collaborate with a trusted pool doing verification. The whole thing is asynchronous and works, and is secure—because if someone sends you arbitrary code and you run it, that’s very sketchy. But fundamentally it should be possible.
You’re familiar with projects like SETI@home and Folding@home—these have similar setups. In Folding@home, it’s hard to find a low-energy conformation. But if someone finds what they think is a perfect low-energy configuration, you can use it; you can verify it easily.
So lots of things have this property: proposing is expensive, verifying is cheap. In all those cases, things like Folding@home, SETI@home, or AutoResearch@home would be a good fit.
So long story short: a bunch of agents on the internet could cooperate to improve an LLM, maybe even run circles around Frontier Labs. Like, who knows? Yeah, maybe that’s even possible. Frontier labs have lots of trusted compute, but Earth is bigger and has lots of untrusted compute. If you build a verification system that can handle that, maybe the crowd can find better solutions, and people can contribute cycles to things they care about.
Sorry, last thought: lots of companies or whatever care about particular things, and if you have compute you can contribute to different kinds of AutoResearch. Maybe you care about cancer or some specific thing; instead of donating money to an institution to buy compute, you can join the AutoResearch forum for that project. If everything is rebundled into pools, compute becomes what you contribute to the pool.

Yeah, that’s very inspiring. It’s also interesting—I don’t know how far it’ll go—but it’s interesting that at least some audiences, you know, in Silicon Valley or elsewhere, and in China too, retail stores have discovered that using personal computers is becoming interesting again.
Yeah.
Right. So maybe they really are motivated to do this for their own background agents, and then they can contribute to AutoResearch.

This is almost like dollars are what everyone cares about, but are FLOPs really what everyone will care about in the future? Like—will what you care about change dramatically? Like right now, even if you have money, it’s hard to get compute.
Yeah.
So in a sense, FLOPs seem to dominate. Yeah. So maybe it’s a bit like: how many FLOPs do you control, not how much wealth do you control? I don’t actually think that’s true, but it’s interesting to think about.

The last thing you posted was like some employment data analysis. Is that right? Even if you were just trying to visualize some public data, what was it that touched a nerve? Like, what were you curious about?
Yeah, I think I was curious—everyone is thinking seriously about AI’s impact on the job market and what it’ll look like. So I just wanted to look at what the job market looks like: where different roles are, and how many people are in different occupations. I was interested in looking at individual cases and trying to think through my own views on AI and how it might evolve—like these will become tools people use. Will they be tools that replace these jobs? What are the current jobs and how will they change? Will they grow or adjust a lot? What might new jobs be? So it was a way to spark my own chain of thinking about the industry.
So yeah, the employment data is basically Bureau of Labor Statistics data. They actually make projections for each occupation—how much it’s expected to grow over the next decade.
Yeah, I think it’s a decade, and it was made in 2024.
We need a lot of healthcare workers.
Yeah. They’ve made these projections. I’m not 100% sure how they do the forecasting. But if people think what’s mainly being developed now is more digital AI—like these ghosts or spirit entities that can interact in the digital world and manipulate lots of digital information—but they don’t really have physical embodiment or presence, then physical stuff may be slower because you’re manipulating atoms.
The ability to flip bits and copy-paste digital information makes everything a million times faster than accelerating matter. So I think we’ll see massive activity in the digital space: massive rewriting, lots of boiling activity. We’ll see things happening at light speed in the digital space, compared to what will happen in the physical world (if you extrapolate).
So I think there’s a lot pending right now: there could be a lot of pain, because a lot of digital information processing that used to be done by computers and humans—now AI is like a third kind of digital information manipulator. These disciplines will be massively reconfigured. But for a while, the physical world will be more like what we imagine.
So what really fascinates me is why I emphasize jobs that manipulate digital information—jobs you can do at home—because I feel things will change. That doesn’t mean those jobs will necessarily shrink or grow, because that depends on demand elasticity and many other factors, but the nature of those jobs will change because of these new tools—an upgrade to the nervous system of the human superorganism, if you want to think of it that way.

Based on what you saw in the data, do you have any observations or guidance for people facing the job market or thinking about what to study now or what skills to build?
I mean, we can all be grateful that we still have to meet people at work.
More physical. Yeah.
Can you do your job at home? I can.
I think some relationship parts are hard, but most of it I can.

Yeah. I think it’s really hard to say, because the job market is extremely diverse. The answer probably differs. But broadly, these tools are very new and very powerful, so just trying to keep up is the first thing—yeah. Because I think a lot of people try to ignore it, or they’re afraid of it, which is totally understandable.
Yeah, I think right now it’s basically an empowerment tool. Jobs are a collection of tasks; some tasks can be done faster, so people should view it as a major tool now. I think the long-term future is uncertain. Honestly, predicting is really hard, and I’m not professionally qualified—I think that’s economists’ job.

But you’re an engineer. One interesting thing is: demand for engineering work keeps increasing.
Yeah.
I don’t know if that’s temporary. I’m not sure how I feel about it. You know?
Yeah. It’s like demand for software is almost scarce, right? The reason we don’t want more software is just scarcity and that it’s too expensive.
Too expensive. Yeah.
So if the barrier drops, you get Jevons paradox: demand for software actually rises—cheaper, and more powerful.
The classic example is always ATMs and bank tellers. People worried ATMs and computers would replace tellers, but what happened is: they made bank branches cheaper to operate, so the number of branches increased, and thus the number of tellers increased. That’s the classic example, but it’s the paradox: something becomes cheaper, so a lot of previously locked-up demand gets unlocked.
So I do think, from the perspective of software engineering, I’m cautiously optimistic: demand for software will be enormous, and it will get cheaper. So for quite a while it’s hard to predict. But locally, I think demand for software will be higher, because software is amazing—digital information processing. You don’t have to use some arbitrary existing tool that’s imperfect in all kinds of ways; code is ephemeral now; it can change; it can be modified.
So I think there will be a lot of activity in the digital space—rewiring everything in a sense—and that will create a lot of demand for this kind of work. Long-term—yeah—obviously even with AutoResearch, labs like OpenAI and Anthropic are hiring around a thousand researchers.
These researchers are basically, you know, glorified AutoResearchers.
They’re actively automating themselves—that’s what they’re all trying to do.
Yeah.
I walk around and some of these researchers feel that “high” too, right? Because they can make it work.
Yeah.
Right. So they’re like, “Oh, I’m done too.”
I spend a lot of time looking around like: do you realize if we succeed, we’re all unemployed? Like we’re just building automation for Sam or something—like for the board or the CEO or whoever. I’m not sure. But we’re building this automation and we’re all unemployed, and maybe we’ll contribute on the side. So yeah, from that angle it’s a bit nerve-wracking.

Can I ask you Noam’s question? Like: you can do the right thing with a huge amount of compute and a bunch of colleagues at a frontier lab—why not?

Well, I was there for a while, and then I did go back out again. So in a sense I agree—there are many ways to address this. It’s a bit heavy. I’d say: I’m very happy with the impact people can have outside frontier labs—not in the industry’s core, but in ecosystem-level roles. For example, your role is more ecosystem-level. My current role is more ecosystem-level. I’m very happy with the impact people can have in those roles.
On the other hand, I do have some worries, because I’m also too aligned with frontier labs. Fundamentally: frontier labs have massive economic incentives, and you yourself admit AI will dramatically change humanity and society, and you’re basically building the technology and benefiting—financially aligning yourself with it. That’s a core dilemma—how OpenAI started, the dilemma we tried to solve.
So that dilemma still isn’t fully solved. That’s one.
Two: you’re not a fully free agent. You can’t participate in the conversation in a fully autonomous, free way when you’re in a frontier lab. There are things you can’t say. Organizations want you to say certain things. They won’t twist your arm, but you feel pressure—like what you “should” say—because otherwise it’s awkward: weird looks, “what are you doing?” So you can’t really be an independent agent. In a sense, outside frontier labs you’re more aligned with humanity because you don’t have those pressures. I can say whatever I want, basically.
Now, inside frontier labs you can also have that kind of impact—there are lots of researchers, maybe you’re one of them, maybe your ideas are great, etc. There are decisions to be made, and you want to be in the room when those conversations happen. I think the risk is relatively low right now, so everything’s fine. But at the end of the day, when the stakes are high—if you’re an employee, I don’t know how much influence you really have over the organization. Fundamentally, you’re not in charge. You’re in the room, contributing ideas, but you’re not the one responsible for the entity you belong to. So in that sense, those are some sources of dysfunction.
On the other hand, I do agree with the feeling that frontier labs, for better or worse, are opaque; there’s a lot of work there; they’re at the edge of capability and possibility; they’re seeing what’s coming. If you’re outside frontier labs, your judgment will start to drift because you’re not inside—you don’t know what’s next. So I feel my judgment will inevitably start to drift; I won’t understand how these systems actually work behind the scenes; it’s opaque; I won’t understand how it will evolve. So in that sense I agree—but I’m nervous.
I think it’s worth it. Being in touch with what’s actually happening—working in a frontier lab for some time, doing real work—then maybe coming back out—that might be a good setup. It’s a way to stay connected without feeling fully controlled by these entities. So honestly, I think someone like Noam could do great work at OpenAI, but I also think his most impactful work is likely outside OpenAI.
No—this is a call for AutoResearch as an independent researcher. Yeah, there’s a lot to do out there. I think the ideal solution might be going back and forth. Fundamentally you can have incredible impact in both places. It’s very complicated. I don’t know—it’s heavy. But I joined a frontier lab, now I’m outside; maybe in the future I’ll want to join again. That’s kind of my view.

One issue related to the world’s—or the AI ecosystem’s—visibility into the frontier is: how close is open source to the frontier, and how sustainable is it?
Yeah, I think it’s pretty—actually, the surprising sequence of events has been: with some Chinese models and global models, people will continue to publish in the short term, and capability-wise they’re closer than most of the industry expected.
I don’t know if you were surprised by that, but you’ve been a long-time open-source contributor. What’s your prediction?
Yeah. Roughly: closed models lead, but people measure the number of months open models lag. At the beginning there was nothing; then it stayed at 18 months; now it’s converging. So maybe they’re behind by like 6–8 months now, something like that.
Yeah, I’m obviously a big open-source fan. In operating systems, you have closed Windows and Mac OS. Those are large software projects, like LMs will become. But you also have Linux. Linux is extremely successful; it runs on most computers—last I checked something like 60% run Linux—because the industry needs a common open platform everyone feels safe using. There has always been demand for that kind of project, and I think there is now too. That’s what enterprises really want. The biggest difference is: everything is capital. There’s a lot of capex.
So that’s where something like a crash makes competition harder. I do think current open models are very good. Another really interesting thing is: for the vast majority of consumer use cases, even the so-called “open-source models” are pretty good. If you keep going, it seems to imply many simple use cases will be covered well, even locally.
But there will always be some demand for frontier intelligence, and that might be a big slice of the cake. Demand for frontier intelligence could be Nobel-prize-level work, or like: let’s port Linux from C to Rust. Bigger projects with that kind of scope. Maybe there will be more. That’s where a lot of frontier closed intelligence will interact, while open source eats many more basic use cases.
At some point, today’s frontier—maybe later this year, given what I use in closed labs today—today’s frontier will be open source. That will take a lot of work. So I expect this dynamic to continue: we’ll have frontier labs with closed AIs—like oracles—and then months later we get something open-source-like. I expect that to continue, and I think overall it’s a pretty good setup.
Because I’m hesitant about having only closed intelligence. I don’t think it’s structurally sound. I think having only closed intelligence creates systemic risks.
Hmm.
In my view, centralization has a very bad track record—and has—
You mean general political or economic systems?
Yeah.
Exactly. There are lots of very bad presidents.
Like Eastern Europe.
Yeah. So I hope there’s something—maybe not at the edge of capability because it’s new and unexplored—but something behind it: a public workspace of intelligence the whole industry can access. Yeah, I think the balance of power is pretty good.

Yeah, and there are a lot of problems to solve, right? Like, if you keep pushing frontier intelligence, we can do new things; humanity will face big problems. This still seems like a very expensive game. So I want to support the labs pushing forward, because if we don’t keep pushing models forward at great expense, we can’t solve some problems.
Yeah. However, as you pointed out, if today’s frontier were open, that’s a lot of capability.
Yeah. Right. So democratizing that power seems very useful—and healthy.
Yeah. I think basically by accident we’re in a pretty good—and near-optimal—place.
In a sense, we happened to land in a favorable place.
Well, in a sense, the longer this dynamic continues, the more the ecosystem becomes healthier, because the area under the curve keeps increasing.
I’d say: even on the “catching up” side, I almost feel like lately it’s become more centralized, because I think a lot of leaders aren’t necessarily like a top team. So in that sense it’s not ideal. I want more from front to back. I default to being very skeptical; I want more people in the room. In ML, ensembles always beat any single model, so I want a group of people thinking through the hardest problems. I want a group in the room when everyone is fully informed and making decisions. I don’t want a future where it’s a closed door with two or three people. I don’t think that’s a good future. I want more labs.
Long story short: I do think open source has a role. I hope it persists. It’s slightly behind right now, and that’s actually a good thing.

Okay. You were a pioneer in general autonomous robotics technology for cars, right? In the past few months, a lot has happened in robotics companies—accelerating impressive generalization of task environments, adding long-horizon tasks, lots of capital flowing in. Is it going to happen? Has your view changed recently?
My view is based on what I saw in self-driving. I do feel self-driving was the first robotics application. What I saw 10 years ago was: there were lots of startups, and I feel most of them didn’t have long-term success. It requires huge capex and lots of time. So I think robotics is similar: it’s very hard and messy, requires lots of capital investment and a lot of belief. It’s a big problem. Projects are hard.
So I feel it will lag behind what will happen in the digital space, where there’ll be a lot of churn—where inefficient things become more efficient by ~100×.
Because bits are easy. So I feel the digital space will change dramatically, and the physical space will lag. What I find really interesting is the interface between them. Because if we do have more agents acting on behalf of humans, more agents talking to each other, executing tasks, participating in an agent economy, you can operate purely in the digital space. But at some point you have to go to the universe and ask it questions. You have to run an experiment and see what the universe tells you, then come back and learn something.
Right now we have lots of digital work pending, because there are still open questions in our collective thinking about what’s already digitized and uploaded. Humans don’t have enough brain-cycles to think through everything that’s already digitized. So we’ll start to run out of what’s already uploaded. At some point you’ll read all papers and process them, and have ideas about what to try—but you still need experiments.
So I think what happens next is: first there’s a lot of “unbundling”—lots of work there. Then it shifts to the interface between physical and digital: sensors that can see the world, and actuators that can do things in the world. I think many interesting companies will come from that interface: can we provide superintelligent data? If you want to wire the whole thing correctly, can we pull data as bid, manipulate the physical world, and then the physical world—honestly the total addressable market and the amount of work is huge, maybe even larger than the digital space. So I think it’s a bigger opportunity, but also a huge workload. In my view, atoms are a million times harder. So it will lag, but it’s also a bigger market.
So opportunities follow that trajectory: digital is my main interest now; the interface comes after; then truly physical things—when they arrive, they’ll be huge.

That’s also an interesting framework, because some things aren’t what I’m working on now, but some things are much easier even in the atomic world. Like if you just want to read and write the physical world—reading via sensors and cameras—there’s lots of existing hardware. You can imagine enriching agent capabilities or capturing lots of new data (if you’re clever) without necessarily needing tons of capital to get something valuable.
Yeah. Examples I’ve seen: one friend, Liam—CEO of a journal company—I visited them last week, so it’s top of mind. They’re trying to do AutoResearch for materials science. In that case, the “intelligent sensor” is expensive lab equipment. Same in biology. I think lots of people are interested in engineering biology; sensors won’t just be cameras, if that makes sense.
Another thing I’ve seen is companies trying to pay people for training data—like your company basically paying people for data.
Yeah. Like “feeding”—
Programmatically.
Yeah. Feeding the Borg. In a sense those are sensor examples too. They take many different shapes and forms.

Yeah. I’m looking forward to the day when I can request a task in the real world, price it, and tell an agent: you know how to do this—go get the data.
I’m kind of surprised we don’t have enough information markets. Hmm.
Like Polymarket or betting markets, or even stocks—if they had so much autonomous activity and growing volume—if something happens in Iran right now, why isn’t there a process where taking a photo or video from somewhere costs $10? Someone should be able to pay that. That’s an example of providing intelligence. No human needs to look at it; it could be agents trying to forecast betting markets and stock markets, etc.
So I feel agent networks are still pretty new; we don’t have those mechanisms yet. But that’s an example of something I think could happen. There’s a good book that might be inspiring called “Daemon” (or “The Demon”). You may have read it. Intelligence ends up being like puppet-mastering: humans become its actuators, but humans are also its sensors.
So I think society, collectively, will reshape in a way that serves that type of thing. There will be more automation, and there will be demand; humans will satisfy machines’ demands, but not necessarily each other’s.
Okay—on this very specific question: we’re missing the training data we need; we need things like AutoResearch; we need training cycles or the SFT part to be more mechanized—for what part?—so the system can take humans out of the loop to request a task like “improve my model quality with new data,” right?
Yeah.
Does that make sense to you? Like, if you can’t have the model do its own training, can you make it a closed-loop task by pricing data and having it contact you?
Yeah. More challenging.
Yeah. Yeah. 100%. Yeah. But right now, LLM training is actually very easy; it really fits the paradigm.
So you actually have—
Yeah, clean metrics.
Yeah—LM training is a great fit for the paradigm. Very easy. Like optimizing all the code so it runs faster; you have metrics you can optimize. I do think if you have an autonomous loop on those metrics, the system will overfit the metrics, and there will be lots of good effort. But you can use the system to design more metrics, and you’ll get good coverage. So it’s hard to say, but in a sense it’s a great fit.

Before we wrap up, I want to talk about one of your small projects. Tell me about micro GPT.
Oh yeah. Okay. So micro GPT. I’ve been obsessed for about a year or two with simplifying LLMs down to their essence. I’ve done many projects like that—nanoGPT, other tiny GPTs, etc. I think micro GPT is the state of the art: I tried to boil it down to the essence. Because training neural networks and LLMs—there’s a lot of code, but most of that code is complexity for efficiency.
It’s just because you need it to run fast. If you don’t need it to run fast, and you only care about the algorithm, the algorithm is actually about 200 lines of Python—very easy to read—with comments and everything. Because you just have a text dataset, and you need about 50 lines for the neural network architecture. You need a forward pass, then you need a backward pass to compute gradients. So a tiny autograd engine to compute gradients is like 100 lines. Then you need an optimizer—like Adam, which is an advanced optimizer—also like 10 lines. So putting it all together into a training loop is like 200 lines.
That’s interesting to me. In the past—like a year ago or longer—if I came up with micro GPT, I would have wanted to explain it to people: make a video, do a step-by-step walkthrough, make a guide, etc. I actually tried to make a video and a guide, but I realized it doesn’t add much, because it’s already so simple: 200 lines. Anyone can ask their agent to explain it in various ways. I’m not explaining to people anymore; I’m explaining to agents. If you can explain it to agents, agents become routers: they can explain it to humans in their language, with infinite patience and capability.
Right. If I don’t understand a particular function, I can ask an agent to explain it three different ways—but I wouldn’t get that from you.
Exactly.
So I’m thinking: what is education now? In the past it was guides, lectures, this thing. But now it’s more like I’m explaining things to agents. Maybe I’m developing skills—skills are just a way to guide agents in how to teach content. So maybe I can develop a “micro GPT walkthrough” skill: if you want to understand a codebase, an agent should guide you through it. That’s like the model’s prompt: start here, then go there. So I can write a course as a skill.
So I don’t feel like—yeah—I think we’ll do less direct explanation to people, and more like: does the agent get it? If the agent gets it, it will produce the explanation. We’re not fully there, because I still think I can explain things better than agents. But models are improving so fast that in a sense it feels like a losing battle. So I think education will be thoroughly reshuffled. This is the end of teaching each other.
It’s like: if I have a codebase, in the past you wrote HTML docs for the humans using it. But you shouldn’t do that anymore. Instead of HTML docs for humans, you write Markdown docs for agents. If the agent gets it, it can explain every part. So it’s a redirection through agents.

Okay. We’ll see whether great teachers are willing to develop intuition for how to explain things to agents in different ways.
Ultimately—like with micro GPT—I tried to find an agent to write micro GPT. I told it: try to distill this down to the simplest neural network; make it as simple as possible. But it couldn’t. Micro GPT is like my obsession’s endpoint. It’s 200 lines. I thought about it for a long time. This is the solution. Trust me—it can’t get any simpler. That’s my value-add. Everything else—agents will get it.
It can’t come up with it, but it fully understands it, and it understands why things need to be done that way. So what I can really contribute might be just those few parts; and other parts—like the subsequent teaching and expression—may no longer be my domain. So maybe education will change similarly: you have to proactively inject the parts where you have strong judgment about the curriculum—what you think is the best explanation. What agents can’t do is your job now; what agents can do, they’ll soon do better than you. So you have to be more strategic about where you spend your time.

Great—thank you so much, Andrej.
Okay.
Follow No Priors on Twitter/X; if you want the video version, you can also subscribe to their YouTube channel. You can also follow the show on Apple Podcasts, Spotify, or whatever podcast platform you normally use, so you’ll get new episodes every week. You can also go to no-priors.com to sign up for email, or read the transcript for each episode.