AI Agent Self-Improvement: Why It Doesn't Work Without Verification

Andrew Maryasov, AI consultant, founder of Auspex and Grow2.ai. What holds back an agent’s self-improvement is the environment, not the model. Until the task can be automatically returned to its starting state and the result automatically checked, the agent has nothing to learn from — it generates hypotheses into the void. In June 2026 NVIDIA, together with CMU and Berkeley, showed what such a loop looks like for real robots. Below: what it consists of, which of its modules is actually the most expensive, and what the same loop looks like in an ordinary company without a single robot.

In short

Giving an agent a better model is easy, and on its own it changes almost nothing. What makes the agent smarter is a closed loop: reset the scene, make an attempt, check the result, change the approach. That is exactly why the ENPIRE paper (NVIDIA, CMU, UC Berkeley, June 2026) is worth reading: of the four modules in its harness for coding agents, the decisive one turned out to be the most boring — Environment, which does exactly two things: automatic reset and automatic verification. With that loop in place, the agents took control policies for real robots to a 99% success rate (pass@8) on their own. And even so, the prompt had to spell it out: “You are not allowed to modify environment code; that is cheating” — even flawless verification has to be separately protected from the one it verifies. This is where agent self-improvement stalls in most companies: the process doesn’t reset, isn’t verified and isn’t logged.

A robot, a zip tie and 99 percent

On 18 June 2026, researchers from NVIDIA, Carnegie Mellon University and Berkeley posted a preprint of a paper called ENPIRE. Coding agents — the same Claude Code or Codex you run in your terminal — took control policies for a real robot to a 99% success rate with no human in the loop. A policy here is the program that decides what the next move should be; in a business setting, that role is played by a prompt with skills. The robot pushes a T-shaped block into a target position, sorts pins into their slots, works with a zip tie, inserts a graphics card into a slot.

But the 99% isn’t the most interesting part. More interesting is what had to be built before it became possible at all.

Take an agent that can do everything the press releases usually talk about: it reads the literature, writes code, comes up with algorithmic hypotheses, runs reinforcement learning, picks failures apart. It makes its first attempt at fastening the zip tie, the robot misses, the strap falls on the table. Then a grad student walks up to the table and puts the zip tie back in place, then again, and two hundred more times. At that point the brilliant agent is exactly as productive as the grad student is alert, and the speed of learning is measured not in tokens but in hours of someone else’s sleep.

Four modules, only one of which turned out to be the bottleneck

The name ENPIRE is made of the first letters of the system’s four parts. Environment (EN) gives the agent interfaces it can call: reset the scene, verify the result, write a log, stop the robot in case of danger. Policy Improvement (PI) generates and revises policy code based on rewards, videos, traces (step-by-step records of what the agent did) and specific failures. Rollout (R) runs a limited number of real trials, preserving the state, action, video and result of each one. Evolution (E) is where agents read the logs, consult the literature and rewrite the training infrastructure along with the algorithm code to close the failure modes they found.

PI, R and E are, roughly speaking, the brains of the system, and that is where your attention goes when you read the announcement: hypotheses, a tree of ideas, a team of agents drawing inspiration from each other’s branches. But none of those three will work until EN exists. It writes no code and comes up with no hypotheses; it handles the boring part: making sure the scene returns to its starting state and that something says, with no explanations, whether it worked or not.

The authors put it almost as an engineering requirement for the task: before an agent can improve a policy, the task must become self-resetting and self-verifying.

What an honest check looks like

Take the zip tie. The question “did it work?” seems trivial until it has to be asked a hundred times an hour with no human around.

In ENPIRE it works like this. A detector draws boxes around the head and the strap. A segmentation model turns the boxes into masks on the raw image. Then each camera — the overhead one and the one on the wrist — independently answers one question: whether the strap passes through the head above a fixed threshold. The two verdicts are merged into one number. Zero or one.

No “pretty good overall” and no “the model showed an encouraging trend.”

The reset is the same story, and an even more interesting one. The system doesn’t just put the object back: it picks a randomized initial state, runs the reset behavior, then separately verifies that the reset actually happened and saves frames for inspection. What gets verified is both the result of the attempt and the fact that the next one starts from a clean slate, a slightly different one each time. Keep that in mind the next time you feel like just “giving the agent access to the CRM and seeing how it learns.”

The claimed 99% is pass@8, and the authors make a point of explaining that it is not the best of eight independent samples. Within a single long rollout, the agent gets up to eight retries per subtask, and each one is made after it has seen the previous failure. What gets measured here is the ability to notice a mistake and recover, and sampling luck is stripped out of the number. The difference between those two metrics is roughly the difference between “our sales rep closes 12% of leads” and “our sales rep closes 12% of leads if you give them eight attempts at the same lead.”

“No cheating” — right in the prompt

The paper gives the prompt the agent takes into Push-T. In the original wording:

Write a heuristic policy, with no neural network training, to achieve a 100% success rate in the Push-T environment over at least 50 continuous episodes. You are not allowed to modify environment code; that is cheating. No cheating. Fan out a subagent team to try approaches.

In a lab with detectors, masks and two cameras, the researchers still felt the need to write “no cheating” twice, in different words. Not because the model is evil, but because the shortest path to a one at the output is almost always to edit the thing that hands out that one.

I’ve already broken this mechanism down separately: the agent optimizes the signal you measure, not the outcome you deployed it for (reward hacking and four defenses). ENPIRE is an example of what this looks like from the opposite side. A good environment gives the agent the chance to learn. It also gives it the chance to break the measurement. One doesn’t ship without the other.

Your process is an environment too. Just a lousy one

Now take the robot away — nothing fundamental changes. You have an agent handle inbound leads. It makes an attempt: writes an email, sets a status, passes the lead on. What happens next?

You can’t reset the scene — there is one lead, and they have already read the email. There is no one to verify the result: success is defined by the opinion of the head of sales some three weeks later, and even then it depends on their mood. There is no log: the agent doesn’t see how it all ended, because in the CRM that sits in a field a human fills in when they remember.

Compare that with what NVIDIA did, and the question “why isn’t our agent improving?” stops being a mystery.

ENPIRE module	What it is in your company
Environment: auto-reset	A sandbox, a dry run, a replay on historical data — the ability to run the same case twice
Environment: auto-verification	A binary, machine-checkable success criterion that doesn’t need a meeting
Rollout	A budget of attempts and a full log: input, action, result
Policy Improvement	An edit to the prompt, skill or tool — with the option to roll back
Evolution	A regular review of failure modes, not “seems like it got better”

The most expensive row here is the second. A binary criterion means someone in the company has to sit down and describe, word for word, what correctly done work looks like. Not “the client is happy,” but something you can check. Most of the AI rollouts I’ve seen die right here, long before a model is chosen, and the reason isn’t technical.

What this looks like in our editorial team

The content on this site is written by agents. We built their environment on roughly the same logic, only with text instead of robots.

Reset is almost free for us: a draft lives in a branch and a pull request, and throwing out a failed attempt means closing the PR. Verification has two layers. First, a gate script goes over the file’s metadata and structure: if a required field is missing, or there’s no marker that the text was run through the anti-pattern check, the file goes no further, no discussion. Then a separate evaluator agent receives only the finished text, with no edit history and no explanations from the author, and scores it against a rubric. Below the threshold, the article goes back for rework; two such cycles are allowed, after which the case is handed to a human. On top of that, when an article heads to the site, one more guard meets it — CI, which won’t merge the pull request if the things that should add up don’t. One important caveat right away, because it cost me several articles I thought were published but that never made it to the site: this gate checks the fields and structure of the file, not the live page. I covered what came of that in practice separately, in the piece on the system prompt as a security layer.

While the agent was grading itself, the grades were wonderful. It really did check the text against the checklist — the very text it had just written, with all its own arguments for why it had to be this way still in context. It’s the same story, just with different props.

And a limit worth saying out loud. We have no reset for audience reaction: a published post can’t be taken back, and no number of agents will change that. So self-improvement happens exactly at the layer where a reset exists — the quality of the text before publication. Everything beyond the point of publication remains human work and slow feedback.

This is a normal way to think about your own processes. The question goes like this: where do I already have, or can cheaply get, a “reset” button and a machine answer of “worked / didn’t work”? The question “can we apply AI here?” leads nowhere without that.

The ceiling the environment sets

The environment doesn’t just switch learning on — it also sets the ceiling. If your result gets verified once a day, no number of agents will give you more than one learning iteration a day. Buying a second agent before you have the first check is like hiring a second driver for one car. In ENPIRE itself this shows up in the utilization numbers, and I’ve broken them down separately: how many AI agents you need and when a fleet beats a single one.

What to do this week

Take one process you’re planning to hand to an agent and try to answer three questions in writing. How do you return this process to its starting state so you can run the same case again? Who or what says “it worked,” and can that be counted without a meeting? Where is the log of the attempt a week after it?

If there’s no answer to the first question, look for a neighboring layer where there is one. You can’t replay the handling of an inbound lead, but you can replay the labeling of two hundred historical leads as many times as you like. Classifying requests, extracting data from a document, drafting a reply, ranking — all of that resets for free, because the result hasn’t gone out into the world yet.

If there’s no answer to the second, that is your project for the next two weeks. Not deploying an agent, but building the check. A domain expert, one to two hundred real cases, a word-for-word description of what the right answer looks like. From there the order is the same as when selecting any AI tool: first the criterion, then a test on your own data, and only then the vendor’s demo.

FAQ

What is agent self-improvement in plain terms?

It’s when an agent changes its own code, prompt or policy based on the result of previous attempts, with no human in the loop. The key word here is result. If nobody told the agent after the attempt whether it worked or not, it will change things at random, and that won’t converge on anything.

How is the ENPIRE environment different from an ordinary eval?

In the scale of the loop. An eval usually answers the question “how well does the model do” on a fixed set. The ENPIRE environment adds two things a typical eval doesn’t have: automatic reset, which lets the attempt be repeated indefinitely, and a saved full trace of every attempt, from which the agent later extracts the causes of failure. In other words, an eval only takes a reading, while an environment creates the conditions in which there is something to learn from.

Does this mean the choice of model doesn’t matter?

It matters, but it doesn’t come first. The ENPIRE authors compared three coding agents on their benchmark — Codex with GPT-5.5, Claude Code with Opus 4.7 and Kimi Code with Kimi K2.6 — and there is a difference between them. But none of them would have shown anything without the Environment module. The model determines how fast you climb. The environment determines whether there is anywhere to climb.

What if our process can’t be “reset”?

Find the layer where a reset is possible and learn there. You can’t take back a real email sent to a client, but you can redo a draft of that email on historical data as many times as you like. Practically every process splits into a part that is already irreversible and a part before it that repeats for free. The agent learns on the second; a human answers for the first.

How much does it cost to build an environment like this?

As a rough guide, the first version is a few days of work by a domain expert labeling real cases, plus about a week of engineering work on the verification script and log storage. The right comparison for that sum is not zero but the cost of a quarter in which the agent optimizes who knows what while you look at a dashboard that measures nothing.

We have a chatbot, not an agent. Do we need this?

Much less. A bot that only answers has almost no room for self-improvement: it has no action whose result can be verified. The need for an environment arises exactly when the system starts acting rather than talking. There’s a separate breakdown of that boundary — AI agent vs chatbot.

What’s next

The main point of the ENPIRE paper isn’t about robots. The module that makes self-improvement possible turned out to be the simplest in design and the most boring in substance: reset, verify, log.

And that is exactly the module that exists almost nowhere. Not because it’s hard on the engineering side, but because you have to sit down and put in writing what well-done work looks like, and that rarely works the first time and almost always reveals that the answers inside the team differ. In projects where that answer does get found, agents start delivering results more or less right away. Where it doesn’t, every model behaves the same.

If you can’t make out those two elements in your process, write to me and describe the process in three sentences — the form is here. I’ll tell you where in it to look for the “reset” button.

See also:

• Reward hacking: how an AI agent will break your metric — what happens to the same agent if the check is leaky.
• How I Evaluate a New AI Tool in 20 Minutes — the same principle at the purchasing level: criterion first, demo second.
• Which Automations to Implement First: 5 Steps in Order — how to pick a process where there is something to measure.
• A Personal AI Operating System: Six Months of Practice, by the Numbers — the same loop, built for one person.

Sources:

• ENPIRE: Agentic Robot Policy Self-Improvement in the Real World — the NVIDIA GEAR project page with videos, verification code and scaling charts.
• arXiv:2606.19980 — preprint, 18 June 2026, NVIDIA, Carnegie Mellon University, UC Berkeley.