A concise review of Haebin Seong, Li Yin, Haoran Zhang, and Zhan Shi’s proposal for agents that improve the systems around them.

Evidence status: This paper proposes a framework and two algorithms. It does not report experiments, benchmark results, costs, or comparisons with human engineers. The authors say empirical results will come later.

The idea

A model is only one part of an AI agent. The harness around it includes prompts, tools, permissions, execution environments, model settings, and rules for planning, retries, and verification.

Building that harness usually requires people to study failures and manually adjust the system. The paper asks: Can agents automate that engineering work?

It proposes two connected optimization loops.

1. Improve a harness for one task

The Harness Evolution Loop uses three agents:

  1. A worker attempts the task with the current harness.
  2. An evaluator checks the trace and final state against explicit success criteria.
  3. An evolution agent studies the failures and modifies the prompts, tools, orchestration, observations, or model configuration.

The task is repeated with each new version. The system keeps the best-performing harness and retains failed attempts in its history, reducing the chance that it repeats an unsuccessful change.

2. Improve the improvement process

The second level, the Meta-Evolution Loop, runs that process across many tasks. It modifies the evaluator, scoring method, information passed between agents, and other rules governing harness evolution.

The goal is to learn a reusable procedure that can quickly create a specialized harness for a new task. This resembles meta-learning: the inner loop adapts to one task, while the outer loop improves the way adaptation happens.

The authors’ conclusion

The authors conclude that harness engineering can be treated as an automated optimization problem. The inner loop improves one task-specific harness; the outer loop learns how to make that improvement process faster and more reliable across tasks.

Their long-term goal is a system that can take a general-purpose agent, point it at a new domain, and automatically evolve it into a specialized agent without manual harness engineering.

That outcome is a proposal, not a demonstrated result. The paper defines how it should be evaluated—using convergence speed, final task performance, and consistency on held-out tasks—but provides no measurements yet.

What is useful here

The paper offers four practical ideas that teams can use now:

  • Optimize more than prompts. Agent failures may come from missing tools, confusing observations, poor retry logic, or weak verification.
  • Separate execution from evaluation. The worker should not be allowed to certify its own success. Check results against external state whenever possible.
  • Preserve failed experiments. A complete history helps prevent repeated mistakes and makes improvement evidence-based.
  • Keep the best version. New changes can regress. Compare versions and retain a known-good harness instead of assuming that iteration always means progress.

What remains unproven

The central limitation is the evaluator. An evolution loop will optimize whatever score it receives—even when that score misses safety, quality, or long-term consequences. A weak evaluator can produce a system that passes the test while failing the real objective.

Other unanswered questions include:

  • Does this approach outperform skilled human engineers or simpler optimization methods?
  • How much time, compute, and money does the two-level search require?
  • Do improvements transfer to genuinely different tasks and domains?
  • Can an evolution agent change tools without weakening permissions or safety checks?
  • How reliably can it avoid exploiting gaps in its evaluator?

The method is most promising for repeatable tasks with clean resets, observable outcomes, and objective success checks—such as software tests, structured data processing, and some browser workflows. It is less convincing for subjective or high-stakes work where quality is difficult to measure immediately.

My conclusion

This is not yet “the last harness.” It is a promising design for continuous harness engineering.

The paper’s strongest insight is that agent performance depends on the whole system, not only the underlying model. Its worker–evaluator–evolution structure is a sensible way to turn verified failures into system improvements.

But automating harness changes does not eliminate human judgment. It moves that judgment upstream. People still need to define success, create reliable evaluations, set boundaries the evolution agent cannot modify, and decide when the resulting system is safe enough to use.

The practical lesson is simple: the better an agent becomes at optimizing itself, the more carefully we must design what it is optimizing for.


Paper reviewed: Haebin Seong, Li Yin, Haoran Zhang, and Zhan Shi, “The Last Harness You’ll Ever Build,” arXiv:2604.21003v3, May 1, 2026. Licensed under CC BY 4.0.