AgentBug-Smith 论文提出自动复现开源智能体系统真实框架缺陷的方法，并构建包含 200 个可复现缺陷的 Live-Harness-Bench。3 个编码智能体中最好的一个仅修复 9% 的缺陷，而常规软件缺陷报告修复率约 40%；基于过往修复总结的简短指南使一个智能体在 79 个未见过缺陷上的正确修复数从 1 个提升到 6 个。
Self-improving AI agents will need to fix their own code.
And this paper from top US+China labs, shows coding agents miss most such bugs but improve with lessons from past fixes.
that real bugs in agent harnesses, can be automatically turned into a growing set of runnable tests.
An agent's own code is everything around the model: tool calls, memory, and prompts. Its bugs depend on live model calls, which makes them hard to recreate and test.
So the researchers built AgentBug-Smith, which turns real GitHub bug reports into runnable tests. The result is a 200-bug benchmark that keeps growing.
The best of 3 coding agents fixed just 9% of those bugs, versus about 40% reported on regular software bugs. A short guide of lessons from past fixes lifted an agent from 1 to 6 correct fixes on 79 unseen bugs.
Before trusting a coding agent with your agent's code, try it on bugs you've already fixed.
来源：Rohan Paul · x.com