NVIDIA 研究人员让小型 AI 智能体在每步起草 8 个命令选项,再由强前沿模型作为评判选择执行,成功率从 50% 升至 68%,且无需重新训练。该方法针对终端 AI 智能体常直接执行首个命令、单条错误命令就会破坏后续步骤的问题。若让小模型评判自己的方案,提升会明显缩小,因为评判者难以区分选项优劣。
Nvidia's new paper tell before giving your agent more options, give it a better judge.
A single bad command can derail an AI agent, so have it draft a few options and let a smart judge pick before acting.
And in this way, You can make an AI agent far more reliable without retraining it.
AI agents that work in a terminal usually run the 1st command they come up with. A single bad move, like installing the wrong package, can throw off every step after it.
NVIDIA researchers had a small agent draft 8 options at each step and let a judge choose which to run. With a strong frontier model as the judge, its success rate jumped from 50% to 68%, no retraining needed.
When the small model judged its own drafts, the gains were much smaller. More options don't help much if the judge can't tell them apart.
来源:Rohan Paul · x.com