Does Multi-Agent Debate Improve AI Feedback on Research Papers?
创建于 更新于
摘要
This study pre-registered and compared three AI feedback configurations applied to finished economics meta-analyses: a single-pass model, a cross-model adversarial debate, and a cross-agent workshop. Across 55 papers (44 with author rankings), authors most often found the plain single-pass report more useful for improving their work than the more computation-intensive debate configurations. The results held under multiple robustness checks and persisted irrespective of whether author or AI judges performed the rankings. Notably, authors recalled human referee feedback as more useful than the AI reports, while AI judges consistently ranked the human referee last when placed alongside AI-generated reports. An external AI judge could reverse the ordering, underscoring caution against substituting AI judgments for authors. The findings highlight the bounded returns to extra compute in fixed-length feedback and suggest that, at least in this setting, simpler AI feedback can be more valuable to authors than more elaborate multi-agent configurations [page::4][page::9][page::13][page::14][page::16].
