对编码代理的最终测试不是本地编辑—— 这是整个回购协议的演变，目前，生存率为 5.4%。 今天我们要发布的是SWERefactor Bench，长期、全存储库软件堆栈迁移的基准。 编码代理越来越擅长修复错误。 但他们能否重构…
对编码代理的最终测试不是本地编辑—— 这是整个回购协议的演变，目前，生存率为 5.4%。 今天我们要发布的是SWERefactor Bench，长期、全存储库软件堆栈迁移的基准。 编码代理越来越擅长修复错误。 但他们能否重构整个系统，C→Rust，Maven→Gradle，POSIX → WebAssembly？ 我们跨项目构建了 20 个真实迁移，包括SQLite、zlib、libsodium 和GraphHopper。 520 次运行。只有 28 名选手通过了全部 3 个阶段。13/20任务没有人解决。 系统规模的迁移仍然是开放的。
查看英文原文
The ultimate test for coding agents isn't local editing— it's whole-repo evolution, and right now, the survival rate is 5.4%. Today we’re releasing SWE Refactor Bench, a benchmark for long-horizon, whole-repository software stack migration. Coding agents are getting very good at fixing bugs. But can they refactor an entire system, C → Rust, Maven → Gradle, POSIX → WebAssembly? We built 20 real migrations across projects, including SQLite, zlib, libsodium, and GraphHopper. 520 runs. Only 28 survived all 3 stages. 13/20 tasks were solved by nobody. System-scale migration is still wide open. Full breakdown 👇 GitHub: [https://github.com/Einsia/SWE-Refactor-Bench] Paper Link: [https://arxiv.org/abs/2608.23564] Einsia Website: [https://lab.einsia.ai/swe-refactor-bench/]
The ultimate test for coding agents isn't local editing— it's whole-repo evolution, and right now, the survival rate is 5.4%. Today we’re releasing SWE Refactor Bench, a benchmark for long-horizon, whole-repository software stack migration. Coding agents are getting very good at fixing bugs. But can they refactor an entire system, C → Rust, Maven → Gradle, POSIX → WebAssembly? We built 20 real migrations across projects, including SQLite, zlib, libsodium, and GraphHopper. 520 runs. Only 28 survived all 3 stages. 13/20 tasks were solved by nobody. System-scale migration is still wide open. Full breakdown 👇 GitHub: [https://github.com/Einsia/SWE-Refactor-Bench] Paper Link: [https://arxiv.org/abs/2608.23564] Einsia Website: [https://lab.einsia.ai/swe-refactor-bench/]
The ultimate test for coding agents isn't local editing— it's whole-repo evolution, and right now, the survival rate is 5.4%. Today we’re releasing SWE Refactor Bench, a benchmark for long-horizon, whole-repository software stack migration. Coding agents are getting very good at fixing bugs. But can they refactor an entire system, C → Rust, Maven → Gradle, POSIX → WebAssembly? We built 20 real migrations across projects, including SQLite, zlib, libsodium, and GraphHopper. 520 runs. Only 28 survived all 3 stages. 13/20 tasks were solved by nobody. System-scale migration is still wide open. Full breakdown 👇 GitHub: [https://github.com/Einsia/SWE-Refactor-Bench] Paper Link: [https://arxiv.org/abs/2608.23564] Einsia Website: [https://lab.einsia.ai/swe-refactor-bench/]
- 来源
- X @_akhaliq · x-akhaliq
- 证据角色
- x_project_network
- 时间状态
- known
- 用户判决
- 未判