Two tests address different problems
OpenAI’s October 19 report separates building software from migrating it. MultiLangBench asks models to implement a SQLite-compatible database engine across 20 languages. A second evaluation migrates COBOL programs to other languages while preserving their observable behavior. Passing one type of task does not establish success on the other.
The reported gains
OpenAI reports an average MultiLangBench pass rate of 87.4% for GPT-6 Astra at maximum reasoning effort. Results vary by language: COBOL reaches 72.1%, while Fortran reaches 83.5%. In the separate migration evaluation, GPT-6.1 Sol achieves 86.6% accuracy at low reasoning effort and 97.9% at higher effort.
Running code is not enough
OpenAI highlights decimal handling as a migration risk: changing COBOL’s treatment of excess decimal places can alter financial outputs even when a rewrite executes successfully. Its migration test therefore requires all hidden tests to pass with identical observable output. Production migrations still need expert review and checks against their own business rules and dependencies.
Limits of the evidence
These are provider-run evaluations, not an independent audit of a production migration. They do not establish that legacy systems can be replaced without specialist oversight.