AI NewsModels & agentsAnnouncement
OpenAI's GPT-6.1 Sol scores 97 percent on its COBOL migration test
OpenAI released two coding evaluations for low-resource languages and reported that GPT-6.1 Sol scores 86.6 percent on its COBOL migration test at low reasoning effort and 97.9 percent at higher effort, against 5.4 percent for GPT-5.6 Sol.
Image: OpenAI
Why it mattersA team that has been quoting six-figure prices to rewrite a COBOL system now has a vendor-reported score near a full pass on a byte-for-byte test, which changes how the first conversation with a client goes even before anyone trusts the model in production.
Most of the code that moves money through ATMs and card networks today is written in a language first standardised in 1960, which is one reason OpenAI picked it to test its newest coding models. The company published two coding evaluations on 19 October 2026 that measure how well a model rewrites that code into another language while keeping the output the same to the cent.
The two tests
The first, MultiLangBench, asks the model to build a SQLite-compatible database engine from scratch in each of 20 programming languages. The list is C, C#, C++, COBOL, Common Lisp, Fortran, Go, Java, JavaScript, Lean, OCaml, PHP, PowerShell, Python, Ruby, Rust, Scala, Swift, Visual Basic and Zig. Each candidate program takes an initial database and a SQL query, writes a modified database, and OpenAI's test harness compares the file against the reference SQLite build. OpenAI says it is releasing the test suite and the per-language results on GitHub.
The second asks a model to migrate a COBOL program to C++, Java, Kotlin, Python or Rust, then runs hidden tests against the rewritten program. A migration counts as successful only when every observable output matches the original byte for byte. OpenAI's post gives one reason the bar is set that way: COBOL's fixed-point arithmetic drops extra decimal digits unless rounding is explicitly asked for, so a rewrite that rounds can finish without an error and still send the wrong amount of money.
The numbers OpenAI reports
On MultiLangBench, OpenAI says GPT-6 Astra scores an average 87.4 percent across the 20 languages at maximum reasoning effort, against 26.8 percent for GPT-5.6 Sol. The company adds that Astra scores above 83 percent in 19 of the 20 languages, with COBOL the hardest at 72 percent. GPT-6 Luna, the cheaper option, reaches 40.3 percent on the same test at an estimated API cost of $0.77 per sample, against $3.48 for GPT-5.6 Sol, which OpenAI reports as about 78 percent lower.
On the COBOL migration test, OpenAI says GPT-6.1 Sol scores 86.6 percent at low reasoning effort against 5.4 percent for GPT-5.6 Sol, at 61 percent of the API cost. At higher reasoning effort the figure reaches 97.9 percent, which OpenAI describes as near saturation of the test.
Who ran the tests, and where production still differs
The tests are OpenAI's own, so the figures sit under the usual rule that a vendor benchmarking its own model is reporting its own result. OpenAI says applying this kind of migration to production code still needs expert review and testing against the system's business rules and dependencies, and names the fixed-point rounding case as an example of a behaviour the model has to preserve rather than fix.
The company's case for why the test matters rests on where COBOL still runs. OpenAI's post states that COBOL supports more than 40 percent of online banking systems, 80 percent of in-person credit card transactions and 95 percent of ATM transactions, and that Fortran is used in weather forecasting and nuclear record safety analysis.
Source
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
