OpenAI reports big gains in converting COBOL code, but the hard part remains
Behaviour-preservation scores are high on simple tasks, yet splitting large legacy systems into replaceable pieces is still unsolved.
OpenAI says its newest models have made substantial gains in migrating aging COBOL and other legacy code into languages such as C++, Java, Kotlin, Python and Rust while keeping the original behavior intact. In its own tests, GPT-6.1 Sol preserved behavior in 87 percent of cases at low reasoning effort and 98 percent at higher effort, with GPT-6 Astra close behind at around 86 percent, while GPT-6 Sol and GPT-5.6 Sol trailed sharply at 21 and 5 percent. The results are still limited: the benchmark covered a relatively simple, well-defined task on systems similar to what the models saw in training. Researchers quoted by The Verge say the hard part is splitting a large legacy system into pieces that can be replaced one at a time, so this does not yet make critical systems safe to replace. Anthropic has also pitched its Claude Code agent for COBOL migration without publishing comparable benchmarks.
Why it matters
For organisations running decades-old COBOL systems, the results suggest AI models can now translate much of the code while keeping its behaviour, which may lower one barrier to modernisation. The evidence is still narrow, though: the benchmark covered a relatively simple, well-defined task on systems similar to training data, and researchers say the harder challenge of breaking a complex system into pieces that can be replaced one at a time is not addressed. Migrating code is therefore not the same as making critical systems safe to replace, and claims from rival vendors remain unverified by published benchmarks.
Matters or hype?
Does this story matter, or is it hype? Vote before you see what everyone else said.