Imagine a computer program that can independently modify its own code without human intervention to become even better and smarter! This futuristic-sounding concept is called the “Gödel machine.”
Jürgen Schmidhuber, a renowned figure in AI research, proposed the idea of self-improving AI more than two decades ago and called it the “Gödel machine.” According to the original idea, the Gödel machine rewrites its own code when it can mathematically prove that a given self-correction leads to improved performance. However, such mathematical proofs are extremely difficult, so the Gödel machine has remained a theoretical concept until now.
In May, however, a research article that could be a significant step toward the realization of the Gödel machine caused a stir on social media. The study, titled “Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents” was authored by researchers at the University of British Columbia in Canada and Sakana AI.
However, the newly presented “Darwin Gödel Machine (DGM)” elegantly circumvents the difficulties of mathematical proof. The DGM uses evolutionary algorithms and empirical (experiential) evaluation methods. This means that multiple self-correcting AI systems compete with each other in various tasks (benchmarks). Continuous competition and evaluation encourage the self-modification and continuous development of AIs.
The research team applied the DGM approach to “coding agents” that automatically generate program code. They allowed these agents to modify their own Python code, for example by adding new tools or suggesting different workflows. The modified agents were then evaluated in coding tests. Interestingly, even the worst-performing agents were archived if their behavior was unique, ensuring evolutionary diversity. This idea helps prevent agents from “getting stuck” in a local optimum and encourages the discovery of innovative solutions.
Thanks to this “evolution,” the performance of the coding agents improved significantly. They achieved a 20-50% increase on the SWE-bench benchmark for solving real-world GitHub problems and a 14.2-30.7% increase on the Polyglot benchmark for measuring multilingual coding.
Of course, there are security concerns associated with such self-improving AI research. Many fear that AI evolution will slip out of human control or that AI will “cheat” during testing. The research team responds to these concerns by enabling AI self-improvement under human supervision in a “sandbox” environment.
The second most mentioned research article in May came from NVIDIA and explores the mystery of AI logical thinking. The study, titled “ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models” examines how the latest AI models, such as OpenAI o1 and DeepSeek-R1, achieve their exceptional logical reasoning abilities.
There is a lively debate among AI researchers about the extent to which reinforcement learning influences the reasoning abilities of foundation models. The crux of the debate is whether reinforcement learning merely unlocks existing reasoning abilities in foundation models or endows them with entirely new reasoning abilities. The latest research tends to support the former view.
However, NVIDIA's research challenges this trend. Using their reinforcement learning method called “ProRL,” which enables long-term, stable learning, they demonstrated that the model was able to “discover” new reasoning strategies and find solutions to tasks that the original foundation model could not answer correctly. This suggests that reinforcement learning can indeed endow base models with new reasoning abilities.
These research breakthroughs show that the development of artificial intelligence is progressing at an astonishing rate. Self-improving AIs, such as the Darwin Gödel machine, could revolutionize software development and many other fields. At the same time, it is crucial that we address the ethical and safety issues involved in a responsible and thoughtful manner, ensuring that the development of AI serves the good of humanity.
