Google’s machine learning specialists from the DeepMind team have developed software that generates program code from natural-language instructions. The special thing about AlphaCode: The AI based on a Transformer model is trained to solve programming tasks, such as those that are often asked in competitions or recruitment tests for developers.
For this purpose, the model was first trained with selected program code from GitHub. The DeepMind team then fed AlphaCode with tasks from programming competitions for fine tuning. The AI then generated a large number of C++ and Python programs for each test task, from which it filtered out ten candidates using cluster methods. Their code quality was then checked.
test in competition
The test was carried out with ten current programming tasks from code forces, a website where developers can enter programming competitions. According to Google, AlphaCode achieved an average ranking of 54.3 percent: almost half of the human participants in the competitions provided worse solutions than the AI.
Google is not the only company researching AIs that generate code: GitHub Copilot, presented in summer 2021, is an AI programming aid for developers based on OpenAI Codex. Codex is derived from OpenAI’s GPT-3 (Generative Pre-trained Transformer 3), a powerful AI for generating speech. GPT-3 is already amazing when it comes to generating valid program code from natural language. Programmers can now find several AI-supported extensions for IDEs that do the work for them when programming.
The DeepMind team describes their research, how the AI works and the test results in the paper Competition-Level Code Generation with AlphaCode. It also includes examples of the generated code. the test tasks are available on GitHub in case anyone wants to compete against AlphaCode.
(odi)