GraphTyper: neural types inference from code represented as graph

G. A. Arutyunov, S. M. Avdoshin

HSE University

Abstract: Although software development is mostly a creative process, there are many scrutiny tasks. As in other industries, there is a trend for automation of routine work. In many cases, machine learning and neural networks have become a useful assistant in that matter. Programming is not an exception: GitHub has stated that Copilot is already used to write up to 30% of code in the company. Copilot is based on Codex, a Transformer model trained on code as a sequence. However, a sequence is not a perfect representation for programming languages. In this work, we claim and demonstrate that by combining the advantages of Transformers and graph representations of code, it is possible to achieve excellent results even with comparably small models.

Keywords: neural networks; transformers; graphs; abstract syntax tree.

DOI: 10.15514/ISPRAS-2024-36(4)-6