Machine Learning Methods in Software Engineering

← Back to laboratory

Evaluation of performance metrics for code generation models

This project aims to understand how to correctly evaluate code generation models, compare existing performance metrics, and introduce a new metric that better correlates with human judgment.

Participants

Mikhail Evtikheev
Egor Bogomolov

Publications

August 2022
Mikhail Evtikhiev, Egor Bogomolov, Yaroslav Sokolov, Timofey Bryksin

Out of the BLEU: How Should We Assess Quality of the Code Generation Models?

arXiv pre-print