Running regression tests during the build process often takes considerable time. If even one of the tests fails, the whole procedure must be executed again after the failing test is fixed, requiring extra time and resources. This is why developers prefer to become aware of test failures as soon as possible in the process. One optimization technique is to identify the tests that are most likely to fail and execute them first.

Previously, we implemented a model that predicts the priority of test failures according to recent code changes and the test failures they caused. In this project, we want to assess this prioritization model by applying it to some JetBrains products that are in active development. To achieve this goal, we plan to prepare a dataset and an easy-to-use evaluation framework.