Hi Kohei,
There are two models. One model is responsible for predicting the probability of each unit test to fail, and the other is predicting the overall probability of the patch to fail any unit test. The first has a failure recall of 95% (meaning 95% failures are captured) and a success recall of 85% (meaning reducing the number of unit tests by 85%), but this model is not directly used in deciding whether to run the fast or normal track. The second model is far less accurate, so smart inference is performed in jenkins, which has 91% failure recall and 57% success recall.
The confusion matrix for the first model is posted here.
Best,
Baole Fang
On Fri, Aug 18, 2023 at 9:45 AM Kohei Yoshida <kohei@xxxxxxxxxxxxxxx> wrote:
Hello,
On 09.08.2023 10:57, Baole Fang wrote:
> Feel free to contact me if you have any questions!
Just out of curiosity, what is the overall accuracy of your model? Do
you have a confusion matrix or something similar that shows the
performance of your model?
Thanks,
Kohei
