3.4 Metrics Results

2.4 Metrics Design and Methods
4.4 Metrics Discussion

Preliminary results show a high similarity across the metrics. Francis Bull implemented Match Ratio, V Measure, and Mallows Distance metrics in Python and ran them on a preliminary data set. Cluster comparison measures were compared treating one of the gaters as a candidate and the others as the experts. You can download the python script from this link

The following figures show one SIV sample gated down to the same target population by 8 experts and measured the level of agreement with the consensus by three methods.

This study shows that the results of each metric agrees with each other overall; however, individual metrics may correlate better with different qualitative results, such as robustness, specificity, sensitivity, and accuracy.