Loading

GvHD Use Case Results

2.2.2 GvHD Description
4.2.2 GvHD Discussion

To evaluate algorithms for selecting target cell populations for FlowDx, we need a quantitative way to measure how close we are to human gating techniques. Additionally we want to evaluate how close one expert is to a group of experts. Therefore, the Match Ratio was born. See pdf for more details about the Match Ratio. The first step is to have a group of experts gate a dataset and create a consensus gate.

GvHD dataset provided by Ryan Brinkman of Terry Fox Laboratory/British Columbia Cancer Agency. See link for the gating instructions.

Goal was to adjust pre-made gates in the workspace to select for the CD3/CD4/CD8b+ for all 12 timepoints in the GvHD run 1.

Looking at sample GvHD1.055, there is a wide variability in cell count of the target population ranging from 0 cells to 627 cells.

gvhd experts dot plot

Figure 1. Overlay of CD3+/CD4+/CD8b+ T-cells gated by several analysts. There is a large
difference between the number of cells selected by contributing lab, and the external analysts.

 

There is a large variability (134 ± 171) between the number of cells selected by Brinkman's lab, which contributed the data, and the rest of the gaters. The discrepancy highlights the need to have a more descriptive and specific procedure for gating, but such a procedure was not provided at the time of this exercise.

The match ratio was calculated on the individual cells in the CD3/CD4/CD8b+ population for each expert and compared to the consensus. A Match Ratio of 1.0 matches the consensus perfectly.
Everyone was very close in their Match Ratio, even when the number of cells in each expert's final population had a high variability. This is due to the experts agreeing that the majority of the cells were not in the final population.

Table 1. Statistics on CD3/CD4/CD8b+ population in Sample GvHD1.055

GvHD1.055
 Match Ratio Count Freq of total MFI CD4 MFI CD8b
Aaron_1 0.99515 131 0.0095 41.27 57.14
Aaron_2 0.99989 0 0.0000 - - - -
Aaron_3 0.99575 307 0.0223 18.24 48.59
Brinkman* 0.97865 627 0.0434 16.5 26.91
Claudio 0.99986 127 0.0092 13.05 55.27
Helene 0.99898 193 0.0140 14.75 63.52
John 0.99991 5 0.0004 11.17 47.73
Junichi 0.99984 143 0.0104 12.62 59.48
Maciej 0.99990 5 0.0004 12.68 41.77
M 0.99993 1 0.0001 14.99 70.41
M2 0.99993 1 0.0001 14.99 70.41
Nick 0.99974 122 0.0089 11.93 68.61
Qianjun 0.99723 296 0.0215 16.42 45.63
Sach 0.99982 51 0.0037 22.08 69.98
Sach2 0.99989 4 0.0003 55.95 139.13
Starting.wsp 0.99723 296 0.0215 16.42 45.63
mean 0.99761 142.5 0.0104 19.54 60.68
st dev 0.00531 165.5 0.0120 12.43 25.08
CV 0.00532 1.16 1.16 0.64 0.41

We learned that:
1) We need a more-specific SOP for analysis. (Ryan Brinkman's lab and M used the same gate definitions for all samples, but most others adjusted the gates for each sample.) Which is the better scientific method -- one gate definition for all samples, or adjust the gates for each sample?

2) If there is high variability within the consensus, it becomes easier for the match ratio to be very close to the maximum of 1.0.

3) We need feedback as to whether the inter-gater variability of the manual gating is within acceptable limits from the contributors.

Additional results show similar situations for the other samples in this run.