4.2.3 SIV Discussion

2.2.3.1 SIV Use Case
2.2.3.2 SIV gating SOP
3.2.3 SIV Results

With this use case, we have highlighted the importance of having a consensus that is acceptable to the collaborating scientist. If the variability within the consensus is high, then any of the comparison metrics that we use to evaluate the algorithms against the consensus do not have to reach a high bar for success because there is too much confusion on which cells are in the target population and the comparison metrics only weigh events that have high correlation within the consensus. If the consensus is less variable and within the acceptable human variation already apparent in the clinical labs, then algorithms can be tuned to fall within this acceptable variability. Qualitatively, even with large amounts of variability between the target populations chosen by the Tree Star Scientists, there is still agreement on the biological outcome or diagnostic decision of the experiment as measured by the cytokine responses.

Despite have a large variability between individual manual analysis, the end evaluation of the Tree Star Scientists analysis was agreement that CD4 T-cell IL-2 responses were up significantly at time-point 2 for all primates, with the exception of one monkey for one scientist. This one sample was not correctly gated and is a good example of where our algorithms could flag a sample for manual analysis.

This is really the experiment that lead us to realize the need for a centralized database at the with utilities to calculate the comparison metrics. The first iteration of analysis was attempted using simple files systems and spreadsheets for building the consensus and was very time consuming.

During the next phase of the project, we will be working more closely with Dr. Schmitz and Michelle Lifton to clarify their gating decisions and get manual analysis from multiple people in their lab to have an acceptable consensus group as we move forward with more iterations of the automated classifiers.