The validation of a mobile sensor-based neurobehavioral assessment with machine learning analytics

Sloane, K. L.; Mefford, J. A.; Glenn, S.; Zhao, Z.; Xu, M.; Zhou, G.; Mace, R.; Wright, A. E.; Hillis, A. E.

Background: A mobile platform for the self-administration of sensor-based cognitive and behavioral assessments was developed. In addition to measurements typical of legacy neuropsychological tests, Miro also quantifies functions that are currently left to subjective clinical impressions such as motor function, language, and speech. Objective: Studies were conducted to measure Miro's concurrent validity, test-retest reliability, and amnestic MCI classification performance. Method: Spearman correlations were calculated to estimate the concurrent validity of Miro and legacy variables using data from 207 study participants. Twenty-six healthy controls were assessed at three time points to evaluate the test-retest reliability of Miro test scores. Reliability was quantified with the scores' intraclass correlations. Learning effects were measured as trends over three assessments. A machine learning algorithm combined Miro scores into a risk score to distinguish 65 healthy controls, 21 amnestic MCI (aMCI) participants, and 17 non-amnestic MCI (naMCI) participants. Results: Significant correlations of Miro variables with legacy neuropsychological test variables were observed. Longitudinal studies show agreement of subsequent measurements and minimal learning effects. The Risk Score distinguished aMCI from healthy controls with an Area Under the Receiver Operator Curve (AUROC) of 0.97; the naMCI participants and controls were separated with an AUROC of 0.80, and the combined (aMCI + naMCI) group was separated from healthy controls with an AUROC of 0.89. Conclusion: Miro includes valid and reliable versions of scores from legacy neuropsychological test scores and a machine-learning derived risk score that effectively distinguishes healthy controls and individuals with MCI.