Author: Washington, Peter; Leblanc, Emilie; Dunlap, Kaitlyn; Penev, Yordan; Varma, Maya; Jung, Jae-Yoon; Chrisman, Brianna; Sun, Min Woo; Stockham, Nathaniel; Paskov, Kelley Marie; Kalantarian, Haik; Voss, Catalin; Haber, Nick; Wall, Dennis P.
Title: Selection of trustworthy crowd workers for telemedical diagnosis of pediatric autism spectrum disorder Cord-id: zpyv1cwy Document date: 2021_1_1
ID: zpyv1cwy
Snippet: Crowd-powered telemedicine has the potential to revolutionize healthcare, especially during times that require remote access to care. However, sharing private health data with strangers from around the world is not compatible with data privacy standards, requiring a stringent filtration process to recruit reliable and trustworthy workers who can go through the proper training and security steps. The key challenge, then, is to identify capable, trustworthy, and reliable workers through high-fidel
Document: Crowd-powered telemedicine has the potential to revolutionize healthcare, especially during times that require remote access to care. However, sharing private health data with strangers from around the world is not compatible with data privacy standards, requiring a stringent filtration process to recruit reliable and trustworthy workers who can go through the proper training and security steps. The key challenge, then, is to identify capable, trustworthy, and reliable workers through high-fidelity evaluation tasks without exposing any sensitive patient data during the evaluation process. We contribute a set of experimentally validated metrics for assessing the trustworthiness and reliability of crowd workers tasked with providing behavioral feature tags to unstructured videos of children with autism and matched neurotypical controls. The workers are blinded to diagnosis and blinded to the goal of using the features to diagnose autism. These behavioral labels are fed as input to a previously validated binary logistic regression classifier for detecting autism cases using categorical feature vectors. While the metrics do not incorporate any ground truth labels of child diagnosis, linear regression using the 3 correlative metrics as input can predict the mean probability of the correct class of each worker with a mean average error of 7.51% for performance on the same set of videos and 10.93% for performance on a distinct balanced video set with different children. These results indicate that crowd workers can be recruited for performance based largely on behavioral metrics on a crowdsourced task, enabling an affordable way to filter crowd workforces into a trustworthy and reliable diagnostic workforce.
Search related documents:
Co phrase search for related documents- accuracy high degree predict and logistic regression: 1
- additional time and logistic regression: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
Co phrase search for related documents, hyperlinks ordered by date