Fatigue detection on face image using Face Net algorithm and k-nearest neighbor classifier

Adhinata, Faisal Dharma; Rakhmadani, Diovianto Putra; Wijayanto, Danur; en,

The COVID-19 pandemic has made people spend more time on online meetings more than ever. The prolonged time looking at the monitor may cause fatigue, which can subsequently impact the mental and physical health. A fatigue detection system is needed to monitor the Internet user’s well-being. Previous research related to the fatigue detection system used a fuzzy system, but the accuracy was below 85%. In this research, machine learning is used to improve accuracy. Objective: This research examines the combination of the Face Net algorithm with either k-nearest neighbor (K-NN) or multiclass support vector machine (SVM) to improve the accuracy. Methods: In this study, we used the UTA-RLDD dataset. The features used for fatigue detection come from the face, so the dataset is segmented using the Haar Cascades method, which is then resized. The feature extraction process uses FaceNet's pre-trained algorithm. The extracted features are classified into three classes—focused, unfocused, and fatigue—using the K-NN or multiclass SVM method. Results: The combination between the FaceNet algorithm and K-NN, with a value of resulted in a better accuracy than the FaceNet algorithm with multiclass SVM with the polynomial kernel (at 94.68% and 89.87% respectively). The processing speed of both combinations of methods has allowed for real-time data processing. Conclusion: This research provides an overview of methods for early fatigue detection while working at the computer so that we can limit staring at the computer screen too long and switch places to maintain the health of our eyes.