Efficient Group-Based Cohesion Prediction in Images Using Facial Descriptors

Gavrikov, Ilya; Savchenko, Andrey V.

In this paper we study the problem of predicting the cohesiveness and emotion of a group of people in photo. We proposed a fast approach, consisting of face detection by using MTCNN, aggregation of facial features (age, gender and embeddings) extracted by multi-task MobileNet, prediction of group cohesion and classification of emotional background using multi-output convolution neural network. Experimental study on the Group Affect Dataset from EmotiW 2019 challenge demonstrated that our approach allows to achieve an improvement of quality and even to reduce the running time of an algorithm’s work when compared to known solutions. As a result, we obtained mean squared error 0.63 for cohesion prediction, which is 0.21 lower when compared to baseline CapsNet.