Factors Affecting the Cost to Accuracy Balance for Real-Time Video-Based Action Recognition

Govender, Divina; Tapamo, Jules-Raymond

For successful real-time action recognition in videos, the compromise between computational cost and accuracy must be carefully considered. To explore this balance, we focus on the popular Bag-of-Words (BoW) framework. Although computationally efficient, the BoW has weak classification power. Thus, many variants have been developed. These variants aim to increase classification power whilst maintaining computational efficiency; achieving the ideal cost-to-accuracy balance. Four factors affecting the computational cost vs accuracy balance were identified: ‘Sampling’ strategy, ‘Optical Flow’ algorithm, ‘Saliency’ of extracted features and overall algorithm ‘Flexibility’. The practical effects of these factors were experimentally evaluated using the Dense Trajectories feature framework and the KTH and HMDB51 (a reduced version) datasets. The ‘Saliency’ of extracted information is found to be the most vital factor - spending computational resources to process large amounts of non-salient information can decrease accuracy.