Mining User Opinions to Support Requirement Engineering: An Empirical Study

DÄ…browski, Jacek; Letier, Emmanuel; Perini, Anna; Susi, Angelo

App reviews provide a rich source of user opinions that can support requirement engineering activities. Analysing them manually to find these opinions, however, is challenging due to their large quantity and noisy nature. To overcome the problem, automated approaches have been proposed for so-called opinion mining. These approaches facilitate the analysis by extracting features discussed in app reviews and identifying their associated sentiments. The effectiveness of these approaches has been evaluated using different methods and datasets. Unfortunately, replicating these studies to confirm their results and to provide benchmarks of different approaches is a challenging problem. We address the problem by extending previous evaluations and performing a comparison of these approaches. In this paper, we present an empirical study in which, we evaluated feature extraction and sentiment analysis approaches on the same dataset. The results show these approaches achieve lower effectiveness than reported originally, and raise an important question about their practical use.