Image-based Social Sensing: Combining AI and the Crowd to Mine Policy-Adherence Indicators from Twitter

Negri, Virginia; Scuratti, Dario; Agresti, Stefano; Rooein, Donya; Shankar, Amudha Ravi; Marquez, Jose Luis Fernandez; Carman, Mark James; Pernici, Barbara

Social Media provides a trove of information that, if aggregated and analysed appropriately can provide important statistical indicators to policy makers. In some situations these indicators are not available through other mechanisms. For example, given the ongoing COVID-19 outbreak, it is essential for governments to have access to reliable data on policy-adherence with regards to mask wearing, social distancing, and other hard-to-measure quantities. In this paper we investigate whether it is possible to obtain such data by aggregating information from images posted to social media. Combining recent advances in image recognition technology with geocoding and crowdsourcing techniques to build a pipeline for image-based social sensing. Our aim is to discover in which countries, and to what extent, people are following COVID-19 related policy directives. We compared the results with the indicators produced within the Covid-19 behavior tracker initiative by ICL and YouGov (CovidDataHub). Preliminary results shows that social media images can produce reliable indicators for policy makers.