A Chronological and Geographical Analysis of Personal Reports of COVID-19 on Twitter

Ari Klein; Arjun Magge; Karen O'Connor; Haitao Cai; Davy Weissenbacher; Graciela Gonzalez-Hernandez

Then, using handcrafted regular expressions, we identified 160,767 of the tweets that contain information potentially indicating that the user or a member of the user's household had been exposed to COVID-19. We removed 30,564 of the matching tweets that were automatically determined to contain "reported speech" (e.g., quotations, news headlines) using a filter we developed in recent work. We manually annotated a random sample of 10,000 of the 130,203 pre-filtered tweets. Annotation guidelines were developed to help two annotators distinguish three classes of tweets. Inter-annotator agreement was κ = 0.73 (Cohen's kappa), considered "substantial agreement." 8 Among the 10,000 tweets, 6.9% (685) were annotated as "probable," 7.8% (780) as "possible," and 85.3% (8535) as "other."