Mining Social Media and Web Searches For Disease Detection

Yang, Y. Tony; Horneffer, Michael; DiLisio, Nicole

When used to undertake infectious disease surveillance, social media-based models can track infectious disease trends in real time in order to predict, observe, and minimise the harm caused by outbreak events. 5 A 2006 study by Eysenbach tracked influenza (flu)-related searches on the web for syndromic surveillance to determine whether an automated analysis of trends in Internet searches could be useful to predict outbreaks such as influenza epidemics. 6 Eysenbach developed a model for predicting a flu outbreak on the basis of changes in Internet search activity for flu-related information. 6 The model was evaluated against a traditional surveillance method which uses sentinel physicians who manually report encounters with sick patients demonstrating flu-like illness (ILI) to a public health agency. 6 To obtain statistics on the prevalence of searches, Eysenbach created an advertising campaign on Google Adsense that appeared for Canadian searchers only who entered flu or flu symptoms into Google. The ad read, Do you have the flu? Fever, Chest discomfort, Weakness, Aches, Headache, Cough? and provided a link to a generic patient education website. Eysenbach aggregated daily statistics on impressions and clicks provided by Google to match the time periods of the weekly national FluWatch reports. The number of advertising clicks correlated better with flu events than flu-like illness (ILI) reports from sentinel physicians. Internet clicks also were a timelier marker than these reports in that they performed better in predicting the flu events of the following week. Correlation coefficients in the reports from sentinel physicians were better for the status of the current week than for predicting the following week. A study conducted by Polgreen et al. 7 also examined the occurrence of ILI and flu as related to information collated by an Internet search engine. They used Yahoo to predict an increase in culture positive for flu 1-3 weeks in advance. 7 Their model was predicated on the theory that the pattern of how and when people search may provide clues or early indications about future health-related concerns and expectations. 7 For instance, when someone believes himself to be sick, he will first search on the internet for disease-related symptoms related to his symptoms before checking in with a public health professional or physician. In developing their model, Polgreen et al. determined whether there was a relationship between the webbased search terms related to a disease and the actual cases of dis-ease. 7 In order to relate the search data to a measure of the actual occurrence of flu, the investigators used reports generated by clinical laboratories (members of the National Respiratory and Enteric Virus Surveillance System) that compiled the total number of respiratory specimens that were positive for the disease. 7 The web-based search data were compiled at national level and included queries that contained information pertinent to seasonal flu events. Specific census regions were studied that identified the geographical location of origin of the Internet protocol addresses conducting the searches. 7 The two sets of data were then correlated and compared. A positive relationship was found between the fraction of flu-related queries and rates of cultures positive for flu two weeks later. 7 Despite discovering a relationship between search queries and new flu cases, the study was limited due to the relatively short period of time in