Silent geographical spread of the H7N9 virus by online knowledge analysis of the live bird trade with a distributed focused crawler

Chen, Chen; Lu, Shan; Du, Pengcheng; Wang, Haiyin; Yu, Weiwen; Song, Huawen; Xu, Jianguo

Information-based connection by a distributed focused crawler After CC data were collected, we first analyzed the information online using the distributed focused crawler technology from 244 public news web sites as well as 56 forums and microblogs, including common sites such as ''Sina,'' ''Sohu,'' and ''Yahoo.'' Keywords were used to select online knowledge and entered using the following format: (city name) and (live poultry or poultry or dove or quail or birds or migratory bird or chicken or duck or goose) and (come from or exchange or source or carrier or travel or supply or market or slaughterhouse or production area). The pages that fit our requirements were included in the hash list. A bloom filter was used to adjust and remove repeated information. These pages were clustered with a support vector machine model and categorized into different groups to analyze the orientation of web pages. These confirmed web pages were then downloaded and coordinated by Apache ZooKeeper (The Apache Foundation, Delaware, US).