Empirical evaluation of the sensitivity of background incidence rate characterization for adverse events across an international observational data network

Ostropolets, A.; Li, X.; Makadia, R.; Rao, G.; Rijnbeek, P.; Duarte-Salles, T.; Sena, A. G.; Shoaibi, A.; Suchard, M. A.; Ryan, P. B.; Prieto-Alhambra, D.; Hripcsak, G.

Background Background incidence rates are routinely used in safety studies to evaluate an association of an exposure and outcome. Systematic research on sensitivity of rates to the choice of the study parameters is lacking. This study investigates the influence of study parameter choices on background rates of adverse events. Methods We used 12 electronic health record and administrative claims data sources to calculate incidence rates of 15 adverse events. We examined the influence of age, race, sex, database, time-at-risk start event and duration, season and year, prior observation and clean window. For binary comparisons we calculated incidence rate ratios and performed random-effect model meta-analysis. Results We observed a wide variation of background rates. The population characteristics had the largest impact with rates varying up to a factor of 1,000 across age groups. Even after adjusting for age and sex, the study showed residual bias due to the other parameters. Anchoring on any type of healthcare encounter yielded higher incidence when compared to anchoring on a random date, especially for short time-at-risk. The rates were highly variable across databases. Conclusion Background rates were highly sensitive to demographic characteristics of population, so comparing background to observed rates requires age, sex and race adjustment, and even with these adjustments, variability remains high among databases. Incidence rates were highly influenced by the choice of the time-at-risk start and duration and less influenced by secular or seasonal trends. All of the above choices are important when generating or interpreting incidence rates in the context of vaccine safety studies.