BETTER: Bayesian Evaluation of Time-To-Event and Reliability (for vaccine surveillance)  

BETTER: Bayesian Evaluation of Time-To-Event and Reliability (for vaccine surveillance)

NOTE: All results are preliminary and subject to change

About the BETTER study

The study aims at developing a Bayesian alternative procedure to improve the performance of vaccine safety surveillance. Empirical evaluations based on large-scale observational healthcare databases are conducted to compare the proposed procedure against the state-of-art approach, MaxSPRT, with testing-oriented and estimation-oriented metrics.

These results are being shared as part of OHDSI's open science community efforts. Until final publication, all results are to be considered preliminary and subject to change, and may only be used under the terms of use of the respective data partner contributors.

Manuscript preprint is available at [LINK PENDING]

To view results:

  • See Testing metrics for results on Type 1 error rates, statistical power, and time-to-signal.
  • See Estimation metrics for comparisons on estimation accuracy and quality, with mean squared errors (MSEs) and coverage rates of 95% CIs.
  • See Database information and Vaccine information for details on databases and vaccine exposures.


  • The study protocol is available here
  • All analytic code is availble on GitHub

This study is performed under the collaborative contract between OHDSI and FDA CBER BEST. Authors of this work received funding from the US National Institutes of Health and the US Food & Drug Administration and contracts from the US Department of Veterans Affairs and Janssen Research and Development.

Plot: Empirical Type 1 error rate over analysis periods. Type 1 error rates are measured by fraction of H0 rejected over all negative control outcomes.
Plot: Statistical power over analysis periods. Power is measured by fraction of positive control outcomes with H0 rejected, stratified by effect sizes.
Plot: Timeliness. Measured by number of analyses (in months) needed to reach a desired sensitivity level for detecting true positive signals.
Table: Estimation-oriented metrics. Mean-squared errors (MSEs) and coverage rate of 95% CIs.
Table: Information about each database.
Table: Information about each vaccine exposure.