Prediction Viewer

This is a demo of the result viewer from the Patient-Level Prediction R package

We have applied the PatientLevelPrediction package to observational healthcare data to address the following patient-level prediction question:

Amongst patients who are newly diagnosed with Atrial Fibrillation, which patients will go on to have Ischemic Stroke within 1 year?

We have defined 'patients who are newly diagnosed with Atrial Fibrillation' as the first condition record of cardiac arrhythmia, which is followed by another cardiac arrhythmia condition record, at least two drug records for a drug used to treat arrhythmias, or a procedure to treat arrhythmias.

We have defined 'Ischemic stroke events' as ischemic stroke condition records during an inpatient or ER visit; successive records with > 180 day gap are considered independent episodes.

The results viewer shows the results of internal validation and external validation on one large database.

Table 1. Performance of the model on the train and test set and information about the study population.

Figure 1. The variable scatter plot shows the mean covariate value for the people with the outcome against the mean covariate value for the people without the outcome. The meaning of the size and color of the dots depends on the settings on the left of the figure.

Table 2. This table shows for each covariate the mean value in persons with and without the outcome and their mean difference.

Train

Test

Figure 2. The Receiver Operating Characteristics (ROC) curve shows the ability of the model to discriminate between people with and without the outcome during the time at risk. It is a plot of sensitivity vs 1-specificity at every probability threshold. The higher the area under the ROC plot the higher the discriminative performance of the model. The diagonal refers to a model assigning a class at random (area under de ROC = 0.5).

Train

Test

Figure 3. The calibration plot shows how close the predicted risk is to the observed risk. The diagonal dashed line thus indicates a perfectly calibrated model. The ten (or fewer) dots represent the mean predicted values for each quantile plotted against the observed fraction of people in that quantile who had the outcome (observed fraction). The straight black line is the linear regression using these 10 plotted quantile mean predicted vs observed fraction points.

Train

Test

Figure 4. This demogrophics plot shows for females and males the expected and observed risk in different age groups and the confidence areas.

Train

Test

Figure 5. The preference distribution plots are the preference score distributions corresponding to i) people in the test set with the outcome (red) and ii) people in the test set without the outcome (blue).

Train

Test

Figure 6. The prediction distribution boxplots are box plots for the predicted risks of the people in the test set with the outcome (class 1: blue) and without the outcome (class 0: red).

Settings
Attrition

Table 3. Settings for model development.

Table 4. Covariate settings.

Table 5. Population definition settings.