Efficient Estimation and Closed-Form Uncertainty Quantification for Net Benefit of Algorithms that Predict Individualized Treatment Benefit

Treatment benefit predictors (TBPs) quantify the expected treatment benefit given individual characteristics. The net benefit function evaluates the expected gain in clinical utility from using a TBP to guide treatment decisions, relative to the default decisions of treating no one and treating everyone. The existing estimator for the net benefit of a given TBP implicitly assumes a 1:1 randomization design in the randomized controlled trial data. When this assumption is violated, the estimator can exhibit biased and unstable finite-sample behaviour. We identify the source of this implicit design restriction and propose a corrected congruent-based modification of the net benefit estimator that remains valid under arbitrary randomization schemes. We further introduce an alternative net benefit formulation based on the average treatment effect among individuals recommended for treatment by a TBP (ATT-based). The ATT-based estimator is asymptotically equivalent to the corrected congruent-based estimator and makes more efficient use of the full sample by estimating the proportion recommended for treatment using all individuals. Three uncertainty quantification procedures are developed for the ATT-based estimator: a large-sample variance approximation, an aggregated nonparametric bootstrap, and a Bayesian analysis. A Monte Carlo simulation study evaluates point estimation of net benefit curves using mean squared error, while uncertainty quantification is assessed through confidence interval length and coverage probabilities for 95% intervals constructed using asymptotic, bootstrap, and Bayesian methods. Simulation studies show that the corrected congruent-based estimator removes the bias under unequal randomization, while the ATT-based estimator demonstrated improved finite-sample efficiency. An empirical application to the GUSTO randomized controlled trial illustrates the ATT-based estimator and accompanying uncertainty intervals in a real-world setting. Overall, the proposed methodology enables valid estimation and uncertainty quantification of net benefit for treatment benefit predictors under arbitrary randomization schemes.

To join this seminar virtually, please request Zoom connection details from hr.ops@stat.ubc.ca.