Evaluation of machine-learning models to measure individualized treatment effects from randomized clinical trial data with time-to-event outcomes
Objective: In randomized clinical trials, prediction models can be used to explore relationships between patient variables and treatment effect magnitude. Our aim was to evaluate flexible machine learning models that incorporate interactions and nonlinear effects from high-dimensional data to estimate individualized treatment recommendations in trials with time-to-event outcomes. Methods: We compared neural networks (CoxCC and CoxTime) and random survival forests, including Interaction Forests (IF) and Causal Survival Forest (CSF), against a Cox proportional hazards model with an adaptive LASSO (ALASSO) penalty as a benchmark. For individualized treatment recommendations, we adapted metrics originally designed for binary outcomes to accommodate time-to-event data with censoring, using both survival probability and restricted mean survival time estimates of treatment benefit. Metrics included C-for-Benefit, E50-for-Benefit, Integrated Calibration Index for benefit, and root-mean-squared error for treatment benefit. We also evaluated decision-making utility using the Area Under the Targeting Operator Characteristic and Population Benefit (PB). An extensive simulation study used two data generation processes incorporating nonlinearity and interactions. The models were applied to three cancer clinical trial datasets. Results: In the first data generation process, neural networks outperformed ALASSO in calibration, while IF showed superior C-for-Benefit performance. In the second, machine learning methods outperformed the benchmark linear ALASSO method across discrimination, calibration, and RMSE. In the cancer trial datasets, machine learning methods often performed better than ALASSO: neural networks consistently achieved the best calibration, IF achieved the highest C-for-Benefit, and CSF showed strong calibration and the highest PB in the trastuzumab dataset.