arXiv subjects

David S. Robertson

Publications and source records attributed to David S. Robertson.

At least 19 recordsLinked to original sources

arXiv2026

Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials

In platform trials, multiple intervention arms can enter or leave the study at any time, which brings the challenge of how to control the type I error rate. Online testing procedures, where a stream of hypotheses are tested over time and the total number of hypotheses is unknown, might be a viable choice for error rate control in platform trials. Interim analyses in a multi-stage design offer the opportunity of early stopping for success or futility in clinical trials by testing the same hypothesis at different time points. Some methods targeting false-discovery rate (FDR) or family-wise error rate (FWER) control for multi-stage platform trials have been proposed. However, a method which controls the FWER and allows new arms to be added at any time without prior information is yet to be established. The ADDIS-Spending (ADaptive DIScarding Alpha-spending) procedure controls the FWER in an online setting. We propose the group sequential ADDIS-Spending algorithm to enable interim analyses for each hypothesis in platform trials. The developed method can update the error level of an individual hypothesis at all (interim) analyses whenever new decisions on other hypotheses are made. We compare alternative approaches with our method in a simulation study. The proposed method yields competitive results in the conducted simulations.

stat.ME↗

arXiv2026

Adaptive Sequential Surveillance of Infectious Diseases Under Limited Sampling Capacity

This paper develops a closed-loop adaptive sequential surveillance framework for infectious disease monitoring with limited surveillance resources. In this framework, we use a Bayesian spatio-temporal model to predict disease prevalence at each location, and propose several adaptive site-allocation strategies based on the posterior probabilities that the predictions exceed a predefined threshold. These strategies exploit high-risk locations while also exploring, by random or recency-based sampling or by incorporating the frontiers of emerging outbreaks. To address the unequal inclusion probabilities induced by adaptive sampling, we integrate inverse-probability weighting (IPW) into each model update. A simulation study showed that adaptive surveillance strategies improved outbreak detection relative to uniform sampling and fixed sentinel sampling. Frontier-based strategies improved detection and timeliness relative to risk-proportional allocation. When IPW was used to update the model, some proposed strategies yielded lower inferential bias compared with the standard unweighted update, particularly for strategies that allowed more flexible exploration. Credible-interval coverage varied by strategy. The results identify trade-offs between exploitation of known high-risk locations and exploration of potential outbreak frontiers.

stat.ME↗

arXiv2026

Ready for the Clinic? A Survey of Open-Source Software for Response-Adaptive Randomization in Clinical Trials

Response-adaptive randomization (RAR) modifies treatment allocation probabilities during a clinical trial as response/outcome data accumulate, with the aim of improving patient benefit, statistical efficiency, or both. Despite substantial methodological development, adoption of RAR in clinical practice has remained limited, and the software available to support its design and implementation has not previously been reviewed. We identified 16 publicly available, open-source software packages implementing RAR methods, spanning urn-based, target-allocation, Bayesian, Markov decision process (MDP)-based, and dose-finding approaches, and evaluated them with respect to their methodological and practical characteristics. We found that while several software packages exist, most are method-specific, and only a small number provide broader, general-purpose adaptive-trial design and analysis capabilities. Explicit support for practically relevant features, including delayed and missing outcome data, temporal trends in response rates, flexible operating-characteristic evaluation, and platform or multi-arm multi-stage trial designs, is rare or absent across the identified software. These findings suggest that while a diverse set of tools exists for exploring RAR designs, gaps remain between the methodological literature and the software available to implement it in practice. Continued development of flexible, practically oriented, and validated software is important for the wider adoption of RAR in clinical research, and we highlight interesting areas of further work.

stat.CO↗

arXiv2026

Days Alive at Home vs Out of Hospital: Why the Difference Matters for Trial Endpoints

Days Alive and Out of Hospital (DAOH) and Days Alive at Home (DAH) are increasingly used as patient-centred primary outcomes in perioperative trials, and a recent editorial has advocated DAOH for its simplicity and reliance on routinely collected data. We argue that the choice between these endpoints affects power, treatment effect estimation and missing data, and should be made at the design stage with these consequences in mind. By treating days in nursing homes or rehabilitation facilities as equivalent to days at home, DAOH assumes that being out of hospital is a valid surrogate for good recovery, an assumption that patient perspectives, including those from the NOTACS trial, call into question. Choosing DAH over DAOH therefore involves a trade-off between measuring what matters to patients more accurately and the added burden of tracking discharge destination, including a greater risk of missing data. If an intervention affects only length of stay, readmission or mortality, the two endpoints yield the same expected treatment effect, and the extra burden of tracking discharge destination brings no benefit. If it instead enables more patients to return directly home rather than to a care facility, DAOH may conceal this benefit and lose statistical power. We propose a baseline-adjusted DAH, which counts only days spent in a setting representing an escalation of care relative to the patient's baseline, as a better proxy for recovery that also accommodates hospital-at-home and virtual ward services.

stat.AP↗

arXiv2026

Introducing precision-weighted bias as a performance measure to inform the inclusion of adaptive designs in meta-analysis

We propose a novel, intuitive measure of statistical performance: precision-weighted bias. Precision-weighted bias is defined as the unconditional bias of an estimator weighted by the degree of information (precision) it contains. Current guidelines, such as GRADE and CONSORT, often view the potential for increased bias in adaptive designs as a deterrent for the inclusion of such designs in systematic reviews. However, we demonstrate that the bias in a common-effect meta-analysis is approximately equal to the precision-weighted average of the precision-weighted biases of its constituent studies, rather than of their unweighted unconditional biases. Through simulation studies, we show that while adaptive designs may exhibit unweighted bias, they frequently have zero precision-weighted bias. Consequently, including these designs often results in a negligible change to the overall meta-analysis bias. These results suggest that precision-weighted bias is a superior indicator for determining whether to include an adaptive design in a meta-analysis. We recommend that precision-weighted bias be used as a standard complement to unweighted unconditional and conditional bias in simulation studies to support more inclusive and accurate evidence synthesis.

stat.ME↗

arXiv2026

Operationalizing Allocation Probability Tests: Practical Guidance on Optimized Implementation for Power and Robustness

Recently, a new testing approach for response-adaptive clinical trials was proposed based on the allocation probabilities (AP) rather than the outcome data. While original work on the AP test focused on binary and normal endpoints and demonstrated that significant efficiency gains are possible, many critical questions remain open regarding its practical implementation and upper limits. In this work, rather than simply proposing novel statistics, we seek to understand the maximum gain that can be obtained with the AP test by optimizing how these probabilities are used to define the test statistic. We expand the method's practical utility by applying it to survival endpoints (exponential distributions) and introducing a rigorous strategy for selecting the null hypothesis to properly calibrate type I error. Our simulation studies reveal that by optimizing the functional form of the AP test, investigators can achieve a substantial increase in power, approaching the theoretical maximum, without sacrificing the patient outcome goals of the design. Furthermore, we explicitly compare the method to a standard Bayesian decision rule, finding that the optimized AP test significantly outperforms traditional frequentist tests while maintaining strict error control. This work provides a missing practical framework for implementing robust and optimized AP tests in complex response-adaptive settings.

stat.ME↗

arXiv2025

Thompson, Ulam, or Gauss? Multi-criteria recommendations for posterior probability computation methods in Bayesian response-adaptive trials

Bayesian adaptive designs enable flexible clinical trials by adapting features based on accumulating data. Among these, Bayesian Response-Adaptive Randomization (BRAR) skews patient allocation towards more promising treatments based on interim data. Implementing BRAR requires the relatively quick evaluation of posterior probabilities. However, the limitations of existing closed-form solutions mean trials often rely on computationally intensive approximations which can impact accuracy and the scope of scenarios explored. While faster Gaussian approximations exist, their reliability is not guaranteed. Critically, the approximation method used is often poorly reported, and the literature lacks practical guidance for selecting and comparing these methods, particularly regarding the trade-offs between computational speed, inferential accuracy, and their implications for patient benefit. In this paper, we focus on BRAR trials with binary endpoints, developing a novel algorithm that efficiently and exactly computes these posterior probabilities, enabling a robust assessment of existing approximation methods in use. Leveraging these exact computations, we establish a comprehensive benchmark for evaluating approximation methods based on their computational speed, patient benefit, and inferential accuracy. Our comprehensive analysis, conducted through a range of simulations in the two-armed case and a re-analysis of the three-armed Established Status Epilepticus Treatment Trial, reveals that the exact calculation algorithm is often the fastest, even for up to 12 treatment arms. Furthermore, we demonstrate that commonly used approximation methods can lead to significant power loss and Type I error rate inflation. We conclude by providing practical guidance to aid practitioners in selecting the most appropriate computation method for various clinical trial settings.

stat.ME↗

arXiv2025

Informed Burn-In Decisions in RAR: Harmonizing Adaptivity and Inferential Precision Based on Study Setting

Response-Adaptive Randomization (RAR) is recognized for its potential to deliver improvements in patient benefit. However, the utility of RAR is contingent on regularization methods to mitigate early instability and preserve statistical integrity. A standard regularization approach is the ''burn-in'' period, an initial phase of equal randomization before treatment allocation adapts based on accrued data. The length of this burn-in is a critical design parameter, yet its selection remains unsystematic and improvised, as no established guideline exists. A poorly chosen length poses significant risks: one that is too short leads to high estimation bias and type-I error rate inflation, while one that is too long impedes the intended patient and power benefits of using adaptation. The challenge of selecting the burn-in generalizes to a fundamental question: what is the statistically appropriate timing for the first adaptation? We introduce the first systematic framework for determining burn-in length. This framework synthesizes core factors - total sample size, problem difficulty, and two novel metrics (reactivity and expected final allocation error) - into a single, principled formula. Simulation studies, grounded in real-world designs, demonstrate that lengths derived from our formula successfully stabilize the trial. The formula identifies a ''sweet spot'' that mitigates type-I error rate inflation and mean-squared error, preserving the advantages of higher power and patient benefit. This framework moves researchers from conjecture toward a systematic, reliable approach.

stat.ME↗

arXiv2025

Bayesian Optimal Phase II design with optimised stopping boundaries and response-adaptive randomisation

The Bayesian Optimal Phase II (BOP2) framework is a flexible trial design that can naturally facilitate complex adaptations due to its Bayesian setting. BOP2 uses equal randomisation and equally placed interim analyses in its design, but it is unclear whether these give the best operating characteristics. By incorporating Bayesian Response-Adaptive Randomisation (BRAR) and optimal interim analysis placement, we show that allocation to the best treatment and expected sample size can be improved with minimal impact on power. We discuss recommendations on implementing these adaptations, using simulation-based evidence, to give practical advice to practitioners. Reproducible code for the simulations is freely provided.

stat.AP↗

arXiv2025

Optimal weighted tests for replication studies and the two-trials rule

Replication studies for scientific research are an important part of ensuring the reliability and integrity of experimental findings. In the context of clinical trials, the concept of replication has been formalised by the 'two-trials' rule, where two pivotal studies are required to show positive results before a drug can be approved. In experiments testing multiple hypotheses simultaneously, control of the overall familywise error rate (FWER) is additionally required in many contexts. The well-known Bonferroni procedure controls the FWER, and a natural extension is to introduce weights into this procedure to reflect the a-priori importance of hypotheses or to maximise some measure of the overall power of the experiment. In this paper, we consider analysing a replication study using an optimal weighted Bonferroni procedure, with the weights based on the results of the original study that is being replicated and the optimality criterion being to maximise the disjunctive power of the trial (the power to reject at least one non-null hypothesis). We show that using the proposed procedure can lead to a substantial increase in the disjunctive power of the replication study, and is robust to changes in the effect sizes between the two studies.

stat.ME↗

arXiv2025

Is 1:1 Always Most Powerful? Why Careful Determination of Allocation Ratios Matters in Trial Design

The principle of allocating an equal number of patients to each arm in a randomized controlled trial remains widely believed to be optimal for maximising statistical power. However, this long-held belief only holds true if the treatment groups have equal outcome variances, a condition that is often not met or, is simply not assessed in practice. This paper reasserts the fact that a departure from a 1:1 ratio can maintain or improve statistical power while increasing the benefits to participants. The benefit is particularly self-evident for binary and time-to-event endpoints, where variances are determined by the assumed success or event rates. To illustrate this, we present two case studies: a small-scale metastatic melanoma trial with a binary endpoint and a larger trial evaluating virtual reality for pain reduction with a continuous endpoint. Our simulations compare equal randomisation, preplanned fixed unequal randomisation, and response-adaptive randomisation targeting Neyman allocation. Results show that unequal allocation can increase the proportion of patients receiving the superior treatment without reducing power, with modest power gains observed in both binary and continuous settings, highlighting the practical relevance of optimised allocation strategies across trial types and sizes.

stat.ME↗

arXiv2025

A comparison of approaches to incorporate patient-selected and patient-ranked outcomes in clinical trials

A key aspect of patient-focused drug development is identifying and measuring outcomes that are important to patients in clinical trials. Many medical conditions affect multiple symptom domains, and a consensus approach to determine the relative importance of the associated multiple outcomes ignores the heterogeneity in individual patient preferences. Patient-selected outcomes offer one way to incorporate individual patient preferences, as proposed in recent regulatory guidance for the treatment for migraine, where each patient selects their most bothersome migraine-associated symptom in addition to pain. Patient-ranked outcomes have also recently been proposed, which go further and consider the full ranking of the relative importance of all the outcomes. This can be assessed using a composite DOOR (Desirability of Outcome Ranking) endpoint. In this paper, we compare the advantages and disadvantages of using patient-selected versus patient-ranked outcomes in the context of a two-arm randomised controlled trial for multiple sclerosis. We compare the power and type I error rate by simulation, and discuss several other important considerations when using the two approaches.

stat.ME↗

arXiv2025

A burn-in(g) question: How long should an initial equal randomization stage be before Bayesian response-adaptive randomization?

Response-adaptive randomization (RAR) can increase participant benefit in clinical trials, but also complicates statistical analysis. The burn-in period (a non-adaptive initial stage) is commonly used to mitigate this disadvantage, yet guidance on its optimal duration is scarce. To address this critical gap, this paper introduces an exact evaluation approach to investigate how the burn-in length impacts statistical operating characteristics of two-arm binary Bayesian RAR (BRAR) designs. We show that (1) commonly used calibration and asymptotic tests show substantial type I error rate inflation for BRAR designs without a burn-in period, and increasing the total burn-in length to more than half the trial size reduces but does not fully mitigate type I error rate inflation, necessitating exact tests; (2) exact tests conditioning on total successes show the highest average and minimum power up to large burn-in lengths; (3) the burn-in length substantially influences power and participant benefit, which are often not maximized at the maximum or minimum possible burn-in length; (4) the test statistic influences the type I error rate and power; (5) estimation bias decreases quicker in the burn-in length for larger treatment effects and increases for larger trial sizes under the same burn-in length. Our approach is illustrated by re-designing the ARREST trial.

stat.ME↗

arXiv2024

Confidence intervals for adaptive trial designs I: A methodological review

Regulatory guidance notes the need for caution in the interpretation of confidence intervals (CIs) constructed during and after an adaptive clinical trial. Conventional CIs of the treatment effects are prone to undercoverage (as well as other undesirable properties) in many adaptive designs, because they do not take into account the potential and realised trial adaptations. This paper is the first in a two-part series that explores CIs for adaptive trials. It provides a comprehensive review of the methods to construct CIs for adaptive designs, while the second paper illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. We describe several classes of techniques for constructing CIs for adaptive clinical trials, before providing a systematic literature review of available methods, classified by the type of adaptive design. As part of this, we assess, through a proposed traffic light system, which of several desirable features of CIs (such as achieving nominal coverage and consistency with the hypothesis test decision) each of these methods holds.

stat.ME↗

arXiv2024

Confidence intervals for adaptive trial designs II: Case study and practical guidance

In adaptive clinical trials, the conventional confidence interval (CI) for a treatment effect is prone to undesirable properties such as undercoverage and potential inconsistency with the final hypothesis testing decision. Accordingly, as is stated in recent regulatory guidance on adaptive designs, there is the need for caution in the interpretation of CIs constructed during and after an adaptive clinical trial. However, it may be unclear which of the available CIs in the literature are preferable. This paper is the second in a two-part series that explores CIs for adaptive trials. Part I provided a methodological review of approaches to construct CIs for adaptive designs. In this paper (part II), we present an extended case study based around a two-stage group sequential trial, including a comprehensive simulation study of the proposed CIs for this setting. This facilitates an expanded description of considerations around what makes for an effective CI procedure following an adaptive trial. We show that the CIs can have notably different properties. Finally, we propose a set of guidelines for researchers around the choice of CIs and the reporting of CIs following an adaptive design.

stat.ME↗

arXiv2023

The use of restricted mean survival time to estimate treatment effect under model misspecification, a simulation study

The use of the non-parametric Restricted Mean Survival Time endpoint (RMST) has grown in popularity as trialists look to analyse time-to-event outcomes without the restrictions of the proportional hazards assumption. In this paper, we evaluate the power and type I error rate of the parametric and non-parametric RMST estimators when treatment effect is explained by multiple covariates, including an interaction term. Utilising the RMST estimator in this way allows the combined treatment effect to be summarised as a one-dimensional estimator, which is evaluated using a one-sided hypothesis Z-test. The estimators are either fully specified or misspecified, both in terms of unaccounted covariates or misspecified knot points (where trials exhibit crossing survival curves). A placebo-controlled trial of Gamma interferon is used as a motivating example to simulate associated survival times. When correctly specified, the parametric RMST estimator has the greatest power, regardless of the time of analysis. The misspecified RMST estimator generally performs similarly when covariates mirror those of the fitted case study dataset. However, as the magnitude of the unaccounted covariate increases, the associated power of the estimator decreases. In all cases, the non-parametric RMST estimator has the lowest power, and power remains very reliant on the time of analysis (with a later analysis time correlated with greater power).

stat.ME↗

arXiv2023

Online multiple hypothesis testing

Modern data analysis frequently involves large-scale hypothesis testing, which naturally gives rise to the problem of maintaining control of a suitable type I error rate, such as the false discovery rate (FDR). In many biomedical and technological applications, an additional complexity is that hypotheses are tested in an online manner, one-by-one over time. However, traditional procedures that control the FDR, such as the Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. To address these challenges, a new field of methodology has developed over the past 15 years showing how to control error rates for online multiple hypothesis testing. In this framework, hypotheses arrive in a stream, and at each time point the analyst decides whether to reject the current hypothesis based both on the evidence against it, and on the previous rejection decisions. In this paper, we present a comprehensive exposition of the literature on online error rate control, with a review of key theory as well as a focus on applied examples. We also provide simulation results comparing different online testing algorithms and an up-to-date overview of the many methodological extensions that have been proposed.

stat.ME↗

arXiv2022

Point estimation for adaptive trial designs I: a methodological review

Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value", and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realised trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This paper is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this paper can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.

stat.ME↗