The second in a series of State of the Art Reviews on statistical principles, published Dec. 7 in the Journal of the American College of Cardiology, looks at several controversial statistical issues that are commonly faced in the presentation and interpretation of trial findings.

Stuart J. Pocock, PhD, et al., posit that a number of problems in trial reporting can make it difficult to provide a balanced account of a trial’s findings. Multiplicity of data, covariate adjustment, subgroup analysis, individual benefits and risks, intention to treat (ITT) analysis, and the challenge of interpreting unexpected results all can affect the explication of a trial’s findings.

Multiplicity of data, i.e., the large number of variables collected at baseline and during follow-up, presents a challenge to objectivity when analyzing results. To avoid “playing up the positive,” the authors explain that investigators must start with a predefined statistical analysis plan that is fully signed off before database locking and study unblinding. “A good statistical analysis plan will not only document exactly which analyses are to be done, but will also elucidate relevant priorities in their interpretation, especially regarding the primary hypotheses, secondary hypotheses, any pre-defined safety concerns, and a potential plethora of exploratory analyses,” state Pocock, et al.

The authors also discuss the value of covariate adjustment, suggesting that investigators adjust for variables affecting prognosis. “In general, we believe that a well-defined appropriate covariate-adjusted analysis is well worth doing in major randomized controlled trials (RCTs),” they state. The authors then present a set of four principles that should be followed when performing a covariate-adjusted analysis.

Presenting subgroup findings from major RCTs is a critical responsibility, and must be done cautiously to avoid overreaction to any subgroup claims, according to the authors. In addition, they explain that investigators should be careful to note any important differences between individuals “as regards absolute treatment benefits.” To illustrate this, the authors provide two examples that show why “one needs to consider the individual’s risk status in determining whether the absolute benefit of an intervention is sufficient to merit its use in each case.” “This becomes particularly important if treatment efficacy is offset by a risk of side effects,” they note. 

The review gives additional recommendations regarding the complications of analysis by ITT, including ways of determining whether a pure ITT or modified ITT is appropriate. The authors close with suggested methods for interpreting unexpected findings so as not to exaggerate their indications. A checklist of items to include in a clinical trial is also included in the review. 

Keywords: Intention to Treat Analysis, Play and Playthings, Prognosis, Randomized Controlled Trials as Topic, Research Personnel, Risk, Risk Assessment, Treatment Outcome

< Back to Listings