Value Proposition

The risky nature of sub-group analyses

Most clinical studies include pre-planned analyses of patient sub-groups.  Sub-groups are a sub-set of the clinical trial population defined by one or more descriptive factors, either intrinsic or extrinsic, and determined a priori. Such factors include demographics, disease characteristics and clinical considerations.  Post-baseline variables are not usually appropriate as a basis for sub-group definition. Factors may be categorical or continuous and consideration should be given to groupings, cut-off points and their relevance to decision making in clinical practice. This includes groupings of “risk score”, based on multiple predictive or prognostic characteristics. If a signal is observed in a sub-set, there may need to be analysis of different cut-offs/groupings that are clinically meaningful.

Generally the more homogenous the study population, the lower the importance of sub-group analyses. Heterogeneity can arise from the indication, the inclusion/exclusion criteria, prognostic factors, the investigational product and the countries/regions where the development programme takes place. However, such analyses can demonstrate consistency of effect e.g. use of Forest plots (with associated confidence intervals) to assess heterogeneity of treatment effects in sub-groups is a well used pathway to identify signals of the applicability of overall treatment effect across a full population. When multiple sub-groups are considered, the risk of false positive findings increases (the effect in the sub-group is thought to differ from the primary analysis population when in fact this is not so). The treatment effect in the complementary sub-population will be correspondingly lower. Generally such an analysis will not be accepted as primary evidence of efficacy and/or safety.
Generally, three situations arise when sub-group analyses are of relevance/benefit?

  • Overall the study is positive and verification of efficacy in sub-groups is sought
  • The study is statistically positive but the therapeutic benefit or the benefit: risk is borderline/unconvincing and a sub-group where clinical efficacy or benefit: risk is more convincingly demonstrated is sought
  • The clinical data is not statistically persuasive but a sub-group where treatment effect is compelling is sought.

The type of scale of change has an important bearing on the estimation of treatment effect. Relative changes are likely to be more homogenous (e.g. patients with mild disease have less headroom for large improvement) than absolute changes (in populations with severe disease who have the capacity to show greater absolute improvement). The scale used should generally be the one that is commonly used/clinically relevant to the eventual benefit: risk consideration.
The more heterogeneous the clinical trial population, the more important to investigate consistency of effects in sub-groups but there should always be biological plausibility. This plausibility is enhanced if there are two or more sources of evidence (e.g. two clinical studies).
The analyses are usually planned by initial investigation of key subgroups, usually those specified by stratification at randomisation in to the study and then at a second level, truly exploratory analyses, where there is an argument for a treatment effect in a sub-group.
The analyses are usually documented in the clinical study protocol and the associated statistical analysis plan and should include the pre-specification of sub-groups and the key stratification factors.
Visual inspection of a Forest plot can help indicate sub-sets of the population that may merit further investigation but there is no didactic rule as to when this should be done and subsets with very small patient numbers and wide confidence intervals should not be over-interpreted.