CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials

Examples

“A study patient advisory group advised on study design before funding, in study set-up, and during recruitment. They chose the term best current treatment and informed the design of clinic procedures (including how best to reduce the burden of intervention), questionnaire design, and participant information. This group informed protocol modifications in response to low recruitment, and guided interpretation of the findings. Two public contributors were members of the independent trial steering committee.”140

“The UK Musculoskeletal Trauma Patient and Public Involvement (PPI) Group co-designed this study. In particular, the group advised on the choice of outcome measures and the follow-up arrangements, which were designed to limit the number of face-to-face hospital visits needed. Subsequently, a patient member from this group became a member of the DRAFFT2 Trial Management Group, overseeing all elements of the set-up and delivery of the trial and the dissemination of the lay summary at completion. Another patient member of the group was also a member of the independent Trial Steering Committee.”141

“Office workers, workplace champions, and managers within the target organisation were involved in the study design during the grant application process and the study delivery phase. During the grant application phase, the purpose and design of the study as well as the suggested intervention strategies were presented to two large groups of council employees. As a result of these meetings, the study design included using finger prick blood testing rather than taking venous blood samples, participants receiving feedback on health measures, and incentives for attending follow-up. During the study set-up and delivery, a council employee advisory group met several times and provided advice on delivery of the interventions (feedback showed that workplace champions would not be comfortable delivering the initial education session because of the training and planning time required, so this session was delivered online instead), recruitment processes (feedback was provided on participant documents and recruitment messages and strategies within the council), installation of the height adjustable desk, and troubleshooting. Two council employees were also part of the trial steering committee, which met twice a year during the study”.142

“In the context of the pandemic and the need to design the study in a short period, no patients were involved in setting the research questions or the outcome measures, nor were they involved in developing plans for recruitment, design, or implementation of the study. No patients were asked to advise on interpretation or writing up of results.”143

Explanation

Patient and public involvement (PPI) has been shown to be particularly beneficial in clinical trials.144 It can help researchers to identify and prioritise research topics and questions; identify relevant outcome measures; boost recruitment and retention; improve trial design and tools; and improve the acceptability of trials.144145 PPI can also improve the communication and dissemination of the trial results to participants. Public involvement in other types of health research has been shown to help researchers to engage under-served populations and recruit diverse participant groups.144 Thus, transparent reporting of PPI is essential to allow readers to appraise the relevance and usefulness of findings to end users and to fully evaluate and understand a trial’s methodology and conduct. If patients and the public were not involved, authors should report this with the reasons why.

PPI in health research entails collaborating or partnering with patients and members of the public to design, conduct, report, interpret, or disseminate research: the research is done by or with patients and the public—rather than done for, at, or about them.146147148 Importantly, this is distinct from including patients or members of the public in a trial as participants. PPI contributors can be people with current or past experience of a health condition; their families, carers, and advocates; members of communities who are target users of an intervention or service; or members of the wider public with a broader perspective. The terminology used differs internationally: for example, such activity is most commonly known as “patient and public involvement” in the UK, whereas “patient engagement” is more common in mainland Europe and North America, 147148, “community and public engagement” is commonly used in part of Africa, and “consumer involvement” is frequently used in Australia.149

The GRIPP (Guidance for Reporting Involvement of Patients and Public) checklist was developed in 2011 with the aim of improving the reporting of PPI activities in health research150; followed in 2017 by GRIPP2.151 The GRIPP2 checklist includes GRIPP2-SF, which is a short form of the GRIPP2 checklist where PPI is the secondary focus of the research.151 Examples of reporting of PPI involvement could include whether and how patients were involved in the trial objectives, whether patients advised on optimising patient recruitment and retention, and whether and how patients were included in selection of the trial outcomes.

Funding bodies are increasingly encouraging or requiring researchers to include PPI in grant applications,146152153 but mandating of PPI reporting by journals has remained uncommon. In 2014, The BMJ introduced a requirement for submitted manuscripts to include a PPI statement,154 and this was extended to other BMJ journals from 2018.155156 A 2023 study of trials addressing chronic conditions found that approximately 80% of trial reports published in these journals included a PPI subsection, and around 40% of these reported that PPI activities had been conducted.157 Few other journals have followed suit in mandating reporting of PPI activities.158159160 In the absence of an explicit requirement, reporting of PPI activities remains infrequent, appearing in an estimated 0-5% of published trial reports.161162163 There is limited evidence to indicate the extent to which this reflects a lack of PPI activity versus PPI activity being conducted but not reported. In a small survey of authors of trials published in high impact factor journals, only one of 29 respondents reported PPI activities having been included but not mentioned in the published article163; while in a survey of authors of pragmatic trials, 47% reported including PPI activities during a trial but less than a quarter of these reported the activity in the corresponding publication.164

Examples

“This was a multicenter, stratified (6 to 11 years and 12 to 17 years of age), with imbalanced randomization [2:1], double-blind, placebo-controlled, parallel-group study conducted in the United States (41 sites).”165

“This multicentre, pragmatic, superiority randomised trial compared three parallel groups for patients referred to secondary care for treatment of primary frozen shoulder, who were recruited from 35 hospital sites in the UK. Individual participants were randomly assigned with unequal allocation (2:2:1) to arthroscopic capsular release, manipulation under anaesthesia, or early structured physiotherapy, to allow for different effect sizes between groups.”80

Explanation

The word “design” is often used to refer to all aspects of how a trial is set up, but it also has a narrower interpretation. Many aspects of the broader trial design, including details of randomisation and blinding, are addressed elsewhere in the CONSORT checklist. This item refers to the type of trial (eg, parallel group or crossover) and the conceptual framework (eg, superiority, equivalence, or non-inferiority).

CONSORT 2025 focuses mainly on trials with participants individually randomised to one of two parallel groups. While many published trials have such a design, the main alternative designs are multi-arm parallel,166 crossover,167 cluster,168 and factorial designs,138 with each having their own specific CONSORT extensions. A detailed review of randomised trials published in PubMed in 2012 showed that of the 1122 trials, 953 (85%) were parallel group; the other main designs were crossover (n=98; 13%) and cluster (n=31; 3%). Of these 1122 trials, 892 (80%) had two groups, 146 (13%) had three groups, and 61 (5%) had four or more groups.169

Most trials are designed to identify the superiority of a new intervention, if it exists, but others are designed to assess non-inferiority or equivalence.129 It is important that researchers clearly describe the design of their trial, including the unit of randomisation (eg, patient, general practice, lesion). These details should also be included in the abstract (item 1b).

If a less common design is used, authors should explain their choice, especially as such designs may imply the need for a larger sample size or more complex analysis and interpretation. Although most trials use equal randomisation (eg, 1:1 for two groups), it is also helpful to provide the allocation ratio explicitly.

Examples

“The original primary endpoint was all-cause mortality, but, during a masked analysis, the data and safety monitoring board noted that overall mortality was lower than had been predicted and that the study could not be completed with the sample size and power originally planned. The steering committee therefore decided to adopt co-primary endpoints of all-cause mortality (the original primary endpoint), together with all-cause mortality or cardiovascular hospital admissions (the first prespecified secondary endpoint).”170

“As described in the published protocol paper, a protocol amendment was made to revise the sample size in response to new information on the minimal clinically important difference of the primary outcome measure, the Patient-Oriented Eczema Measure (POEM). Our original sample size used a POEM score for minimal clinically important difference of 3, which was based on research carried out in secondary care among people with moderate or severe eczema. Fresh evidence, however, suggested that a change in POEM score of 2.1 to 2.9 represents a change likely to be beyond measurement error. A protocol amendment was therefore made to change the target sample size for the trials based on seeking to detect a difference in POEM score of 2.5 points between groups, increasing the target sample size from 200 to 303 for each trial.”171

Explanation

A protocol for a randomised trial serves as the foundation for planning, conduct, reporting, and appraisal, specifying in detail how the trial will be conducted. There may be deviations from the original protocol, as it is impossible to predict every possible change in circumstances during the course of a trial. Some trials will therefore have important changes to the methods after trial commencement. There are many reasons for deviations from the initial study protocol; for example, changes could be due to external information becoming available from other studies, or internal financial difficulties, or lower than anticipated recruitment rates. In some trials, an independent data monitoring committee will have as part of its remit the possibility of recommending protocol changes based on seeing unblinded data. Authors should report the nature and timing of protocol changes because changes made at different times (eg, before or after breaking the blinding) might be associated with different risks of bias.

Authors should report all major changes to the trial after it commenced indicating the reason for the changes and when the changes occurred.172 Such changes might affect the trial methods, such as the randomisation ratio, eligibility criteria, interventions, outcomes, method of analysis or duration of follow-up; or might affect the trial conduct, such as dropping a trial site with poor data quality.173

Some trials are set up with a formal adaptive design, which allows pre-planned changes to an ongoing trial without compromising the validity of conclusions.174 It is therefore essential to distinguish pre-planned changes from unplanned changes that may also occur. Such adaptive trial design modifications are usually to the sample size and number of treatment groups, and can lead to decisions being made more quickly and with more efficient use of resources than would be possible with traditional, non-adaptive parallel group trials. Specific guidance has been developed for reporting trials with a formal adaptive design; authors could consult this for more detailed information.174

Most trials record multiple outcomes, with the risk that results will be reported for only a selected subset. Prespecification and reporting of completely defined primary and secondary outcomes for both benefits and harms (item 14) should remove such a risk. In some trials, however, circumstances require a change in the way an outcome is assessed, the designation of outcomes as primary or secondary or even, as in the example above, a switch to a different outcome. For example, there may be external evidence from other trials or systematic reviews suggesting the time point for the primary outcome might not be appropriate; or recruitment or the overall event rate in the trial may be lower than expected.170 Changing an endpoint based on unblinded data are much more problematic, although may be specified in the context of an adaptive trial design.174

Whether the modifications are explicitly part of the trial design or in response to changing circumstances, it is essential that they are fully reported and the reason for the change explained to help the reader interpret the results. Such information is not always reported. A comparison of protocols and publications of 102 randomised trials found that 62% of trial reports had at least one primary outcome that was changed, introduced, or omitted compared with the protocol.51 Primary outcomes also differed between protocols and publications for 40% of a cohort of 48 trials funded by the Canadian Institutes of Health Research.175 None of these subsequent 150 trial reports mentioned, let alone explained, changes from the protocol. Similar results from other studies have been reported in a systematic review of empirical studies, comparing trial registers or protocols to published trial reports.176

Examples

“The study was conducted in paediatric diabetes services experienced in the use of CSII [continuous subcutaneous insulin infusion], in nine University and six local hospitals within the NHS in England and Wales.”177

“Setting: One large health board area with a materially deprived, inner city population in the west of Scotland, United Kingdom. Although described as a single centre, NHS Greater Glasgow and Clyde has the largest health board population (1.2m) in the United Kingdom. It is spread over a wide geographical area including severely materially deprived post-industrial areas as well as some more affluent communities. Maternity booking and antenatal care are provided in both hospital and local healthcare settings. Delivery (13 844 infants in 2013[reference]) takes place in three major maternity hospitals.”178

Explanation

Along with the eligibility criteria for participants (item 12a) and the description of the interventions (item 13), information on the settings and locations where the trial was conducted is crucial to enable readers to judge the applicability and generalisability of a trial. Were participants recruited from primary, secondary, or tertiary healthcare, or from the community? Healthcare institutions vary greatly in their organisation, experience, and resources and the baseline risk for the condition under investigation. Other aspects of the setting, including the social, economic, and cultural environment, might also affect a study’s external validity.

Authors should report the number and type of settings and describe the care providers involved. They should report the locations in which the study was carried out, including the country, city if applicable, and immediate environment (eg, community, general practice, hospital outpatient clinic, or inpatient unit). In particular, it should be reported whether the trial was carried out in one site (single centre trials) or several sites (multicentre trials). The description of the setting should provide enough information so that readers can judge whether the results of the trial could be relevant to their own setting. The environment in which the trial is conducted might differ considerably from the setting in which the trial’s results are later used to guide practice and policy.179180 Authors should also report any other information about the settings and locations that could have influenced the observed results.

Example

“Patients aged 18 years or older were recruited from 20 UK National Health Service (NHS) trusts. Patients were eligible if they had a diagnosis of shoulder pain attributable to a rotator cuff disorder (eg, cuff tendonitis, impingement syndrome, tendinopathy, or rotator cuff tear) that had started within the past 6 months. We used the diagnostic criteria set out in the British Elbow and Shoulder Society (BESS) guidelines. Patients were excluded if they had a history of significant shoulder trauma (eg, dislocation, fracture, or full-thickness tear requiring surgery), neurological disease affecting the shoulder, other shoulder conditions (eg, inflammatory arthritis, frozen shoulder, or glenohumeral joint instability), received corticosteroid injection or physiotherapy for shoulder pain in the past 6 months, or were being considered for surgery. Detailed criteria are in the protocol.”181

Explanation

A comprehensive description of the eligibility criteria used to select the trial participants is needed to help readers interpret the study. In particular, all inclusion and exclusion criteria should be reported to judge to whom the results of a trial apply—that is, the trial’s generalisability (applicability) and relevance to clinical or public health practice (item 30).179 A description of the method of recruitment, such as by referral or self-selection (eg, through advertisements) is also important in this context. Because they are applied before randomisation, eligibility criteria do not affect the internal validity of a trial, but they are central to its external validity.

Typical and widely accepted selection criteria relate to the nature and stage of the condition or disease being studied, the exclusion of persons thought to be particularly vulnerable to harm from the study intervention, and to issues required to ensure that the study satisfies legal and ethical norms. The informed consent of study participants, for example, is typically required in intervention studies. Where relevant, it is important to describe whether sex and/or gender were taken into account in the design of the trial, including whether there was adequate representation of men and women (or diverse genders), and justify the reasons for any exclusion.182

Despite their importance, eligibility criteria are often not reported adequately. For example, in an analysis of 283 reports of trials published between 1994 and 2006 in high impact general medical journals, reporting of exclusion criteria was often poor and incomplete: 84% of published trials contained at least one poorly justified exclusion criteria, and in 61% more than a quarter of the trial’s exclusion criteria were poorly justified.183 Any differences in eligibility criteria between the trial protocol and final publication should also be highlighted and reasons for any discrepancies reported. A review of 52 protocols and 75 subsequent full publications submitted to a German medical ethics committee between 2000 and 2006 identified modifications to the eligibility criteria for 85% of trial publications, with 41% of final publications containing newly added criteria.184 Similar deficiencies have been found in other studies.185186

Example

“All participating centres . . . were major neurosurgical centres, treating large numbers of patients after aneurysmal subarachnoid haemorrhage (SAH), each centre treating between 60 and 200 cases annually . . . Centres had to have expertise in both neurosurgical and endovascular management of ruptured aneurysms. Only accredited neurosurgeons with experience of aneurysm surgery were permitted to manage patients in the trial. Endovascular operators had to have done a minimum of 30 aneurysm treatment procedures, before they were permitted to treat patients in the trial”.187

Explanation

For all types of trials, it is important to define the eligibility criteria for clinical sites, and to consider the characteristics of the treatment providers who will provide both the experimental and comparator interventions.22188 Evidence suggests that patient outcomes can be associated with hospital and care provider volume.188 A systematic review of 135 trials found that 71% observed a positive association between hospital volume and outcomes, and 70% observed an association between care provider volume and outcomes.189 Different levels of expertise of care providers in each intervention group can bias treatment effect estimates.190 Furthermore, an intervention might be found to be safe and effective in a trial performed in high volume sites by high volume care providers but could have different results in low volume sites. For example, in an analysis of Medicare National Claim files of 167 208 patients who had undergone coronary stent surgery, patients treated by high volume physicians and at high volume sites experienced better outcomes than those treated by low volume physicians at low volume sites.191 Similar studies show that in most non-pharmacological trials, care provider expertise and site volume will influence treatment effects.192193194

Eligibility criteria for care providers and sites are often poorly reported. A systematic review of randomised trials in surgery found that the setting and the site volume of activity were reported in only 7% and 3% of articles, respectively. Eligibility criteria were reported for care providers in 41% of the articles, and the number of care providers performing the intervention was reported in 32%.195 A careful description of care providers involved in the trial, as well as details of the sites in which participants were treated, helps readers appraise the risk of bias and the applicability of the results.22188 Eligibility criteria for sites typically relate to site volume for the procedure under investigation or similar procedures. Eligibility criteria for care providers might include professional qualifications, years in practice, number of interventions performed, skill as assessed by level of complication when performing the intervention, and specific training before trial initiation. Exclusion criteria should be justified because they will influence the applicability of the trial results.188

Examples

“Each sulfadoxine–pyrimethamine course consisted of three tablets containing 500 mg of sulfadoxine and 25 mg of pyrimethamine (unbranded generic sulfadoxine–pyrimethamine, Medopharm, Chennai, India; quality controlled by Durbin, Hayes, UK) given as a single oral dose for 1 day (appendix p 2). Each dihydroartemisinin–piperaquine course was dosed according to the bodyweight of each participant and consisted of three to five tablets containing 40 mg of dihydroartemisinin and 320 mg of piperaquine (Alfasigma, Bologna, Italy), given orally once a day for 3 consecutive days. Each dose of azithromycin consisted of two tablets containing 500 mg (Universal Corporation, Nairobi, Kenya) given orally once daily for 2 consecutive days (cumulative dose of 2 g) at the same time as the first and second daily dose of dihydroartemisinin–piperaquine at enrolment. The placebo tablets were also provided by Universal Corporation and had the same appearance as active azithromycin (appendix p 2). The first daily dose was administered in the study clinic under the direct supervision of the study staff, combined with a slice of dry bread or a biscuit. The daily doses on the second and third days were self-administered at home at approximately the same time of the day and in a similar manner as the first dose taken under observation in the clinic.”196

“The experimental group received 6 sessions of standard OMT (osteopathic manipulative treatment), and the control group 6 sessions of sham OMT, each session at 2-week intervals. For both experimental and control groups, each session lasted 45 minutes and consisted of 3 periods: (1) interview focusing on pain location, (2) full osteopathic examination, and (3) intervention consisting of standard or sham OMT. Briefly, in both groups, practitioners assessed 7 anatomical regions for dysfunction (lumbar spine, root of mesentery, diaphragm, and atlantooccipital, sacroiliac, temporomandibular, and talocrural joints) and applied sham OMT to all areas or standard OMT to those that were considered dysfunctional. All health care providers were board-certified nonphysician, nonphysiotherapist osteopathic practitioners (Répertoire National de la Certification Professionnelle, niveau 1). They all received a 2-day training according to international standards to deliver both standard and sham OMT. Full descriptions of osteopathic practitioner training and interventions are provided in eAppendices 3 and 4 in Supplement 2. In both groups, pharmacological interventions, nonpharmacological interventions, and spinal surgery were allowed. Cointerventions were self-reported at 3, 6, and 12 months by use of a standardized checklist (eAppendix 5 in Supplement 2).”197

Explanation

Complete reporting of the intervention and comparator details is essential to enable readers to understand the study results and adequately translate them to clinical practice. Several studies have shown poor reporting of interventions and comparators in randomised trials.195198199200201202203204205206 Authors should describe each intervention thoroughly, including control interventions, or use of placebo procedure.207208 The description should provide sufficient detail to allow replication, such as to allow a clinician wanting to use the intervention to know exactly how to administer the intervention/comparator that was evaluated in the trial.198 Key information includes the different components of the intervention/comparator, how and when it should be administered, the intervention/comparator material (ie, any physical or informational materials used in the intervention/comparator, including those provided to participants or used in intervention/comparator delivery or in training of providers and where it can be accessed (eg, online appendix, URL)); the procedure for tailoring the intervention/comparator to individual participants, and how fidelity209 (ie, the extent to which the intervention/comparator were implemented as planned in the protocol by care providers) or adherence209 (ie, the extent to which trial participants implement the intervention/comparator as planned in the protocol) were assessed or enhanced (box 2).

Assessing fidelity and adherence can be complex and vary according to the intervention/comparator (eg, one-off, short term repeated, long term repeated). Various deviations to the protocol can occur. For example, participants might initiate the intervention but then discontinue the intervention completely and permanently after a specific period of time, discontinue the intervention temporarily, reduce the dose, or modify the schedule. If relevant, authors should provide the prespecified definition for classifying participants as being treated as planned or not.

In addition, authors should indicate whether criteria were used to guide intervention/comparator modifications and discontinuations and where applicable describe these criteria. This information could be particularly important to evaluate the risk of bias due to deviations from the intended interventions,210211 an important domain of the risk-of-bias tool developed by Cochrane.210 Assessing this domain requires a clear understanding of, and ability to distinguish between, deviations that occur as planned in the protocol and deviations that arise due to the experimental context.

The research question (ie, explanatory v pragmatic) will affect the standardisation of the intervention/comparator as well as how adherence or fidelity is assessed or enhanced. In explanatory trials, the aim is to estimate treatment effect under ideal circumstances. The intervention/comparator are usually highly standardised with close monitoring of fidelity and adherence to the intervention/comparator and strategies to increase them. In contrast, pragmatic trials aim to determine treatment effect in clinical conditions. The intervention and comparator are usually highly flexible, and measurement of fidelity and adherence are unobstructive with no strategies to maintain or improve them. Nevertheless, assessing fidelity and adherence to the intervention/comparator, or at least recording the most important components of the intervention/comparator, is necessary to understand what was actually administered to participants. This is particularly important for complex interventions where diversity in the implementation of the intervention is expected. For example, in a pragmatic trial assessing a surgical procedure where the procedure is left to surgeons’ choice, investigators should plan to systematically record key elements related to pre-operative care, anaesthesia, the surgical approach, and post-operative care. This information is essential to provide a relevant description of the intervention that was actually provided when the trial is completed.

If the control group or intervention group received a combination of interventions, the authors should provide a thorough description of each intervention, an explanation of the order in which the combination of interventions were planned to be introduced or withdrawn, and the triggers for their introduction if applicable. Some complex interventions will require the development of specific documentation (eg, training materials, intervention manuals). Authors should make these available and indicate where they can be accessed.

If the control group is to receive usual care, it is important to describe what that constitutes so that readers can assess whether the comparator differs substantially from usual care in their own setting.212 Various approaches could be used: standardising usual care to be in line with specific guidelines; or asking practitioners to treat control patients according to their own preference, which could result in heterogeneity of the care provided particularly between centres and over time.213 Usual care can vary substantially across sites and patients, as well as over the duration of the trial. Further, it is important to clarify if the experimental group also received usual care in addition to the experimental intervention, and what actually differed between the groups. Usual care is frequently incompletely reported. In a review of 214 paediatric trials, the descriptions of standard of care were more often incomplete than the description of the intervention arms within the same study as measured by the TIDieR checklist (mean 5.81 (standard deviation (SD) 2.13) v 8.45 (SD 1.39)).205

If the control group is to receive a placebo, specific considerations need to be accounted for. Some evidence showed that placebos are insufficiently described.214 Placebo could have several different aspects from pills to saline injections or more complex interventions such as sham interventions (eg, sham surgery) or attention control interventions. Authors should report the same level of details as required for the intervention—that is, content of the placebo or different component of the placebo, how and when it should be administered, material, procedure for tailoring the placebo to individual participants, and how fidelity and adherence were assessed or enhanced.207 Complete reporting of the placebo is needed to understand what intervention effect is measured in the trial.215 A network meta-analysis of osteoarthritis trials showed that different placebo interventions (oral, intra-articular, topical, oral and topical) had different effects and can impact the relative effect estimate of active treatments.216

Further, the trial groups could receive different concomitant care in addition to the assigned trial interventions. Concomitant care could impact trial outcomes and bias effect estimates. To facilitate interpretation of study results and risk-of-bias assessments, authors should report relevant concomitant care that was allowed or prohibited where relevant.

Specific guidance has been developed to improve the reporting of interventions, particularly TIDieR,23 TIDieR-Placebo for placebo and sham controls,207 and the CONSORT extensions for non-pharmacological treatments.22 Authors could consult these for more detailed information.

Example

See table 5.21

Explanation

All randomised trials assess outcomes, for which the groups are compared. Most trials have several outcomes, some of which are of more importance than others. The primary outcome is the prespecified outcome considered to be of greatest importance to relevant stakeholders (such as patients, policy makers, clinicians, and funders) and should be the one used in the sample size calculation (item 16). The primary outcome should be explicitly indicated as such in the report of a randomised trial. Other outcomes of interest are secondary outcomes.

It is important to explain the rationale and clinical relevance for chosen efficacy and harm outcomes, including whether they are part of a core outcome set.217218 A core outcome set is an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical trials in specific areas of health or health care. The COMET (Core Outcome Measures in Effectiveness Trials) initiative and COMET database facilitate access to core outcome sets (https://www.comet-initiative.org/).

Most trials have a single primary outcome. Having several primary outcomes can incur potential problems of interpretation associated with multiplicity of analyses (items 28 and 30). There are typically multiple secondary outcomes (ie, the outcomes prespecified in the trial protocol to assess any additional effects of the intervention). Secondary outcomes can include harms that may include unintended effects of the intervention (item 27).

The primary and secondary outcomes reported should be consistent with the outcomes prespecified in the trial protocol and the registry. Evidence shows important discrepancies between the outcome reported in the registry or protocol and outcomes reported in final publications, frequently in favour of statistically significant results. Any change in outcome(s) specified in the protocol should be reported, with reasons (item 10).51175176

All outcomes, whether primary or secondary, should be described and completely defined. This information is typically also detailed in the trial’s protocol and the trial registry. The principle here is that the information provided should be sufficient to allow others to use the same outcomes.198 For each outcome, it is important to detail: (1) the specific measurement variable, which corresponds to the data collected directly from trial participants (eg, Beck Depression Scale; all cause mortality) with definition where relevant (eg, major bleeding was defined as fatal bleeding or symptomatic bleeding in a critical area or organ; all cause mortality per hospital database); (2) the specific participant level analysis metric, which corresponds to the format of the outcome data that was used from each trial participant for analysis (eg, change from baseline; final value or value at a time point; time to event); (3) the method of aggregation, which refers to the summary measure format for each trial group (eg, mean; proportion of participants with score >2); and (4) the measurement time point of interest for analysis.219 For composite outcomes, all individual components of the composite outcome should be described as secondary outcomes.21 Only half of randomised trials published in PubMed indexed journals in 2000 and 2006 specified the primary outcome.220221 In recent samples of trials published in specific fields, reporting has improved but still two thirds did not provide a complete definition.222223

The use of previously developed and validated scales can help to enhance quality of measurement.224225 For example, assessment of health related quality of life using a validated instrument is critical to the integrity and applicability of the trial.226 Authors should report measurement properties of outcome measurement instruments to assist in interpretation and comparison with similar studies.227

In most trials, information on outcomes is set to be collected as part of the trial conduct. However, some trials may use existing data collecting structures (eg, national, healthcare or administrative registries). This should be clarified in the methods. There is empirical evidence that treatment effect estimates may be different in trials where outcomes are obtained from routinely collected data.228

Examples

“Adverse events (AE) were assessed clinically and analytically at each monthly follow-up visit. The severity of AE were classified according to the National Cancer Institute Common Toxicity Criteria version 4.0. Following the onset of the first cases, the criteria for considering the presence of tenosynovitis were established as spontaneous pain that increased with movement in any tendon insertion with tenderness at that level and observation of localized inflammatory signs of at least 72 hours in duration . . . Patients were considered to have hepatotoxicity when they presented alanine transaminase, aspartate aminotransferase, or bilirubin elevations more than 2 times the upper limit of the normal range. Toxicity was considered severe, and therefore the drug was discontinued when symptomatic elevations were more than 3 times or asymptomatic elevations were more than 5 times the normal levels. All AE were recorded and additional information was required in case of serious adverse events.”229

“Immediate adverse events were assessed by monitoring participants 30 min after injection in the trial centre. All participants were required to report all local and systemic adverse reactions and adverse events after the injection using the trial’s mobile application. Solicited adverse reactions were defined as any events that occurred from day zero to day seven after each injection. Unsolicited adverse reactions were defined as any adverse reactions which occurred from day eight to day 28 after each injection. The severity of adverse reactions was defined using the Food and Drug Administration guidance for industry and toxicity grading scale for healthy adult and adolescent volunteers enrolled in preventive vaccine clinical trial.”230

Explanation

Evaluation and reporting of harms in randomised trials can be useful to inform decision makers on the benefit-risk balance of an intervention.2036 Randomised trials usually lack power and sufficient follow-up to adequately estimate harms231; nevertheless, they can provide data about harms that can be synthesised in meta-analyses if adequately reported.2036 For example, the Women’s Health Initiative trials on hormone therapy provided important data on the cardiovascular risk of hormone replacement therapy.232

Harm relates to the unwanted effects of an intervention. According to the specific context, a given event could be considered for assessing harm (eg, myocardial infarction in a trial assessing non-steroidal anti-inflammatory drugs in patients with osteoarthritis) or benefit (eg, myocardial infarction in a trial assessing aspirin in patients with cardiovascular risk factors) of an intervention.232 The use of the term “harms” is preferred over “safety” to better reflect the negative effect of interventions.20

Despite the importance of having access to data on harms, reporting of this information is poor.127128233234 A review of 184 drug trials published between 2015 and 2016 in four medical journals with high impact factors showed that 28% did not provide any details on how harm data were collected and 89% did not report who decided whether the harm was attributable to the study drug.128 An overview of 13 reviews assessing the reporting of harms in randomised trials using the 2004 CONSORT extension for harms showed that only 40% of the included trials addressed harms outcomes with definitions for each and 44% clarified how harms-related information was collected.127

How harms are defined and assessed will affect the results and effect estimates. Harm can be prespecified or not. They can be systematically assessed or rely on spontaneous declaration (ie, non-systematically assessed). To increase the study’s power, harm can be aggregated in a composite outcome (eg, cardiovascular diseases). Some trials implement a procedure to determine whether harms could be attributed to the intervention (ie, causality).

For each systematically assessed harm, authors should report the definition, the measurement variable (eg, name of a validated questionnaire), and where appropriate, the analysis metric for each participant (eg, time to event), the summary measure for each trial group (eg, proportion), and the time point of interest for analysis. They should describe the procedures for harm assessment including who did the assessment, whether they were blinded to the treatment allocated, the assessment time points, and the overall time period for recording harms. For non-systematically assessed harms, authors should report the mode of data collection with the time point and overall time period for recording harms. Nevertheless, non-systematically assessed harms can be difficult to analyse and interpretation of the results should be viewed with caution. To overcome this issue, trialists can code and group events into specific categories. Nevertheless, the lack of standardisation in data collection could result in selective and incomplete reporting.231 Access to individual participant data may be needed to adequately synthesise this information.231235236

Where appropriate, the process for coding each harm and grading its severity should be described including who did the coding and severity grading, and whether they were blinded to the allocated trial group.

If harms outcomes are aggregated (eg, cardiovascular events, serious events, severe events, withdrawals due to harms, harms imputed to treatment), authors should describe the process for classifying harms, including the grouping system used (eg, grading system to define severity), who did the grouping, and whether they were blinded to the treatment allocated.237238

Box 3 summarises the essential information to be reported related to harms. More detailed information can be found in the CONSORT extension for harms, which was updated in 2022.20

Examples

“We expected an improvement in PFS [progression free survival], in favor of avelumab, with a hazard ratio (HR) of 0.58. Considering a fixed design with a 2-sided α risk of 5% and a power of 80%, 106 events (progression or death) are needed to demonstrate this difference based on the Schoenfeld method. With an estimated recruitment rate of 3 patients per month, a follow-up period for each patient of 24 months, and a percentage of patients lost to follow-up or not evaluable of 15%, 132 patients had to be randomized, and we planned to enroll a total of 66 patients per group.”239

“The target sample size was 300 (150 per arm) over a 3-and-a-half-year recruitment period. This was based on an assumed proportion of individuals with clinically meaningful improvement in VA [visual acuity] (>10 letters) of 55% in the standard care arm and a 19% increase in the adjunct group to 75%, with approximately 7% loss to follow-up, at least 90% power and two-sided 5% type 1 error.”240

“In order to detect a minimum clinically important difference (MCID) in mean volume of daily PA [physical activity] of 2.1 m g [milligravity] at 12 months, and assuming a standard deviation (SD) of 5.3 m g, power of 80%, and a statistical significance level of 5%, a total of 202 participants were required. Allowing for 20% loss to follow-up and 20% non-compliance of accelerometer/intervention attendance meant that at least 338 participants were required (169 per group). The value of 2.1 m g was chosen as it represents an increase in PA that is equivalent to walking at the threshold between light intensity and moderate intensity (for example, 4 km per hour) for 30 min per day or 10–15 min of brisk walking per day.”241

“Sample size was based on the primary outcome measure, HOS ADL [hip outcome score activities of daily living subscale] at eight months post-randomisation, and was calculated using a minimum clinically important difference between groups of 9 points. We estimated the standard deviation to be 14 points; however, summaries presented at a planned interim data monitoring meeting found that the standard deviation was 18 points. A revised calculation (significance level 5%, power 90%, loss to follow-up 20%) gave a sample size of 214 (107 participants in each group). The data monitoring committee approved the sample size increase from 120 to 214 participants.”242

Explanation

Sample size calculations are a key design component for a trial and need careful planning. Sample size calculations need to balance ethical and logistical considerations alongside medical and statistical considerations so that the scientific question can be reliably and precisely answered in a timely manner without unnecessarily exposing individuals to ineffective or harmful interventions. They are generally based on one primary outcome. A trial should therefore be sufficiently large to have a high probability (power) of identifying a clinically important difference of a prespecified size that meets a criterion of statistical significance, if such a difference exists. The magnitude of the effect has an inverse relationship with the sample size required for its detection; that is, larger sample sizes are needed to detect smaller differences. Moreover, the inverse relationship is not linear: very small differences require enormous sample sizes to have good power to detect.

All details on how the sample size was determined should be reported to allow replication (in principle). Elements of the sample size calculation that need to be specified are the primary outcome (and time point) on which the calculation was based (item 14); the anticipated values for the outcome in each trial group (which implies the clinically important target difference between the intervention groups) at a specific time point with rationale or provenance of all quantities, including any relevant citations; or continuous outcomes, the standard deviation of the measurements243; the statistical test; the α (type I error) value and whether it is two sided; the statistical power (or the β (type II error) value); and the resulting target sample size per trial group (box 4). Details should be given of any inflation of the sample size made for attrition or non-adherence during the study. Reference to any formulas or software packages used for the sample size calculation should all be reported. The reporting will have additional considerations for crossover trials,167 factorial trials,138 cluster trials,168 multi-arm trials,166 within-person trials,245 and non-inferiority and equivalence trials.129

Transparency in the sample size reveals the power of the trial to readers and gives them a measure by which to assess whether the trial attained its planned size. Any differences in the planned sample size described in the trial registration (item 2), study protocol (item 3), or statistical analysis plan should be explained.

Interim analyses are used in some trials to help decide whether to stop early or to continue recruiting sometimes beyond the planned trial end (item 16b). If the actual sample size differed from the originally intended sample size for some other reason (eg, because of poor recruitment or revision of the target sample size), an explanation should be given alongside details of the revised sample size. Many reviews have found that few authors report how they determined the sample size.222246247248249250251252

There is no value in conducting and reporting a post hoc calculation of statistical power using the results of a trial, for example, as a pretext to explain non-significant findings; this may even mislead and confuse readers.253254

Examples

“Interim analyses of effectiveness and safety endpoints were performed on behalf of the data monitoring committee on an approximately annual basis during the period of recruitment. These analyses were done with the use of the Haybittle–Peto principle and hence no adjustment was made in the final p values to determine significance.”255

“One interim analysis of the primary endpoint and safety data was planned for when approximately 50% of the participants had completed D28 [day 28]. Statistical significance and futility boundaries were estimated for the interim and final analysis based on 50,000 simulations from the PASS® software (NCSS, Kaysville, Utah) by simulating a group sequential test for two means assuming normality testing. At the interim analysis, the two-sided significance boundary for clinical efficacy was 0.00312 and for futility of detecting μAUT00063 > μplacebo, the one-sided O’Brien-Fleming boundary was 0.39,141. Hence, at the final analysis, the two-sided significance boundary for clinical efficacy would be 0.04761. The Independent Data Monitoring Committee (IDMC) was advised to consider making recommendations for early termination only where there was a clear demonstration of futility.”256

“Three planned analyses (two interim analyses and one final analysis) were performed when the observed number of events were 25, 47, and 84, respectively. Data were released by DSMC [data and safety monitoring committee] after final analysis. Efficacy stopping boundaries were based on the O’Brien-Fleming spending function. Futility boundaries were based on testing the alternative hypothesis at the 0.039 level.”257

“Two interim analyses to be performed using the Haybittle-Peto approach were scheduled, after enrolment of 1000 and 2000 patients, respectively. The significance level associated with both interim analyses was 0.001 and the significance level associated with the final analysis was 0.049. With this method, the overall risk of type 1 error was 5%.”258

Explanation

Numerous randomised trials enrol participants over extended periods of time. If an intervention demonstrates exceptional efficacy, the study might require early termination on ethical grounds. To mitigate this concern, assessing results as data accumulates is advisable, ideally through an independent data monitoring committee (DMC), sometimes referred to as a data and safety monitoring board (DSMB).259 However, conducting multiple statistical evaluations on accruing data without proper adjustment may result in misleading conclusions. For instance, examining data from a trial at five interim analyses using a P value of 0.05 would elevate the overall false-positive rate closer to 19% rather than the expected 5%. Further to stopping early for efficacy, interim analyses can be used to evaluate (1) futility, to assess whether a trial is likely to meet its objectives; or (2) safety, to assess whether there is evidence for increased risk of harms (in the intervention group relative to the comparator group).259 Interim analyses can also be used to reassess the sample size, using updated information from interim trial data (eg, through an internal pilot), to ensure adequate power of the trial.

Various group sequential statistical approaches exist to adjust for multiple looks (ie, analyses) at the data, and these should be predetermined in the trial protocol (see item 27a of the SPIRIT 2025 statement78). Using these methods, data are compared at each interim analysis, where a P value below the specified critical value by the chosen group sequential method signifies statistical significance. Some researchers view group sequential methods as a tool for decision making, while others regard them as a definitive stopping point, intending to halt the trial if the observed P value falls below the critical threshold.

Authors should disclose whether they or the DMC/DSMB performed multiple looks at the data (interim analyses). If such multiple looks occurred, it is important to specify the frequency; the triggers prompting them; the statistical methods applied (including any formal stopping rules); and whether these procedures were planned and documented in the trial protocol before the trial commenced, before the DMC examined any interim data, or at a later stage. Authors should also report the time point at which any interim analyses where conducted (and by whom); and state who decided to continue, stop or modify the trial, and whether they were blinded to the treatment allocation. Unfortunately, the reporting of interim analyses and stopping rules is frequently inadequate in published trial reports,260 even in cases where trials indeed halted earlier than originally planned.

Examples

“Randomization was done using computer-generated random numbers (Stat Trek software) by trained staff at the Soltan Mirahmad Clinic (Kashan, Iran).”261

“The randomization was conducted by two independent researchers who were not involved in the study using a computer random sequence generator.”262

Explanation

Randomisation eliminates selection bias at trial entry and is the crucial component of high quality randomised trials (box 5).271 Successful randomisation hinges on two steps: generation of an unpredictable allocation sequence and concealment of this sequence from the investigators enrolling participants (item 18).268269

Who generated the random allocation sequence is important mainly for two reasons. Firstly, someone, or some group, should take responsibility for this critical trial function. Secondly, providing information on the generator might help readers to evaluate whether anyone had access to the allocation sequence during implementation. Investigators should strive for complete separation, independence, between the trial staff involved with generation of the allocation sequence and those staff who implement assignments (see explanation for item 19).

Participants should be assigned to comparison groups in the trial on the basis of a chance (random) process characterised by unpredictability (box 5). Successful randomisation in practice depends on two inter-related aspects: adequate generation of an unpredictable allocation sequence and concealment of that sequence until assignment occurs. A key issue is whether the sequence is known or predictable by the people involved in allocating participants to the comparison groups. The treatment allocation system should thus be set up so that the person enrolling participants does not know in advance which treatment the next person will receive, a process termed allocation concealment (item 18). Proper allocation concealment shields knowledge of forthcoming assignments, whereas proper random sequences (item 17) prevent correct anticipation of future assignments based on knowledge of past assignments (box 5).

Authors should provide sufficient information such that the reader can assess the methods used to generate the random allocation sequence and the likelihood of bias in group assignment. Any software used for random sequence generation should also be reported. It is important that information on the process of randomisation is included in the body of the main article and not as a separate supplementary file, where it can be missed by the reader.

The term “random” has a precise technical meaning. With random allocation, each participant has a known probability of receiving each intervention before one is assigned, and the assigned intervention is determined by a chance process and cannot be predicted. However, “random” is sometimes used inappropriately in the literature to describe trials in which non-random, “deterministic” allocation methods were used, such as alternation, hospital numbers, or date of birth. When investigators use such non-random methods, they should describe them precisely and should not use the term “random” or any variation of it. Even the term “quasi-random” is unacceptable for describing such trials. Trials based on non-random methods generally yield biased results4268272273274275276277; bias presumably arises from the inability to adequately conceal these more predictable, non-random sequence generation systems.

Many methods of sequence generation are adequate. However, readers cannot judge adequacy from such terms as “random allocation,” “randomisation,” or “random” without further elaboration. Authors should specify the method of sequence generation, such as a random-number table or a computerised random number generator. The sequence may be generated by the process of minimisation, a non-random but generally acceptable method.

In some trials, participants are intentionally allocated in unequal numbers to each intervention: for example, to gain more experience with a new procedure or to limit costs of the trial. In such cases, authors should report the randomisation ratio (eg, 2:1, or two treatment participants per control participant; item 9).

In a representative sample of PubMed indexed trials in 2000, only 21% reported an adequate approach to random sequence generation 220; this increased to 34% for a similar cohort of PubMed indexed trials in 2006.221 Two more recent studies showed further small increases to about 40%,278279 but another reported a stubbornly similar level of 32%.277 When authors report an adequate approach to random sequence generation, in over 90% of cases they report using a random number generator on a computer or a random number table.221279

Examples

“Treatment assignment was generated using a simple randomization scheme . . . given the open-label nature of the intervention to limit the potential bias due to predictable treatment assignment.”280

“Randomization (1:1) was performed by an independent researcher using computer generated random table numbers, with a block size of 20 and stratified for the indication of the IUI [intrauterine insemination] (mild male factor or unexplained subfertility).”281

“Participants were randomized at an individual-level (1:1 ratio) and were stratified by recruitment location (VU [Vrije University] and UvA [University of Amsterdam]). Block randomization was applied with randomly varied block sizes (6–12 allocations per block).”262

“Randomization was stratified by treatment centre, clinical severity (<4 vs >4 on a Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) pain subscale standardized to range from 0 to 10), and by whether patients had previously received TENS [transcutaneous electrical nerve stimulation] with randomly varied block sizes of 2, 4, and 6.”282

“Randomization sequence was created using Stata 9.0 (StataCorp., College Station, TX) statistical software and was stratified by center with a 1:1 allocation using random block sizes of 2,4, and 6.”283

Explanation

In trials of several hundred participants or more, simple randomisation can usually be trusted to generate similar numbers in the two trial groups 284 and to generate groups that are roughly comparable in terms of known and unknown prognostic variables.285 For smaller trials of fewer than around 200 participants,286 which are common, some form of restricted randomisation procedure to help achieve balance between groups in size or characteristics may be useful (box 6). However, larger trials of greater than approximately 200 participants may also benefit from registration. For example, they may stop before reaching their target size, they may need more power at interim analyses, or they may benefit from stratification with restriction.

It is important to indicate whether no restriction was used by stating such or by stating that simple randomisation was done. Otherwise, the methods used to restrict the randomisation, along with the method used for random selection, should be specified. For blocked randomisation, authors should provide details on how the blocks were generated (eg, by using a permuted block design with a computer random number generator), the block size or sizes, and whether the block size was fixed or randomly varied. If the trialists became aware of the block size(s), that information should also be reported as such knowledge could lead to them correctly deciphering future treatment assignments. Authors should specify whether stratification was used and, if so, which factors (eg, recruitment site, sex, disease stage) were involved; the categorisation cut-off thresholds within stratums; and the method used for restriction. Although stratification is a useful technique, especially for smaller trials, it can be complicated to implement and may not perform as well as expected if many stratifying factors are used. If minimisation (box 6) was used, it should be explicitly identified, as should the variables incorporated into the scheme; whether a random element was used should also be stated.

With blocking, although the order of interventions varies randomly within each block, individuals running the trial could deduce some of the future treatment allocations if they discovered the block size.288 Discovering block sizes is much more likely in unblinded trials, where treatment allocations become known after assignment (box 6). Certain techniques, such as large block sizes and randomly varying block sizes, can help prevent the deciphering of future treatment allocations. Unfortunately, particularly with unblinded trials, a review “found that very few trials used techniques that would eliminate the risk of selection bias,” and that “These findings indicate that a substantial proportion of unblinded trials are at risk of selection bias.”292 Indeed, in a recent study of 179 open, unblinded randomised trials, small block sizes were associated with subversion.293

Only 9% of 206 reports of trials in specialty journals269 and 39% of 80 trials in general medical journals reported use of stratification.294 In each case, only about half of the reports mentioned the use of restricted randomisation. Those studies and that of Adetugbo and Williams295 found that the sizes of the treatment groups in many trials were very often the same or quite similar, yet blocking or stratification had not been mentioned. One of a few possible causes of this close balance in numbers is under-reporting of the use of restricted randomisation, although non-random manipulation of treatment assignments is also suspected. A more recent study of 298 reports of trials in general medical journals found 69% reported the use of a stratified block method.296

Examples

“Participants were centrally assigned to randomised study treatment using an interactive web response system (IWRS) . . . Block randomisation schedules were computer generated by a vendor with a block size of 6 in a randomisation ratio of 2:1 and distributed to the IWRS vendor (endpointClinical) for participant randomisation.”297

“For allocation concealment, numbered containers were used. The interventions were sealed in sequentially numbered identical opaque containers according to the allocation sequence.”298

“Furthermore, we employed syringes sequentially numbered and packaged in opaque and sealed containers. Specifically, syringes containing esmolol or placebo were centrally prepared, pre-coded based on the randomization list, and sent sequentially to the operating room immediately before administration.”299

“Allocation was concealed using sequentially numbered, opaque, sealed envelopes (SNOSE) prior to making the incision.”300

“Allocation concealment was done using sequentially numbered, sealed, opaque packages.”261

“The allocation sequence was concealed from the researcher (JR) enrolling and assessing participants in sequentially numbered, opaque, sealed and stapled envelopes. Aluminium foil inside the envelope was used to render the envelope impermeable to intense light. To prevent subversion of the allocation sequence, the name and date of birth of the participant was written on the envelope and a video tape made of the sealed envelope with participant details visible. Carbon paper inside the envelope transferred the information onto the allocation card inside the envelope and a second researcher (CC) later viewed video tapes to ensure envelopes were still sealed when participants’ names were written on them. Corresponding envelopes were opened only after the enrolled participants completed all baseline assessments and it was time to allocate the intervention.”301

Explanation

Item 17 discussed generation of an unpredictable sequence of assignments. Of considerable importance is how this sequence is applied when participants are enrolled into the trial (box 5). A generated allocation sequence should be implemented by using allocation concealment,269 a critical mechanism that prevents foreknowledge of treatment assignment and thus shields those who enrol participants from being influenced by this knowledge. The decision to accept or reject a participant should be made, and informed consent should be obtained from the participant, in ignorance of the next assignment in the sequence.302 In summary, adequate allocation concealment safeguards knowledge of forthcoming assignments, whereas proper random sequences (item 17) prevent correct anticipation of future assignments based on knowledge of past assignments.

Allocation concealment should not be confused with blinding (item 20). Allocation concealment seeks to prevent selection bias (box 5), protects the assignment sequence before and until allocation, and can always be successfully implemented.268 In contrast, blinding seeks to prevent ascertainment bias, protects the sequence after allocation, and cannot always be implemented.269 Without adequate allocation concealment, however, even random, unpredictable assignment sequences can be subverted.268303

Centralised or third party assignment is especially desirable. Many good allocation concealment mechanisms incorporate external involvement. Use of a pharmacy or central computer or telephone randomisation system are common techniques. Automated assignment systems are likely to become more common.221 When external involvement is not feasible, an excellent method of allocation concealment is the use of numbered containers. The interventions (often medicines) are sealed in sequentially numbered identical containers according to the allocation sequence.304 Enclosing assignments in sequentially numbered, opaque, sealed envelopes can be a good allocation concealment mechanism if it is developed and monitored diligently.288305 This method can be corrupted, however, particularly if it is poorly executed. Investigators should ensure that the envelopes are opaque when held to the light, and are opened sequentially and only after the participant’s name and other details are written on the appropriate sequentially numbered sealed envelope.288305306

A number of methodological studies provide empirical evidence to support these precautions.4268275276277307308309 Trials in which the allocation sequence had been inadequately or unclearly concealed yielded larger estimates of treatment effects than did trials in which authors reported adequate allocation concealment. These findings provide strong empirical evidence that inadequate allocation concealment contributes to bias in estimating treatment effects.

Despite the importance of the mechanism of allocation concealment, published reports frequently omit such details. Among older studies, the mechanism used to allocate interventions was omitted in reports of 89% of trials on rheumatoid arthritis,310 48% of trials in obstetrics and gynaecology journals,269 and 44% of trials in general medical journals.294 In a more broadly representative sample of all PubMed indexed randomised trials, only 18% reported any allocation concealment mechanism and some of those reported mechanisms were inadequate.220 At the same time, some trials where there is no reporting of allocation concealment may have been properly concealed, as demonstrated by inspection of their protocols.311

Newer studies further illuminate poor reporting of allocation concealment. Unclear reporting (ie, the authors did not provide sufficient information in the paper to allow judgment to be made on the adequacy of method of allocation concealment) was found in 78%312 and 85%277 of trials. Moreover, those two studies determined that only 27%312 and 14%277 used an adequate allocation concealment mechanism, while another found a similar level of 12%.278 A review of trials in journals of traditional Chinese medicine found that only 7% used adequate allocation concealment.313

Fortunately, reporting and conduct may be improving in recent years, for example, after the CONSORT 2010 guidelines were published.314 Another study found that reporting on allocation concealment and sequence generation was much better in journals that endorsed the CONSORT 2010 guidelines than in non-endorsing journals.305 Moreover, that study found that 57% of trials in the sample had used an adequate allocation concealment mechanism.305 However, other empirical studies show only modest improvements, for example, an evaluation of over 176 000 trials315 found that allocation concealment reporting increased from 5.1% in 1966-90 to 19.3% in 2010-18. While any improvement is encouraging, more efforts to improve conduct and reporting remain necessary.

Examples

“Sequential randomisation codes were computer generated using permuted blocks . . . An independent study statistician generated the randomisation codes . . . Good Clinical Practice (GCP) trained research nurses or First Nations health practitioners allocate the study medicine to each mother–infant pair by selecting the next sequentially labelled (prerandomised) study medication from the appropriate stratification group. The allocation sequence number is recorded by the research team on the data collection form (DCF), the database and in the participant’s medical record.”316

“LabCorp Drug Development (a subcontractor to the IWRS [interactive web response system] vendor) generated the live randomisation schedules. Site personnel enrolled participants in the IWRS. The IWRS assigned participants to the trial groups per live randomisation schedules. LabCorp Drug Development didn’t have any involvement in the rest of the trial . . . Site personnel were involved in participant care and performing trial procedures throughout the trial; however, they were masked to treatment assignment.”297

“All participants were screened by a masked trial doctor who also obtained informed consent . . . The masked trial doctor then provided all participants with guideline-recommended care and a prescription for the trial medicine kit . . . The randomisation sequence was created using randomly permuted blocks by an independent statistician (who had no involvement in the rest of the trial).”317

“The details of sequence generation and group allocation were unavailable to research team members bar one unblinded research assistant. The unblinded research team member that created the randomisation sequence had no contact with participants and was not involved with data collection or analysis.”318

“Block randomisation was by a computer generated random number list prepared by an investigator with no clinical involvement in the trial . . . After the research nurse had obtained the patient’s consent, she telephoned a contact who was independent of the recruitment process for allocation consignment.”319

Explanation

As noted in item 18, concealment of the allocated intervention at the time of enrolment is especially important. Thus, in addition to knowing the methods used, it is also important to understand how the random sequence was implemented; specifically, whether the personnel who enrolled and those who assigned participants to the interventions had access to the random allocation sequence.

In practice, the process of randomising participants into a trial has three different steps: sequence generation, allocation concealment mechanism, and implementation (table 6). Although the same individuals may carry out more than one process under each heading, investigators should strive for complete separation of the people involved with generation and allocation concealment from the people who implement assignments. Thus, if someone is involved in the sequence generation or allocation concealment steps, ideally, they should not be involved in the implementation step. When this separation is not possible, importantly the investigators should ensure that the assignment schedule is unpredictable and locked away from even the person who generated it.

Even with flawless sequence generation and allocation concealment, failure to separate creation and concealment of the allocation sequence from assignment to trial group may introduce bias. For example, the person who was enrolling and assigning participants and who also generated an allocation sequence would likely have access to the sequence list and could consult it when interviewing potential participants for a trial. Thus, that person could bias the enrolment or assignment process, regardless of the unpredictability of the assignment sequence and the impenetrability of the allocation concealment mechanism. Investigators must therefore ensure that the assignment schedule is unpredictable and locked away (eg, in a safe deposit box in a building inaccessible to the enrolment location) from even the person who generated it. In that instance, the report of the trial should specify where the investigators stored the allocation list.

Thus, for full assessment of randomisation in a trial report, authors should confirm that personnel who enrolled and those who assigned participants to the interventions did not have access to the random allocation sequence. At minimum, authors should confirm complete separation of the people involved with generation and allocation concealment from the people involved in the implementation of assignments. If complete separation did not occur, then authors should describe how the people involved in the implementation were prevented from accessing the sequence (eg, specifying that the allocation sequence was locked in a secure location).

Sometimes those who enrol and those who assign are different people, but frequently the same individuals do both. These individuals may be termed differently by authors (eg, recruiters), but the functions remain the same.

Full assessment of implementation is not always possible from trial reports. In a sample of 199 medical journals, 63% of trial reports did not provide sufficient information to assess whether the person who generated the allocation sequence was not also the person who allocated participants to treatment groups.306 Only 31% of trial reports provided sufficient details on who recruited participants and who generated the allocation sequence.306

Examples

“Whereas patients and physicians allocated to the intervention group were aware of the allocated arm, outcome assessors and data analysts were kept blinded to the allocation.”320

“Blinding and equipoise were strictly maintained by emphasizing to intervention staff and participants that each diet adheres to healthy principles, and each is advocated by certain experts to be superior for long-term weight-loss. Except for the interventionists (dieticians and behavioural psychologists), investigators and staff were kept blind to diet assignment of the participants. The trial adhered to established procedures to maintain separation between staff that take outcome measurements and staff that deliver the intervention. Staff members who obtained outcome measurements were not informed of the diet group assignment. Intervention staff, dieticians and behavioural psychologists who delivered the intervention did not take outcome measurements. All investigators, staff, and participants were kept masked to outcome measurements and trial results.”321

“This was a double-blind study with limited access to the randomisation code . . . The treatment each patient received was not disclosed to the investigator, study site staff, patient, sponsor personnel involved with the conduct of the study (with the exception of the clinical supply staff and designated safety staff), or study vendors.”297

“Physicians, patients, nurses responsible for referring the patients, the statistician, also the investigators who rated the patients and administered the drugs, were all blinded to the allocation.”261

Explanation

The term “blinding” (masking) refers to withholding information about the assigned interventions from people involved in the trial who may potentially be influenced by this knowledge. Blinding is an important safeguard against bias, particularly when assessing subjective outcomes.308

Benjamin Franklin has been credited as being the first to use blinding in a scientific experiment.322 He blindfolded participants so they would not know when he was applying mesmerism (a popular healing technique of the 18th century) and in so doing demonstrated that mesmerism was a sham. Since then, the scientific community has widely recognised the power of blinding to reduce bias, and it has remained a commonly used strategy in scientific experiments.

Box 7 on blinding terminology defines the groups of individuals (ie, participants, healthcare providers, data collectors, outcome assessors, and data analysts) that can potentially introduce bias into a trial through knowledge of the treatment assignments. Participants may respond differently if they are aware of their treatment assignment (eg, respond more favourably when they receive the new treatment).308 Lack of blinding may also influence adherence with the intervention, use of co-interventions, and risk of dropping out of the trial.

Unblinded healthcare providers may introduce similar biases; and unblinded data collectors may differentially assess outcomes (eg, frequency or timing), repeat measurements of abnormal findings, or provide encouragement during performance testing. Unblinded outcome assessors may differentially assess subjective outcomes, and unblinded data analysts may introduce bias through the choice of analytical strategies, such as the selection of favourable time points or outcomes and by decisions to remove patients from the analyses. These biases have been well documented.32308325327328329

Blinding, unlike allocation concealment (item 18), may not always be appropriate or possible. In pragmatic trials (trials that try to make the experience as close as real life so as to understand real world effectiveness), blinding of participants and healthcare providers would decrease the pragmatism of the trials, since patients in real life are not blinded.330 An example where blinding is impossible is a trial comparing levels of pain associated with sampling blood from the ear or thumb.331 However, in randomised trials for which blinding is possible, lack of blinding has usually been associated with empirical evidence of exaggeration in treatment effect estimates.276308332333334335336 Blinding is particularly important when outcome measures involve some subjectivity, such as assessment of pain. Yet, blinding may not be as important in certain fields or with certain outcomes. For example, blinding of data collectors and outcome assessors is unlikely to matter for objective outcomes, such as death from any cause. Indeed, some methodological investigations have not found that lack of blinding is associated with empirical evidence of bias in treatment effect estimates.337338339340341342343 Even then, however, lack of participant or healthcare provider blinding can lead to other problems, such as differential attrition.344 In certain trials, especially surgical trials, blinding of participants and healthcare providers is often difficult or impossible, but blinding of data collectors and outcome assessors for both benefits and harms is often achievable and recommended. For example, lesions can be photographed before and after treatment and assessed by an external observer.345 Regardless of whether blinding is possible, authors can and should always state who was blinded (ie, participants, healthcare providers, data collectors, data analysts, and/or outcome assessors).346347

However, authors frequently do not report whether blinding was used.348349 For example, reports of 51% of 506 trials in cystic fibrosis,350 33% of 196 trials in rheumatoid arthritis,310 and 38% of 68 trials in dermatology295 did not state whether blinding was used. Similarly, a more recent review found that the reports of 38% of 622 trials in high impact anaesthesiology journals did not explicitly describe the trial as blinded or non-blinded.351 Moreover, when describing some form of blinding, the most used term was the ambiguous “double blind.”351 Authors should explicitly state who was blinded, but only 14% of 622 trials explicitly reported whether the three key groups of individuals—that is, the participants, healthcare providers, and data collectors—were blinded or not.351 The rate did improve from 10% to 26% over the years of that review, but more improvement is needed. Until authors of trials improve their reporting of blinding, readers will have difficulty in judging its adequacy.

The term “masking” is sometimes used in preference to “blinding” to avoid confusion with the medical condition of being without sight. However, “blinding” in its methodological sense appears to be more universally understood worldwide and to be generally preferred for reporting clinical trials.344345352

Examples

“Jamieson Laboratories Inc. provided 500-mg immediate release niacin in a white, oblong, bisect caplet. We independently confirmed caplet content using high performance liquid chromatography . . . The placebo was matched to the study drug for taste, color, and size, and contained microcrystalline cellulose, silicon dioxide, dicalcium phosphate, magnesium stearate, and stearic acid.”353

“Placebo tablets were identical to NAC [N‐acetylcysteine] tablets in color, shape, size, and odor. They were all kept in identical containers and were administered by an investigational drug pharmacist.”261

“The study treatment and placebo tablets and bottles were identical in physical appearance . . . The IWRS [interactive web response system] housed treatment codes and bottle numbers for study treatment. In case of an emergency, the investigator had the sole responsibility for determining if unmasking of a participant’s treatment assignment was warranted to provide appropriate medical care. Participant safety was always the first consideration in making such a determination. The IWRS was programmed with blind-breaking instructions to guide the investigator on how to obtain treatment assignment in the event of an emergency unmasking. The investigator was requested to contact the medical monitor promptly in case of any treatment unmasking. If a participant’s treatment assignment was unmasked, the sponsor was to be notified within 24 h after unmasking. The date and reason for the unmasking were recorded in the source documentation and electronic case report form, as applicable. Investigators broke the masking for four participants: one in ELEVATE UC 12 (on etrasimod) and three in ELEVATE UC 52 (on etrasimod).”297

Explanation

Blinding of participants, healthcare providers, data collectors, and outcome assessors in a trial requires adequate procedures to both achieve and maintain blinding.346347 Just as we seek evidence of adequate allocation concealment to assure us that assignment was truly random, we seek evidence on the method of blinding.

If researchers contend that the trial investigators, participants, and assessors were blinded, then they should provide information about the mechanism used to establish blinding (eg, placebo identical to the experimental intervention, sham intervention, sham surgery).346347 They should describe the similarity of treatment characteristics (eg, route of administration, appearance, smell, taste) and where relevant methods used to mask some characteristics of the treatments (eg, use of special flavours to mask a distinctive taste, opaque coverage to conceal intravenous treatments with different appearances, double-dummy procedures).346347

Blinding can be difficult to maintain over time because of dosage adaptation over time or the occurrence of specific side effects. Specific procedures to maintain blinding can be implemented (eg, centralised assessment of side effects, centralised adapted dosage, or provision of sham results of complementary investigations).

Even if blinding of participants and healthcare providers is not possible, blinding data collectors and outcome assessors could still be implemented to limit ascertainment bias. This could be achieved, for example, through centralised assessment of complementary investigation (eg, anonymised radiography), physician mediated data (eg, video, photography, audiotape), and clinical events (eg, adjudication of clinical events from extract of the case report form).

Details of how blinding was achieved are important because slight, but discernible, differences between interventions can lead to large problems in bias. Notably, inadequate matching related to discernible differences in colour and taste seem particularly problematic.354 It is important that authors report any known compromises in blinding. For example, authors should report if it was necessary to unblind any participants at any point during the conduct of the trial. Moreover, authors should report the risk of unblinding but, unfortunately, such reporting is rare. In a random sample of 300 publications describing blinded randomised clinical trials indexed in PubMed, only 8% reported on risk of unblinding.355 It is also important to report any procedures, pretrial or concurrent, that are intended to reduce or evaluate risk of compromised blinding.354355356 Indeed, some pretrial assessments of unblinding may be helpful in reducing the risk of unblinding in the eventual randomised trial. Thus, authors of randomised trial articles should report procedures to avoid, document, and address cases of overt unblinding.354355

Where appropriate, authors should also describe any procedures used for emergency unblinding (ie, disclosing the assigned intervention of a trial participant for specific reasons such as harms). They should indicate whether they used fixed code to indicate group assignment (eg, A=group 1; B=group 2) or a unique code for each participant. Use of a fixed code will increase the risk of unblinding because unblinding a participant could result in unblinding several or all trial participants.6286357

Some people have advocated testing for blinding by asking participants or healthcare providers at the end of a trial whether they think the participant received the experimental or control intervention.358 Because participants and healthcare providers will frequently know whether the participant has experienced the primary outcome, this makes it difficult to determine whether their responses reflect failure of blinding or accurate assumptions about the efficacy of the intervention.359 Thus, given the uncertainty this type of information provides, the usefulness of tests of blinding has been questioned.354355360 Testing for blinding was not an included item in CONSORT 2010, and still is not in CONSORT 2025. Nevertheless, if investigators decide to conduct tests of blinding, we encourage them to completely report their findings with appropriate limitations.356

Examples

“The primary outcome was analysed using a mixed effects log-binomial model to generate an adjusted risk ratio (RR) and an adjusted risk difference (using an identity link function), including centre as a random effect. Statistical significance of the treatment group parameter was determined (p value generated) through examination of the associated χ2 statistic (obtained from the log-binomial model which produced the RR). Binary secondary outcomes were analysed as per the primary outcome. Time to hCG [human chorionic gonadotrophin] resolution was considered in a competing risk framework to account for participants who had surgical intervention for their ectopic pregnancy. A cumulative incidence function was used to estimate the probability of occurrence (hCG resolution) over time. A Fine and Gray model was then used to estimate a subdistribution adjusted hazard ratio (HR) directly from the cumulative incidence function. In addition, a further Cox proportional hazard model was fitted and applied to the cause-specific (non-surgical resolution) hazard function and used to generate an adjusted HR. Return to menses was analysed using a Cox regression model. Number of hospital visits associated with treatment was analysed using a Poisson regression model, including centre as a random effect to generate an adjusted incidence ratio.”255

“For the primary continuous outcome and secondary outcomes, linear mixed-effect models were used, with outcome measurement (at the two follow-up timepoints) as the dependent variable. The models included fixed effects for timepoint, treatment, timepoint by treatment interactions, the baseline measure of the outcome, and therapist, assuming a linear relationship between baseline and outcome. The dichotomous outcome of recovery in the delusion was analysed using a logistic mixed-effect model. Persecutory delusion conviction was analysed as a continuous and also as a dichotomous (recovery) variable. The models included a random intercept for participant, an unstructured correlation matrix for the residuals, and were fitted using restricted maximum likelihood estimation . . . For each outcome and timepoint, we report the treatment effect estimate as the adjusted mean difference between groups, its SE [standard error], 95% CIs [confidence intervals], and p value. In addition, we report estimates for Cohen’s d effect sizes as the adjusted mean difference of the outcome (between the groups) divided by the sample SD [standard deviation] of the outcome at baseline.”361

“Analyses followed a prespecified statistical analysis plan. The primary outcome (ODQ [Oswestry Disability Questionnaire] score at 18 weeks after randomisation) was compared between groups with a linear regression model, adjusted for baseline ODQ, with centre as a random effect. ODQ score, visual analogue scores (VAS) for back pain, VAS for leg pain, MRM [modified Roland-Morris] outcome score, and COMI [Core Outcome Measures Index] score at all follow-up visits were analysed with a repeated measures mixed-effects model adjusting for baseline outcome measure, treatment group, time (as a continuous variable), and a time-treatment arm interaction (if significant). Centre and participant were random effects in the repeated measures models. A second model adjusted for other prespecified variables, age, sex, duration of symptoms, body-mass index, and size of disc prolapse (as a percentage of the diameter of the spinal canal, categorised as <25%, 25–50%, or >50%).”362

“We analysed the primary outcome (between-group difference in the SPPB [short physical performance battery] at 12 months) using linear mixed models, adjusted for baseline measurements, minimisation variables (age, sex and CKD [chronic kidney disease] category) and a random effect variable for recruitment site. We analysed secondary outcomes using repeated measures mixed models, including all participants and including data from all available timepoints. Models were adjusted for baseline values and the minimisation variables. We conducted time-to-event analyses (time to death, time to commencing renal replacement therapy) using Cox proportional hazards models adjusted for minimisation variables. All participants were included in these analyses, with participants censored at the point of dropout or truncation of follow-up for those not reaching the analysis endpoint before 24 months. For all analyses, we took a two-sided p value of < 0.05 as significant with no adjustment for multiple testing.”363

Explanation

Various methods can be used to analyse data, and it is crucial to ensure that the chosen approach is suitable for the specific context. Specifying the statistical procedures and software used for each analysis is essential, and additional clarification may be required in the results section of the report. Authors should describe the statistical methods insufficient detail to allow a knowledgeable reader with access to the original data to verify the reported results, as emphasised by the ICMJE (https://www.icmje.org/). It is also important to elaborate on specific aspects of the statistical analysis, such as the intention-to-treat approach.

Details of all statistical analyses are frequently prespecified in a statistical analysis plan, a document that accompanies the trial protocol. In the report of the trial results, authors should detail and justify any deviation from the statistical analysis plan or from the protocol if no statistical analysis plan was developed. They should clarify which analyses were prespecified and which were post hoc.

Most analysis approaches provide an estimate of the treatment effect, representing the difference in outcomes between comparison groups, and authors should also indicate the effect measure (eg, absolute risk) considered. Authors should accompany this with a CI for the estimated effect, delineating a central range of uncertainty regarding the actual treatment effect. The CI may be interpreted as the range of values for the treatment effect that is compatible with the observed data. Typically, a 95% CI is presented, signifying the range anticipated to encompass the true value in 95 of 100 similar studies.

Study findings can also be assessed in terms of their statistical significance. The P value represents the probability that the observed data (or a more extreme result) could have arisen by chance when the interventions did not truly differ. The statistical significance level that will be used should be reported. In the results section, actual P values (for example, P=0.001) are strongly preferable to imprecise threshold reports such as P<0.05.364365

Some trials may use bayesian methods.366367368369 In this case, the choices of priors, computational decisions, and any modelling methods used should be described. Most bayesian trials so far have been for early phases of drug development, but this approach can be applicable to any phase. Typically, results are presented as treatment effects along with credible intervals.

Where an analysis lacks statistical power (eg, harms outcomes), authors may prefer descriptive approaches over formal statistical analysis.

While the necessity for covariate adjustments is generally reduced in randomised trials compared with epidemiological studies, considering an adjusted analysis can have value in terms of increased power and precision, particularly if there is an indication that one or more variables may have prognostic value.370 It is preferable for adjusted analyses to be explicitly outlined in the study protocol (item 3). For instance, it is often advisable to make adjustments for stratification variables,371 in keeping with the principle that the analysis strategy should align with the study design. In the context of randomised trials, the decision to make adjustments should not be based on whether there are baseline covariates that are statistically significantly different between randomised groups. The testing of baseline imbalance in covariates should be avoided,370 as if randomisation is properly conducted, then by definition, any differences in baseline covariates between treatment arms are random. The rationale for any adjusted analyses and the statistical methods used should be specified, along with clarifying the choice of covariates that were adjusted for, indicating how continuous variables were handled (eg, linear, modelled with splines),372 and specifying whether the analysis was planned or post hoc. Reviews of published studies show that reporting of adjusted analyses is inadequate with regard to all of these aspects.373374

Multiplicity issues are prevalent in trials and merit special consideration, especially in cases involving multiple primary outcomes, multiple time points stemming from repeated assessments of an outcome, multiple planned analyses for an outcome (such as interim or subgroup analyses (item 21d)), or analyses of numerous secondary outcomes (see CONSORT outcomes extension for more details).21 Any methods used to mitigate or account for multiplicity should be described. If no methods have been used to account for multiplicity (eg, not applicable, or not considered), then this should also be reported, particularly when a large number of analyses has been carried out.

Examples

“The primary statistical analyses were performed according to the treatment to which the participants were randomly assigned. The analyses of the efficacy and safety outcomes (other than adverse events) included all available data from all randomized participants who contributed at least 1 value after baseline for the outcome of interest. The data that were obtained after a participant enrolled in another trial of an investigational treatment were excluded from the analyses. However, the participant was included in the analyses if that participant contributed at least 1 value after baseline for the outcome of interest prior to enrolling in another trial.”375

“Efficacy outcomes were assessed using intention-to-treat analysis (ie, the full set of all randomly assigned patients). Safety outcomes were assessed using the safety analysis set of all randomly allocated patients exposed to at least one dose of randomised intervention.”376

“Efficacy analyses and other exploratory analyses were performed in the full analysis set (defined as all patients randomly assigned to the study, including those who did not receive a dose of study treatment). Safety analyses were performed in the safety analysis set (defined as patients who received at least one dose of study treatment). The per protocol set was defined as all patients in the full analysis set who complied with the protocol in terms of exposure to study treatment, availability of tumour assessments, and absence of major protocol deviations likely to affect efficacy outcomes. Sensitivity analyses of the primary endpoint were performed on the per protocol analysis set.”377

“The primary analysis population was defined as all participants who completed baseline and 36-week assessments. The primary analysis of the primary outcome, AMCA [amended motor club assessment] score at 36 weeks, followed a modified intention-to-treat approach, regardless of compliance to the intervention, but did exclude patients who were deemed ineligible after randomisation, those who withdrew from the trial and were unwilling for their previously collected data to be used, or those who did not provide baseline and week 36 measurements.”378

Explanation

A key strength of a randomised trial design is the avoidance of bias when randomly allocating trial participants to interventions. To preserve the benefits of randomisation, all randomised participants are included in the analysis and retained in the group to which they were allocated. Meeting these two conditions defines an intention-to-treat analysis—which is widely recommended as the preferred analysis strategy.379380381 However, strict adherence to an intention-to-treat analysis is often difficult to achieve owing to missing outcomes for some trial participants (item 21c) or non-adherence to the trial intervention protocol. While imputation of missing outcomes would allow an intention-to-treat analysis, it does not guarantee an avoidance of bias except under strong assumptions about the missing data which may be unknown.

Various strategies for performing intention-to-treat analyses in the presence of missing outcome data are available.382 When the number of missing outcomes is not large, the analysis population could be all randomised participants with outcome observed (known as an “available case” population) under a plausible missing data mechanism, and sensitivity analyses could be performed exploring departures from this assumption (thereby using all randomised participants at least in sensitivity analyses).382 Concerns may arise when the frequency or the causes of dropping out differ between the intervention groups. Striving for intention-to-treat analysis by imputing values for participants with missing outcomes may lead to use of inadequate methods such as last observation carried forward.382383384385

Regardless of whether all randomised participants (completely observed outcomes or imputed outcomes) or a subset of randomised participants with observed outcomes are included in the primary analysis, the analysis population should be described. Authors often describe performing analyses on a “modified intention-to-treat” population to cover departures from a strict intention-to-treat that excludes participants who did not adequately adhere to the protocol such that they did not receive some minimum amount of the intervention—in such cases, what defines the minimum amount of the intervention should be explained (eg, those participants receiving at least one dose of the medication). It is also common to include analyses based on a per protocol population, which includes participants completing the study with no major protocol deviations. Excluding participants may compromise the randomisation and lead to biased estimates of treatment effects if appropriate methods are not used. Other analysis populations are possible (eg, a safety population), and their rationale and definition should be explained. Thus, authors should clearly define which participants are included in each analysis and in which intervention group and avoid terms such as “modified intention-to-treat” or “per protocol” analysis.

Examples

“Regarding the multiple imputation procedure, briefly, for each outcome, the analysis model used was a linear regression with treatment arm, baseline outcome, and ethnicity (randomization stratifier) as explanatory variables. The imputation models contained all the variables of the analysis model(s) as well as factors associated with missingness: age (identified empirically to predict missingness, P = .03) and adherence (number of doses taken of either vitamin D or placebo, P < .001).”386

“To consider the potential impact of missing data on trial conclusions, we used multiple imputation (data missing at random) and sensitivity analysis (data not missing at random). Multiple imputation by chained equations was performed using the “mi impute chained” command in Stata. We used a linear regression model to impute missing outcomes for the HOS ADL [activities of daily living subscale of the hip outcome score] at eight months post-randomisation. Variables in the imputation model included all covariates in the analysis model (baseline HOS ADL (continuous), age (continuous), and sex). In addition, we included other variables that were thought to be predictive of the outcome (lateral centre-edge angle, maximum α angle, Kellgren-Lawrence grade, and baseline HADS score). Imputations were run separately by treatment arm and based on a predictive mean matching approach, choosing at random one of the five HOS ADL values with the closest predicted scores. Missing data in the covariates that were included in the multiple imputation model were imputed simultaneously (multiple imputation by chained equation approach). Sensitivity analysis was performed using the “rctmiss” command in Stata, and we considered scenarios where participants with missing data in each arm were assumed to have outcomes that were up to 9 points worse than when data were missing at random.”242

“Analyses for the 2 primary outcomes compared each treatment with usual care using multiple imputation to handle missing data and a Bonferroni-corrected 2-tailed type I error of .025. We performed 20 imputations with a fully conditional specification using Proc MI in SAS. Imputation was performed with the following prespecified variables: age, study group, study site, clinic, sex, race and ethnicity, body mass index, exercise frequency at baseline, education, employment status, smoking status, other medical conditions at baseline, number of medications used for spine pain at baseline, duration of pain at baseline, number of previous pain episodes, STarT Back score, baseline ODI, baseline self-efficacy, baseline EQ-5D-5L, and scores for patient-reported outcomes at every follow-up point (ODI [Oswestry Disability Index], cost, Lorig et al self-efficacy scale, and EQ-5D-5L [EuroQol 5-dimensional 5-level questionnaire]). Each imputed data set was analyzed separately using Proc GENMOD in SAS (with an identity link and normally distributed errors for ODI and a log link and Poisson-distributed errors for spine-related spending).”387

“Missing peak V̇o2 [oxygen consumption] data at week 20, regardless of the type of intercurrent event, was imputed using multiple imputation methodology under the missing at random assumption for the primary analysis. Sensitivity analyses were performed by exploring a missing not at random assumption in the imputation of peak V̇o2. The imputation model used a regression multiple imputation, which includes treatment group, baseline respiratory exchange ratio, persistent atrial fibrillation (yes or no), age, sex, baseline peak V̇o2, baseline hemoglobin level, baseline estimated glomerular filtration rate, baseline body weight, baseline KCCQ [Kansas City cardiomyopathy questionnaire] total symptom score, baseline NYHA [New York Heart Association] class, and baseline average daily activity units (refers to 10 hours of wearing during the awake time for ≥7 days unless otherwise specified). Treatment group, persistent atrial fibrillation (yes or no), baseline NYHA class, and sex were treated as categorical variables. Fifty imputed data sets were generated. Each of the imputed data sets was analyzed using the analysis of covariance model of the primary analysis. Least square mean (LSM) treatment difference and the standard error were combined using Rubin’s rules to produce an LSM estimate of the treatment difference, its 95% CI [confidence interval], and P value for the test of null hypothesis of no treatment effect.”388

“Multiple imputation was preplanned for the primary outcome measure in the case of missing data; however, because there were no missing data relating to ventilator-free days, imputation was not required.”389

Explanation

Missing data are common when conducting medical research. Collecting data on all study participants can be challenging even in a trial that has mechanisms to maximise data capture. Missing values can occur in either the outcome or in one or more covariates, or usually both. There are many reasons why missing values occur in the outcome. Patients may stop participating in the trial, withdraw consent for further data collection, or fail to attend follow-up visits; all of which could be related to the treatment allocation, specific (prognostic) factors, or experiencing a specific health outcome.390 Missing values could also occur in baseline variables, such that all the necessary data needed to conduct the trial have been only partially recorded. Despite the ubiquity of missing data in medical research, the reporting of missing data and how they are handled in the analyses is poor.391392393394395396397398399

Many trialists exclude patients without an observed outcome. Once any randomised participants are excluded, the analysis is not strictly an intention-to-treat analysis. Most randomised trials have some missing observations. Trialists effectively must choose between omitting the participants without final outcome data, imputing their missing outcome data, or using model based approaches such as fitting a linear mixed model to repeated measures data.371 A complete case (or available case) analysis includes only those participants whose outcome is known. While a few missing outcomes will not cause a problem, many trials have more than 10% of randomised patients with missing outcomes.391392393394395396397398 This common situation will result in loss of power by reducing the sample size, and bias may well be introduced if being lost to follow-up is related to a participant’s response to treatment. There should be concern when the frequency or the causes of dropping out differ between the intervention groups.

Participants with missing outcomes can be included in the analysis if their outcomes are imputed (ie, their outcomes are estimated from other information that was collected) or if using a model based approach. Imputing the values of missing data allows the analysis to potentially conform to intention-to-treat analysis but requires strong assumptions, which may be hard to justify. Simple imputation methods are appealing, but their use may be inadvisable as they fail to account for uncertainty introduced by missing data and may lead to invalid inferences (eg, estimated standard errors for the treatment effect will be too small).400 For randomised trials with missing data within repeated measures data, model based approaches such as fitting a linear mixed model can be used to estimate the treatment effect at the final time point which is valid under a missing-at-random assumption. A model is fit at a (limited) number of time points following randomisation, by including fixed effects for time and randomised group and their interaction.371

Another approach that is sometimes used is known as “last observation carried forward,” in which missing final values of the outcome variable are replaced by the last known value before the participant was lost to follow-up. Although this method might appear appealing through its simplicity, the underlying assumption will rarely be valid, so the method may introduce bias, and makes no allowance for the uncertainty of imputation. The approach of last observation carried forward has been severely criticised.383384401 Sensitivity analyses should be reported to understand the extent to which the results of the trial depend on the missing data assumptions and subsequent analysis (item 21d).402 When the findings from the sensitivity analyses are consistent with the results from the primary analysis (eg, complete case for the primary analysis and multiple imputation for a sensitivity analysis), trialists can be reassured that the missing data assumptions and associated methods had little impact on the trial results.403

Regardless of what data are missing, how such data are to be analysed and reported needs to be carefully planned. Authors should provide a description on how missing data were handled in sufficient detail to allow for the analysis to be reproduced (in principle; box 8).

Examples

“We conducted prespecified sensitivity analyses to examine the effect of our assumption that participants who withdrew or were lost to follow-up returned to smoking: (1) a complete case analysis and (2) multiple imputation to impute missing smoking abstinence and reduction data. Multiple imputation was performed using the fully conditional specification approach with 5 imputed data sets and results combined using the Rubin rules (eMethods in Supplement 2). Other prespecified sensitivity analyses examined the effect of imbalances in baseline participant characteristics using multiple logistic regression models to estimate odds ratios and 95% CIs [confidence intervals] for point prevalence abstinence at 12 and 24 weeks, adjusting for characteristics for which the absolute value of the standardized difference was 0.1 or greater. We conducted additional post hoc analyses: (1) to examine potential clustering by site using generalized linear mixed models with a random effect for site to estimate odds ratios and 95% CIs for point prevalence abstinence at 12 and 24 weeks, and (2) to compare the baseline characteristics of participants with self-reported smoking data at 12 weeks (primary end point) with those of participants without self-reported smoking data. Statistical analyses were performed using SAS statistical software (version 9.4; SAS Institute).”406

“Several prespecified sensitivity analyses were done. First, assessment of the effect of missing data on the primary outcome was done using multiple imputation by chained equations method (MICE). This imputation model included all the variables in the primary ITT [intention to treat] analysis, secondary outcomes (from each timepoint), and baseline variables associated with the missingness of the primary outcome. 20 imputed datasets were generated and combined using Rubin’s rules, and the primary analysis model was then repeated using the imputed data. We specified a priori the following potential exploratory analyses to assess effect modification on the primary outcome: baseline hypertension, baseline MMSE [Mini-Mental State Examination], baseline age, time since Alzheimer’s disease diagnosis, baseline brain volume, and change in systolic blood pressure. A post-hoc analysis was also done to investigate for differences between aggregated and disaggregated MRI [magnetic resonance imaging] data (according to MRI scanner modality) for the primary outcomes.”407

“Four sensitivity analyses were done examining the primary outcome: restricted to women who had not received antibiotics in the 7 days before delivery, to examine whether any masking of a prophylactic effect was occurring by inclusion of pretreated women; excluding women prescribed antibiotics (other than the trial intervention) within the first 24 h after delivery, and who might therefore already have had an infection at the time of administration of the intervention; restricted to women whose primary outcome was obtained between weeks 6 and 10 after delivery to exclude any biases by over-reporting of outcomes from data returned at a later timepoint or under-reporting of outcomes in data returned at an earlier timepoint; and including centre as a random effect. No subgroup analyses were planned; however, we did a post-hoc subgroup analysis of the primary outcome according to mode of birth (forceps or vacuum extraction). More stringent 99% CIs [confidence intervals] are presented for the estimate of RR [risk ratio] for this post-hoc subgroup analysis.”408

“A prespecified subgroup analysis for the primary outcomes, testing for an interaction for baseline anxiety, depression, and opioid use, defined using their median values was completed. Prespecified sensitivity analyses for the primary outcome, excluding participants included in process evaluation interviews, adjusting for the imbalance of death, and split by baseline pain disorders were also completed. Because of the potential for type I error due to multiple comparisons, findings for analyses of secondary end points should be interpreted as exploratory. Statistical analyses were conducted using Stata version 16.1 (StataCorp).”409

Explanation

Sensitivity analyses can be important additional analyses to examine the robustness of the primary trial results under a range of assumptions about the data, methods, and models that differ from those of the primary analysis. When the findings from a sensitivity analysis are consistent with the primary trial findings, trialists can be confident that any assumptions in the primary analysis had little impact—strengthening the trial results. Morris and colleagues provide a principled approach to guide any sensitivity analyses by posing three questions to trialists: does the proposed sensitivity analysis address the same question as the primary analysis; is it possible for the proposed sensitivity analysis to return a different result to the primary analysis; and if the results do differ, is there any uncertainty as to which will be believed.402410

Subgroup analyses are another set of additional analyses that are widely carried out and reported.411412413414 Here, the focus is on those analyses that look for evidence of a difference in treatment effect in complementary subgroups (eg, older and younger participants), a comparison known as a test of interaction.415 Empirical analyses of subgroup difference claims for factors such as age, sex, race, ethnicity, and other factors show selective reporting, frequent lack of proper statistical support, and poor independent corroboration.416417418

A common but misleading approach is to compare P values for separate analyses of the treatment effect in each group. Categorising continuous variables to create subgroups is often done for simplicity and because it is perceived as easier to understand and communicate. Major limitations of the approach include the splitting of a continuous variable into discrete subgroups by arbitrarily chosen cut-off points that lack clinical or biological plausibility, which loses information, and thus reduces statistical power.419 Choosing cut-off points based on achieving statistical significance should be avoided. It is incorrect to infer a subgroup effect (interaction) from one significant (in one subgroup) and one non-significant P value (in another subgroup). The rationale for any subgroups should be outlined (including how they are defined), along with whether the subgroups were specified a priori in the protocol or statistical analysis plan or were done post hoc. Because of the high risk for spurious findings, subgroup analyses are often discouraged. Post hoc subgroup comparisons (analyses done after looking at the data) are especially likely not to be confirmed by further studies. Most of these analyses do not have substantial credibility.

An alternative and stronger approach, which avoids the need to specify cut-off points to assess the interaction between a continuous variable (eg, age) and treatment effect would be to fit a regression model, which can be presented graphically to examine how the estimated treatment effects varies with the level of the variable.420 These analyses are more complex, requiring model assumptions to capture the relationship (linear or non-linear) between the variable and the treatment effect. Authors should clearly describe the statistical methods used to explore the treatment-covariate interaction.