The guidelines here have been condensed from How To Report Statistics in Medicine; Annotated Guidelines for Authors, Editors, and Reviewers.1Authors should also consult other checklists for reporting specific research designs, such as the Consolidated Standards of Reporting Trials statement (CONSORT)2for reporting randomized controlled trials, the Transparent Reporting of Evaluations With Nonrandomized Designs (TREND) statement3for reporting nonrandomized trials, and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement4 for reporting of observational studies.
Describe the Background, Nature, Scope, and Importance of the Problem That Led to the Study
Many authors assume, incorrectly, that readers of their article will know why the research was done and, by extension, why the research is important. This assumption often results in a scientific report that begins with what was done, rather than why it was done.2,5
State the General Purpose of the Study and Identify Any Theoretical or Scientific Approach Taken To Address the Problem
The importance of stating the purpose of a research project is obvious, but such a statement is often missing from research articles.2,6
Tell Who Funded the Study and Describe the Role of the Funding Agency in the Conduct of the Study and the Publication of the Results
Descriptions of the role of the funding organization in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; and the preparation, review, or approval of the manuscript are now often required by medical journals.7
State the Specific Objectives of the Study, Including Any Formally Stated Research Questions or Hypotheses
Identify the Research Design and Explain Why This Design Was Chosen
At a minimum, report whether the study was a randomized controlled trial, a cohort study, a case-control study, or a cross-sectional study.
Identify the Institutional Review Board(s) That Approved the Protocol and Confirm That Written Informed Consent Was Obtained
Describe the Target Population of Interest
The description of the population of interest should include relevant demographic, diagnostic (including the stage of disease), prognostic, and comorbid factors.8–11 In addition, major subgroups of interest in the population should be identified.12
“Race” is a social construct that has no precise biological meaning.13–14 So, if race or ethnicity is reported, indicate the following: (1) the classifications used, (2) who classified the patients, (3) whether the classification options were defined by the patient or the researcher, and (4) why race or ethnicity was assessed for the study.7
Describe How Potential Participants Were Identified and Recruited and Report the Eligibility Criteria for Participating in the Study
It may be necessary to specify how the criteria were diagnosed or assessed in the patients.15 In particular, report whether the study was restricted to patients with isolated disease (more typical of explanatory studies) or included patients with comorbidities (more typical of pragmatic studies), and provide some assessment of the spectrum of disease included in the study.2,5
Report the Target Sample Size and How It Was Determined
In many studies,2–6 sample size is ideally determined with the aid of a statistical power calculation. Such a calculation is based on several factors but especially on the minimum difference to be detected and how willing investigators are to miss this difference.
Identify the Location(s) and Setting(s) Where the Data Were Collected
It may also be appropriate to describe the referral pattern of patients.16–17 Patients who are referred to, and who are admitted to, a tertiary care center may differ from those who present to a physician in private practice or to a community hospital, for example.
Tell How and When Patients Were Assigned to Groups
Random assignment and a procedure called “minimization” are acceptable and preferred for clinical studies. Nonrandomized assignment has substantial drawbacks and is not preferred.2,5–6
The purpose of random assignment is to prevent selection bias, or systematic variation in the assignment of patients, by introducing chance, or random variation, into the group assignment process. Simple random assignment (as opposed to random assignment using blocking and stratification) does not ensure that the experimental and control groups are of equal size, or of similar composition. It does ensure that any differences in size or composition are the result of chance and not bias, however.
In observational studies, participants are assigned to groups on the basis of their exposure or disease status. Thus, the case definition and the procedures for assessing exposure and diagnosis must be carefully defined and rigorously applied.
Describe the Intervention or Exposure of Interest
In addition to a complete description of the intervention,2,5 the indications for initiating, modifying, and discontinuing it may need to be described,18as well as the details of diagnosis and management.19
Identify the Outcomes Assessed, Explain How Each Was Quantified, and Indicate Whether the Measures Were Validated
The term operational definition describes a variable in quantifiable or measurable terms. Using standard, established definitions and measures makes comparing results across studies easier.20–21
Outcomes used in lieu of direct clinical end points that measure how a patient “feels, functions, or survives” are termed surrogate end points.22–23 They are usually laboratory measurements, such as CD4 cell counts (a surrogate end point for AIDS). However, the validity of surrogate end points is rarely rigorously established.23
Groups of outcomes are termed composite end points, the occurrence of any one of which is counted as an event. For example, a common composite end point for atherosclerotic heart disease is the occurrence of acute angina, heart attack, or stroke. Such end points are useful because they capture the larger effect of a treatment on a disease. When composite end points are used, the frequency of events for each of the component end points should be reported, especially when the component end points differ greatly in severity.
Identify Possible Sources of Bias, Confounding, and Error, and the Measures Taken To Control for Them
The term bias refers to systematic error: anything that results in consistently underestimating or overestimating the size or direction of the treatment effect. Activities to prevent bias include random sampling, random assignment, blinding, administering interventions, and data collection procedures according to tightly written protocols, third-party oversight of research, and checks and balances in data collection, analysis, and interpretation.
Confounding refers to factors that obscure the relationship between the presumed cause and presumed effect. Activities to prevent confounding include restricting enrollment in the study by excluding potentially confounding variables, matching patients on key variables to reduce variation, random assignment, stratification, and multivariate analysis, which controls for confounding statistically.
Finally, the term error refers to nondifferential, random error (biological variation) as well as to imprecision in measurements and mistakes in data collection, analysis, or interpretation. Hypothesis testing can provide a measure of random error (the p value); validated and reliable data collection methods can reduce measurement error; and careful, rigorous attention to detail throughout the research process can minimize mistakes.
Indicate the Minimum Change or Difference in the Outcome(s) Considered To Be Clinically Important
The minimum difference considered to be clinically important is often not stated in scientific articles, but it should be.24
Identify the Relationships Analyzed and the Statistical Techniques Used To Analyze Them
Confirm that the Assumptions of the Statistical Analysis Were Met
A statement that the assumptions were met is usually all that need be included.2,5
Identify Any Planned Subgroup or Covariate Analyses
Subgroup analyses planned in advance of the study are less likely to be the result of “data dredging,” in which investigators search for something that is statistically significant.5 A more powerful alternative to subgroup analysis, however, is to assess the potential interaction between two variables on the outcome of interest.
Identify the Statistical Software Package(s) Used To Analyze the Data
Although commercial programs generally are validated and updated and have met the test of time, the performance characteristics of privately developed programs are often unknown.
Identify the Time Frame of the Study: Give the Dates of the Enrollment, Treatment, and Data Collection Periods, and the Reasons for Selecting Those Dates
Provide a Schematic Summary of the Study Identifying the Number and Disposition of Participants at Each Stage of the Study
A schematic summary is a diagram that depicts the research design and identifies the size of the groups at each stage of the research (Fig 1
).5,25 Such a summary helps account for all patients throughout the study, identifies the denominators of the groups at each stage of the study, and usually indicates the study design. The form of the diagram is not as important as its ability to communicate visually.
Indicate the Degree to Which Participants Adhered to the Protocol and Explain Any Exceptions or Deviations From the Protocol
Deviations from the protocol can introduce bias, so the degree to which the protocol was followed is important. In particular, the number, reasons, and timing of deviations from the protocol or withdrawals from the study should be reported.26–27
Studies with high dropout or withdrawal rates (say, ≥ 15%) should be interpreted cautiously.2,28 High dropout rates may indicate serious problems with the treatment under investigation, problems with the conduct of the study, or large losses of data, all of which can bias the interpretation of the results.
Report the Results of the Study, Preferably in Figures or Tables29–31
At a Minimum, Report Absolute Values for All End Points, Including Within-Group Changes or Between-Group Differences
An absolute difference is the actual difference between measurements expressed in the units of the difference. For example, if the mean weight of a group dropped from 72 to 65 kg, the absolute difference is 7 kg. The relative difference is expressed as a percentage; here, the percentage change is 9.7% (7 kg/72 kg = 9.7%). Because a change from, say, 2 to 1 kg is the same percentage reduction as a change from 2,000 to 1,000 kg, reporting a 50% decrease for both examples, although accurate, can be misleading; hence, the absolute difference becomes important.
Report the Number or Percentage of Patients Who Improved (or Not), as Well as Group Values for Important End Points
Many studies report only group values for end points, which can hide individual variation among patients. For example, in studies20,32 of HIV/AIDS patients, the median gain or drop in T4 cell counts is often reported for each treatment group. Individual patients may have different responses than others in their group, however, so it is often useful to know how many patients had, say, improved T4 cell counts, as well as the median value for the group.
Provide Confidence Intervals for All End Points
The results of a study are actually estimates of what might be expected if the treatment were to be given to the entire population of interest.5 Confidence intervals indicate the precision of such an estimate.
Describe the Nature and Frequency of Common or Severe Side Effects and Adverse Events for Each Group
Adverse events have been conspicuously underreported in journal publications.2,5,33–34 For each group, give the following: (1) frequency, (2) severity (a matter of degree), (3) seriousness (a matter of threat to health or well being), and (4) timing of adverse events.33 General statements about the frequency of side effects (“there were few side effects”) are uninformative.33 It is also helpful to distinguish between adverse clinical events and laboratory-determined toxicities.
Account for All Observations and Participants and Explain Any Missing Data
To avoid charges of selective reporting, all observations should be accounted for. Missing data can be handled in several ways.35 In complete case reporting, patients with missing data are simply excluded from the analysis. Such exclusion effectively reduces the sample size and also violates the intent-to-treat strategy in experimental studies, both of which are undesirable. Values can also be imputed for missing data using any of several methods. Patients with missing data can also be included in the denominator of rates to provide more conservative estimates.26
The primary comparison should be discussed first. Secondary analyses of interest should be discussed later and should be presented as exploratory.
Interpret the Results and Suggest an Explanation for Them
Someone once said that “group means do not present for treatment”; therefore, do not confuse statistical significance with clinical importance! Biological plausibility, the cogency of the theory being tested, and the strength of other related evidence are more important than p values in interpreting results.2,5,36
Describe How the Results Compare With What Else Is Known About the Problem: Review the Literature and Put the Results in Context
Placing the results in the context of existing knowledge helps readers to interpret the work. Science is also cumulative and systematic and thus depends on investigators to show how their work is related to the rest of science.5
Suggest How the Results Might Be Generalized
The purpose of any single research study is to produce results that can apply to the population of interest.5 The population of interest should be defined by the inclusion and exclusion criteria. However, the more specific the definition of a population and the more tightly controlled the experiment, the more difficult it may be to generalize the results to a larger, more heterogeneous population outside the controlled environment of medical research, in the day-to-day activities of health care.
Discuss the Implications of the Results
The two questions most readers (and journal editors) want answered about a research study is “So what?” and “Who cares?” In other words, how will medicine be different as a result of this research? If the topic of the research was of marginal interest to begin with, the implications of the results will also be of marginal interest.
Discuss the Limitations of the Study
If possible, describe the sources and implications of potential bias, confounding, and error in the research design or problems with data collection, analysis, or interpretation. Disclosing weaknesses or limitations may be difficult, but honesty is an integral part of science. Identifying difficult areas in research may also help other investigators to avoid similar problems.
Listing your conclusions will help you be more specific about what, exactly, your study adds to the practice of medicine.
Figure Jump LinkFigure 1. A schematic summary of a randomized clinical trial.Grahic Jump Location
Lang, TA, Secic, M (2006)How to report statistics in medicine 2nd ed. American College of Physicians. Philadelphia, PA:
for the CONSORT Group. Moher, D, Schulz, K, Altman, DG CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials.Ann Intern Med2001;134,657-662. [PubMed]
Des Jarlais, DC, Lyles, C, Crepaz, N, the TREND Group. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement.Am J Public Health2004;94,361-366. [CrossRef] [PubMed]
von Elm E. The STROBE statement. Available at: http://www.strobe-statement.org/. Accessed August 22, 2006.
Durant, RH Checklist for the evaluation of research articles.J Adolesc Health1994;15,4-8. [CrossRef] [PubMed]
Altman, DG Statistical reviewing for medical journals.Stat Med1998;17,2661-2674. [CrossRef] [PubMed]
Instructions for preparing structured abstracts.JAMA2005;294,119-127
Lionel, ND, Herxheimer, A Assessing reports of therapeutic trials.BMJ1970;3,637-640. [CrossRef] [PubMed]
Walker, AM Reporting the results of epidemiological studies.Am J Public Health1986;76,556-558. [CrossRef] [PubMed]
Gifford, RH, Feinstein, AR A critique of methodology in studies of anticoagulant therapy for acute myocardial infarction.N Engl J Med1969;280,351-357. [CrossRef] [PubMed]
Mahon, WA, Daniel, EE A method for the assessment of reports of drug trials.Can Med Assoc J1964;90,565-569. [PubMed]
Zelen, M Guidelines for publishing papers on cancer clinical trials: responsibilities of editors and authors.J Clin Oncol1983;1,164-169. [PubMed]
Bhopal, R, Donaldson, L White, European, Western Caucasian, or what? -inappropriate labeling in research on race, ethnicity, and health.Am J Public Health1998;88,1301-1307
Witzig, R The medicalization of race: scientific legitimization of a flawed social construct.Ann Intern Med1996;125,675-679. [PubMed]
Bracken, MB Reporting observational studies.Br J Obstet Gynaecol1989;96,383-388. [CrossRef] [PubMed]
Haynes, RB How to read clinical journals: II. To learn about a diagnostic test.Can Med Assoc J1981;124,703-710. [PubMed]
Tugwell, PX How to read clinical journals: III. To learn the clinical course and prognosis of disease.Can Med Assoc J1981;124,869-872. [PubMed]
Davis, NM, Cohen, MR. Medication errors: causes and prevention. 1981; George Stickley Company. Philadelphia, PA:.
White, SJ Statistical errors in papers in the British Journal of Psychiatry.Br J Psychiatry1979;135,336-342. [CrossRef] [PubMed]
Felson, DT, Anderson, JJ, Meenan, RF Time for changes in the design, analysis, and reporting of rheumatoid arthritis clinical trials.Arthritis Rheum1990;33,140-149. [CrossRef] [PubMed]
Gotzsche, PC Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis.Control Clin Trials1989;10,31-56[erratum appears in Control Clin Trials 1989; 50:356]. [CrossRef] [PubMed]
for the Evidence-Based Medicine Working Group. Bucher, HC, Guyatt, GH, Cook, DJ, et al User’s guides to the medical literature XIX: Applying clinical trial results; A. How to use an article measuring the effect of an intervention on surrogate endpoints.JAMA1999;282,771-778. [CrossRef] [PubMed]
Fleming, TR, DeMets, DL Surrogate end points in clinical trials: are we being mislead?Ann Intern Med1996;125,605-613. [PubMed]
Freeman, KB, Back, S, Bernstein, J Sample size and statistical power of randomized, controlled trials in orthopaedics.J Bone Joint Surg2001;83,397-402. [CrossRef]
for the Consort Group. Egger, M, Juni, P, Bartlett, C Value of flow diagrams in reports of randomized controlled trials.JAMA2001;285,1996-1999. [CrossRef] [PubMed]
Hollis, S, Campbell, F What is meant by intention to treat analysis? Survey of published randomised controlled trials.BMJ1999;319,670-674. [CrossRef] [PubMed]
Grant, A Reporting controlled trials.Br J Obstet Gynaecol1989;96,397-400. [CrossRef] [PubMed]
Simon, R, Wittes, RE Methodologic guidelines for reports of clinical trials.Cancer Treat Rep1985;69,1-3. [PubMed]
Altman, DG Statistics and ethics in medical research: vIII. Improving the quality of statistics in medical journals.BMJ1981;282,44-47. [CrossRef] [PubMed]
Bailar, JC, III Science, statistics, and deception.Ann Intern Med1986;104,259-260. [PubMed]
Moskowitz, G, Chalmers, TC, Sacks, HS, et al Deficiencies of clinical trials of alcohol withdrawal.Alcohol Clin Exp Res1983;7,42-46. [CrossRef] [PubMed]
Ioannidis, JPA, Lau, J Completeness of safety reporting in randomized trials.JAMA2001;285,437-443. [CrossRef] [PubMed]
Ethgen, M, Boutron, I, Baron, G, et al Reporting of harm in randomized, controlled trials of nonpharmacologic treatment for rheumatic disease.Ann Intern Med2005;143,20-25. [PubMed]
Simon, G, Wagner, E, Vonkorff, M Cost-effectiveness comparisons using real world randomized trials: the case of new antidepressant drugs.J Clin Epidemiol1995;48,363-373. [CrossRef] [PubMed]
Goodman, SN Toward evidence-based medical statistics: 1. The P value fallacy.Ann Intern Med1999;130,995-1004. [PubMed]