0
Correspondence |

ResponseResponse FREE TO VIEW

Kathleen A. Clark, MS, RRT; J. Jay Flynn, III, MD; Julie E. Goodman, PhD; Ke Zu, PhD, ScD; Wilfried J. J. Karmaus, MD, MPH; Lawrence C. Mohr, MD, ScD, FCCP
Author and Funding Information

From the Department of Epidemiology and Biostatistics (Ms Clark), Arnold School of Public Health, University of South Carolina; Health Network America (Dr Flynn); Gradient Corporation (Drs Goodman and Zu); the Department of Epidemiology (Dr Goodman), Harvard School of Public Health; the Division of Epidemiology, Biostatistics and Environmental Health (Dr Karmaus), School of Public Health, University of Memphis; and the Department of Medicine (Dr Mohr), the Department of Public Health Sciences (Dr Mohr), and the Environmental Biosciences Program (Dr Mohr), Medical University of South Carolina.

CORRESPONDENCE TO: Lawrence C. Mohr, MD, ScD, FCCP, Environmental Biosciences Program, Medical University of South Carolina, 135 Cannon St, Ste 405, PO Box 250838, Charleston, SC 29425; e-mail: mohrlc@musc.edu


FINANCIAL/NONFINANCIAL DISCLOSURES: The authors have reported to CHEST the following conflicts of interest: Ms Clark received a graduate student research grant from Health Network America for the collection and analysis of medical surveillance data from the LMP. Funding for this research grant was provided to Health Network America by W. R. Grace and Co. Dr Goodman is a Principal at Gradient Corp, an independent environmental and risk science consulting firm, in Cambridge, MA. Dr Zu is a research associate at Gradient Corp. Through a contract with Gradient Corp, Drs Goodman and Zu have served as scientific consultants to W. R. Grace and Co on issues related to asbestos exposure and asbestos-related health risks. They both contributed to the data analyses reported in this manuscript with funding received through the Gradient consulting contract with W. R. Grace and Co. W. R. Grace and Co has recently provided research funding to Gradient Corp for a preliminary weight-of-evidence study related to associations between PPs and lung function. Dr Goodman has served as an expert witness on cases involving chrysotile asbestos and cancer risk and has received funding from Tucker Ellis & West LLP for the preparation of scientific manuscripts regarding radiation, mesothelioma, and lung cancer. She previously received funding from the National Cancer Institute as a Cancer Prevention Fellow. Dr Flynn is an independent physician contractor for Health Network America, a health-care consulting company, in Eatontown, NJ. He served as Medical Director of the LMP from January 2001 to January 2013, as a Health Network America employee. Health Network America received fees from W. R. Grace and Co for providing administrative services to the LMP. Dr Flynn participated in all public comment sessions for the Environmental Protection Agency Toxicological Review of Libby Amphibole Asbestos in 2011 and 2012, in his capacity as Medical Director of the LMP. Dr Karmaus has received no funding for his work on this project and has no potential conflict of interest; he has received research funding from the National Institutes of Health for work on other projects in collaboration with Ms Clark and Dr Mohr. Dr Mohr has served as a research consultant to Exponent and Health Network America, scientific research and consulting firms, which received funding from W. R. Grace Co for research and scientific consultation on asbestos-related health risks. He has also received funding through a research contract with Gradient Corp for a literature review and preliminary weight-of-evidence study related to associations between PPs and lung function. He has submitted public commentary reports to the Scientific Advisory Board of the US Environmental Protection Agency on the Draft Toxicological Review Pertaining to Libby Amphibole Asbestos (2011) and the Association Between Localized Pleural Thickening (Pleural Plaques) and Lung Function; both reports were submitted in his capacity as a research consultant to Exponent, which received funding from W. R. Grace and Co for this work. Dr Mohr has received research grant funding from the National Institutes of Health, the National Cancer Institute, the US Department of Energy, the Agency for Healthcare Research and Quality, the Health Resources and Services Administration, the South Carolina Universities Research and Education Foundation, and the North Atlantic Treaty Organization for work on other research projects. He has served on scientific advisory boards for Marine Polymer Technologies, Inc and Entegrion. The authors wish to report that W. R. Grace and Co operated the Libby Vermiculite mine from 1963 to 1990.

Reproduction of this article is prohibited without written permission from the American College of Chest Physicians. See online for more details.


Chest. 2015;147(3):e124-e126. doi:10.1378/chest.14-3041
Text Size: A A A
Published online
To the Editor:

We welcome the opportunity to respond to Dr Noonan’s letter regarding our recent article in CHEST.1 Statistical power is always a potential limitation in biomedical studies. This is especially true in retrospective observational studies, in which an a priori power analysis cannot be performed. As mentioned in our article, although our reference group of miners with normal high-resolution CT scan (NCTS) studies was relatively small, there was enough power to detect statistically significant differences in mean lung function data between the NCTS reference group and the miner group with pleural plaques and interstitial fibrosis and the miner group with other high-resolution CT (HRCT) scan findings. However, when we compared the miner group with pleural plaques only (PPO) with the NCTS reference group, we were unable to detect a statistical significance in mean lung function data, even though the PPO miner group had many more subjects than did the miner group with pleural plaques and interstitial fibrosis or the miner group with other HRCT scan findings. Information contained within the 95% CI for comparison of the PPO miner group with the NCTS reference group indicates that we should not reject the null hypothesis. Tukey analysis was also used to minimize type 1 error. Thus, our analysis supports the conclusion that pleural plaques alone have no significant effect on lung function in Libby vermiculite miners.

In his letter, Dr Noonan includes a post hoc power analysis. However, his analysis is meaningless with respect to the statistical significance of our results. Current statistical convention dictates that post hoc power analysis should not be performed because it is fundamentally flawed, does not provide any meaningful information, and may be misleading.2-6 There are a number of reasons for its inappropriateness. First, post hoc power analysis can mislead the reader into concluding that an observed result is a false-negative caused by type 2 error, by incorrectly implying that a low post hoc power is caused by “too few” subjects. Apparently, this is what Dr Noonan has attempted to imply. However, once a statistically insignificant result has been obtained, the effect of sample size and associated variance should be obtained from information within the 95% CI, not by post hoc power analysis.2-6 In addition, calculated power has relevance only when the null is known to be false. To calculate post hoc power for a statistically insignificant result requires one to make a biased assumption that the null is false, even though it is already known that this assumption is not supported by the data. This is “nonsensical.”3 Furthermore, a statistically insignificant result will always result in a low post hoc power. That is, P ≥ .05 will always produce a post hoc power ≤ 0.5, regardless of how large or small the sample size is. This is because post hoc power is directly related to the P value mathematically. In fact, the greater (less significant) the observed P value, the lower the post hoc calculated power. Thus, the basic logic of using post hoc power analysis is convoluted and fundamentally flawed. In this regard, calculating post hoc power adds no new information once the P value is known4,5; it simply tells us what we already know. With respect to our study, Dr Noonan’s post hoc power analysis simply tells us that our data analysis does not detect a statistically significant difference in mean lung function data between the miner group with pleural plaques alone and the miner group with normal HRCT scan studies. Again, this result is supported by information contained within the 95% CI.

Furthermore, based on the means in Table 3, we would not expect any significant lung function differences between the PPO group and the NCTS reference group if we were to conduct a new investigation with a larger sample size, with the possible exception of expiratory reserve volume. However, because elevated BMI is well known to reduce expiratory reserve volume, this possibility is most likely related to the high proportion of our cohort subjects with an elevated “obese range” BMI and not caused by any effect of pleural plaques.

For readers of CHEST who desire a clinically oriented explanation of the senselessness of post hoc power analysis, we recommend an excellent article by Goodman and Berlin.6 These authors state that post hoc power analysis is “like trying to convince someone that buying a lottery ticket was foolish (the before-experiment perspective) after they hit a lottery jackpot (the after-experiment perspective).”6 This is a good way to think about the impropriety of post hoc power analysis. These authors also state, “Once the data are in, the only way to avoid confusion is to not compress the results into dichotomous significance verdicts and to avoid post hoc power analyses entirely.”6

In his letter, Dr Noonan characterizes our study as an “occupational study.” We respectfully disagree with this characterization. Although the data for this study were derived from an occupational cohort of Libby vermiculite miners, the objective and design of the study were simply to determine the effect of pleural plaques, as detected by HRCT scan, on miner lung function. Therefore, our study is a physiology study and was appropriately published as such by the editors of CHEST. Our study does not address “causal association” of occupational exposures with either pleural plaques or lung function. It addresses the effect of an HRCT scan anatomic variant (pleural plaques) on lung function. We did adjust for the length of employment and the time since last occupational asbestos exposure, but neither covariate significantly contributed to lung function outcomes. We agree with the information that Dr Noonan has provided about the basic elements of an “occupational study”; however, this information has no relevance to our study conclusion.

We greatly appreciate Dr Noonan’s thought-provoking comments. They have enabled us to further describe the nature of our physiology study and provide additional support for our study conclusion.

References

Clark KA, Flynn JJ III, Goodman JE, Zu K, Karmaus WJJ, Mohr LC. Pleural plaques and their effect on lung function in Libby vermiculite miners. Chest. 2014;146(3):786-794. [CrossRef] [PubMed]
 
Smith AH, Bates MN. Confidence limit analyses should replace power calculations in the interpretation of epidemiologic studies. Epidemiology. 1992;3(5):449-452. [CrossRef] [PubMed]
 
Zumbo BD, Hubley AM. A note on misconceptions concerning prospective and retrospective power. J R Stat Soc Series D. 1998;47(2):385-388. [CrossRef]
 
Levine M, Ensom MH. Post hoc power analysis: an idea whose time has passed? Pharmacotherapy. 2001;21(4):405-409. [CrossRef] [PubMed]
 
Hoenig JM, Heisey DM. The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55(1):19-24. [CrossRef]
 
Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med. 1994;121(3):200-206. [CrossRef] [PubMed]
 

Figures

Tables

References

Clark KA, Flynn JJ III, Goodman JE, Zu K, Karmaus WJJ, Mohr LC. Pleural plaques and their effect on lung function in Libby vermiculite miners. Chest. 2014;146(3):786-794. [CrossRef] [PubMed]
 
Smith AH, Bates MN. Confidence limit analyses should replace power calculations in the interpretation of epidemiologic studies. Epidemiology. 1992;3(5):449-452. [CrossRef] [PubMed]
 
Zumbo BD, Hubley AM. A note on misconceptions concerning prospective and retrospective power. J R Stat Soc Series D. 1998;47(2):385-388. [CrossRef]
 
Levine M, Ensom MH. Post hoc power analysis: an idea whose time has passed? Pharmacotherapy. 2001;21(4):405-409. [CrossRef] [PubMed]
 
Hoenig JM, Heisey DM. The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55(1):19-24. [CrossRef]
 
Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med. 1994;121(3):200-206. [CrossRef] [PubMed]
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Find Similar Articles
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543