From the Clinical Epidemiology Unit, General Medicine Division (Dr Singer), Massachusetts General Hospital; the Department of Medicine (Drs Fang and Go) and the Departments of Epidemiology and Biostatistics (Dr Go), University of California, San Francisco; and the Division of Research (Dr Go), Kaiser Permanente of Northern California, Oakland.
Correspondence to: Daniel E. Singer, MD, Clinical Epidemiology Unit, S50-9, Massachusetts General Hospital, Boston, MA 02114; e-mail: email@example.com
Financial/nonfinancial disclosures: The authors have reported to CHEST the following conflicts of interest: In the past 5 years, Dr Singer has consulted for Boehringer Ingelheim, Bayer HealthCare, Sanofi-Aventis, Merck, Daiichi Sankyo, Inc, and Johnson & Johnson, manufacturers of novel anticoagulants, and has received research support from Daiichi Sankyo, Inc. Dr Singer has also participated in a symposium sponsored by Bristol-Myers Squibb Company and Pfizer Inc. In addition, Dr Singer has received research support from the National Institutes of Health (NIH) (National Institute on Aging [NIA] and National Heart, Lung, and Blood Institute [NHLBI]). Dr Fang has received research support from the NIH (NIA and NHLBI). Dr Go has received research support from Johnson & Johnson. Dr Go has also received research support from the NIH (NIA and NHLBI) and from the American Heart Association.
Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (http://www.chestpubs.org/site/misc/reprints.xhtml).
© 2010 American College of Chest Physicians
In a recent issue of CHEST (February 2010), Lip et al1 assert that their “novel … schema could improve our approach to stroke risk stratification in patients with AF [atrial fibrillation].” We respectfully disagree because the following limitations, among others, substantially undercut this assertion:
Most importantly, the scheme’s predictive ability was poor, with a C statistic of 0.606. Although the authors suggest that their model was better than the widely used CHADS2 (an acronym for Congestive heart failure, Hypertension, Age >75, Diabetes, prior Stroke/transient ischemic attack) scheme, they provide no evidence that the two were statistically significantly different. Indeed, in one analysis, the authors note that their scheme did not predict thromboembolism “better than chance.” The study’s clearest finding was that the Framingham scheme performed best in their limited cohort.
Modern prognostic model building demands validation of the exact model to avoid deterioration in performance when used outside the original data.2 The authors claim that their study provides a validation for their Birmingham 2009 scheme. However, we were unable to identify any formal validation of the model (eg, through testing in an independent sample). The authors should make clear what they mean by validation.
The statistical approach described in “Methods” was not followed in the analyses. The “Methods” state, “Variables were removed stepwise from the model when the P value exceeded .10.” In the final model, six of the eight variables included did not meet the P < .10 criterion.
The authors claim that a distinctive feature of their scoring system is that their low-risk patients sustained no strokes during follow-up. They report a rate of 0 with a CI of (0-0). First, we note that this is an impossibly precise CI. The true upper bound actually includes stroke rates >3% per year. The low observed rate is largely the consequence of setting a very low threshold for categorizing patients as more than low risk. As a result, the scheme also categorizes 75% of their patients as high risk. It is particularly important to validate the performance of these thresholds since the threshold can be chosen post hoc.
The Euro Heart Survey had limitations that could bias the reported analyses. Most importantly, follow-up for thromboembolism was missing for a substantial (31%) portion of the cohort. An earlier report from Euro Heart conceded that some patients might have been lost because of stroke events.3
In sum, we need better stroke-risk prediction schemes for patients with AF, but the analysis reported by Lip et al is too premature for clinical application.
Register for a FREE personal account to access these and other personalization features:
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited:
Customize your page view by dragging & repositioning the boxes below.
Enter your username and email address. We'll send you a reminder to the email address on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.