Affiliations: Western Hospital, Footscray, Australia,
Technion-Israel Institute of Technology, Haifa, Israel
Correspondence to: David Cunnington, MBBS, Respiratory and Sleep Disorders Medicine, Western Hospital, Footscray, VIC 3011, Australia; e-mail: firstname.lastname@example.org
The article by Bar et al (March 2003)1
reports an evaluation of the Watch PAT100 (WP100) device (Itamar Medical; Caesarea, Israel) for the diagnosis of obstructive sleep apnea. The manner in which the authors have presented their data does not adequately evaluate the accuracy of the WP100 device.
The authors have used correlation and Bland-Altman plots to compare calculations of the apnea hypopnea index (AHI) derived from the WP100 and conventional attended polysomnography. Although a statistically significant correlation was found, this does not imply agreement between the two measurements. The finding that there is a statistically significant relationship between the two measurements is not surprising as they are measuring the same thing. The real question is by how much measurements using the new method differ from the old. For this reason, Bland and Altman2
described the use of Bland-Altman plots to assess agreement. A critical part of this analysis is the calculation of the mean difference between the measurements and the limits of agreement (± 2 SDs) and plotting these on the graph. Used in this way the Bland-Altman plot gives a measure of agreement that allows the evaluation of the agreement between two measurements over a range of values.
Bar et al1
have included a Bland-Altman plot in their article, but failed to include the mean difference or the limits of agreement. From visual inspection of the data, it appears that there is a large difference between measurements, with the WP100 underscoring events at low AHIs and overscoring events at high AHIs. They report “good agreement” and a “slight tendency” to underscore and overscore events. However, given that the magnitude of the difference between the two measurements is as large as 45 events per hour, with many other points being > 10 events per hour, I do not agree with their conclusions. Differences of this magnitude are clinically significant and impact on the diagnosis and treatment of patients.
The lack of important data in this article assessing a new diagnostic device reminds us all of the importance of careful evaluation of new technologies before integrating them into clinical practice. Hopefully the initiatives of the Standards for Reporting of Diagnostic Accuracy group will help to improve reporting of new diagnostic techniques in the future.3
We have read the letter by Dr. Cunnington suggesting that the significant relationship between the respiratory disturbance index derived from the polysomnography and the Watch PAT100 (WP100) device (Itamar Medical; Caesarea, Israel) is not surprising, and that the real question is by how much measurements using the new peripheral arterial tonometry method differ from traditional polysomnography. We showed that the WP100 can accurately detect patients with obstructive sleep apnea (OSA), and believe that this diagnosis is the crux of the article rather than by how much the methods differed on a patient-by-patient basis. In order to test the ability of the WP100 to accurately detect OSA, we have used three different statistical methods, the most important of which is probably the area under the receiver operating characteristic curve to determine the sensitivity and specificity of the new device to accurately diagnose OSA. The other two methods were correlation and a Bland-Altman plot.
We do agree with Dr. Cunnington that looking at the difference rather than just at the agreement in the Bland-Altman plot may add some useful information, and thus have performed further analysis on the differences between the methods. We have found that the mean difference was 3.5 ± 11.9 events per hour of sleep (± SD), which further support the accuracy of the WP100. The mean difference is relatively small, and the relatively large SD of the differences is primarily driven by five patients (with very severe disease) in whom the differences between the two methods were > 30 events per hour. In those patients with severe OSA, these differences have no clinical significance, and we do not think that this further analysis should change anything in the conclusions presented in the original report.
Become a CHEST member and receive a FREE subscription as a benefit of membership.
Individuals can purchase this article on ScienceDirect.
Individuals can purchase a subscription to the journal.
Individuals can purchase a subscription to the journal or buy individual articles.
Learn more about membership or Purchase a Full Subscription.
Institutional access is now available through ScienceDirect and can be purchased at myelsevier.com.
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited:
Customize your page view by dragging & repositioning the boxes below.
Enter your username and email address. We'll send you a reminder to the email address on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.