Antibiotics and Pneumonia : Is Timing Everything or Just a Cause of More Problems? FREE TO VIEW

Sooner is probably better than later for the curative treatment of diseases that can rapidly be fatal. Pneumonia in older people is one such disease, accounting annually for > 750,000 hospitalizations of Americans ≥ 65 years old,¹ with a mortality rate of about 7% in hospital and 12% within 30 days.² Since 1998, the National Pneumonia Project (NPP) of the Centers for Medicare and Medicaid Services (CMS) has promoted hospital practices that are associated with improved outcomes.³ These include timely antibiotic administration following arrival at the hospital. While it was initially 8 h, the current NPP performance measure is 4 h. It was originally applied to persons in the age group ≥ 65 years, from which it was derived. The NPP sought an increased proportion of care to meet the measure, not 100% compliance. State-level performance rates were reported publicly, and no financial incentive was involved.

While this measure has generally been popular, complaints to the NPP leadership at CMS (ie, to me until December 2004) increased noticeably when payers began to use it for hospital-level public reporting and “pay-for-performance” (PFP) programs. Three common complaints were that it was being applied to patients < 65 years old, that the measure could not be met for some patients with “atypical” presentations, and that financial pressures from public reporting and PFP caused hospitals to seek 100% rates by administering antibiotics to patients when pneumonia had not been diagnosed. Some investigators doubted that timing affects the outcome. This issue of CHEST describes two relevant studies that explore why patients “fail” this measure, especially the role of diagnostic uncertainty. Metersky et al (see page 16)⁴ focus on how application of the measure might be modified to avoid unintended consequences such as antibiotic misuse. Waterer et al (see page 11)⁵ suggest that the association between the time to first antibiotic dose (TFAD) and mortality is specious.

The first question is, does TFAD at the hospital affect mortality? A modest number of articles in the literature address this question.⁶ The NPP measure is based on three large retrospective Medicare studies. Kahn et al⁷ noted reduced mortality with 4-h TFAD at 297 hospitals. Meehan et al⁸ analyzed data from 14,069 hospitalized pneumonia patients aged ≥ 65 years and observed a 15% reduction in the 30-day mortality rate with a TFAD of ≤ 8 h. Houck et al² examined data from 18,209 inpatients aged ≥ 65 years. Like Meehan et al,⁸ they included nursing home residents, required clinical and radiographic diagnoses, and used logistic regression with factors including the pneumonia severity index (PSI)⁹ to adjust for severity and confounding. A TFAD of ≤ 4 h was associated with a 15% reduction in the 30-day mortality rate among 13,771 patients who had not received outpatient therapy with antibiotics, while no reduction was observed among 4,438 pretreated patients.

Waterer et al⁵ followed up 451 patients. The mean age of patients was 58.2 years, and 158 were ≥ 65 years of age. The authors excluded patients from “nonambulatory” nursing homes and used PSI components in logistic regression. Altered mental status predicted both a TFAD of > 4 h and mortality in the full cohort and among older patients. In models that included mental status, a TFAD of > 4 h was not significantly associated with mortality among all patients (odds ratio [OR], 1.85; p = 0.117) or older patients (OR not given; p = 0.158). Since altered mental status was associated both with prolonged TFAD and increased mortality, it could act as a confounder, making it falsely appear that there was a TFAD-mortality association.

Do the findings of Waterer et al⁵ support their conclusions that TFAD performance measures are “… based on incomplete understanding… ,” and that altered mental status “… is the key factor driving both mortality and prolonged TFAD”? Few would argue that our understanding of pneumonia is complete. However, there are three reasons why these findings do not overturn the support by the Medicare studies²⁷⁸ for the use of TFAD measures in appropriate populations. First, the study populations differ substantially. The cohort of Waterer et al⁵ was small, excluded an undisclosed portion of nursing home patients, and garnered most cases and statistical power from patients < 65 years old. Interestingly, since Houck et al² could detect no timing-mortality association among 2,000 patients aged < 65 years who had been excluded from their main analyses,⁶ the findings of the two studies are actually consistent for younger patients. Second, the analytic methods differed. Houck et al² stratified their analysis by prior antibiotic treatment and did not detect any timing-mortality association among the 25% of patients who had been pretreated. The lack of adjustment for prior treatment and inadequate statistical power might account for the failure of Waterer et al⁵ to detect a significant association among older patients. Third, there is no evidence that altered mental status was an important confounder in the Medicare studies. In the national sample of Houck et al,² the documentation of altered mental status was similar among patients with TFADs of ≤ 4 h (23.2%) and > 4 h (24.4%; p = 0.11), making substantial confounding unlikely. While published findings² were adjusted for PSI score, parallel analyses using individual PSI components (including altered mental status) in the model gave equivalent results (Wato Nsa, MD; personal communication; January 19, 2006). Initial stratified analyses ruled out heart failure and shock as major confounders,⁶ while in recent analyses the association between a 4-h TFAD and the 30-day mortality rate remained (OR, 0.74; p < 0.001) when patients with altered mental status were excluded (Wato Nsa, MD; personal communication; January 19, 2006).

Waterer et al⁵ suggest that no mechanism explains why a few hours matter. A proposed mechanism is that each patient for whom therapy with antibiotics can potentially prevent death has a unique point in their deterioration beyond which that potential is lost.²⁶ If true, 4 to 8 h after arrival would represent the average of those points over thousands of cases. Since about 1,250 deaths might be prevented annually by timing improvement at all US hospitals,² a few hours are unlikely to affect demonstrably a small number of patients at an individual hospital. However, if one’s perspective is several hundred thousand Medicare patients at 4,000 hospitals, then an impact of 1,250 deaths is substantial and demonstrable.

The second question is, how often does clinical presentation produce such diagnostic uncertainty that delayed antibiotic administration is reasonable? Of 86 randomly selected pneumonia patients in the cases reviewed by Metersky et al,⁴ 19 patients (22%) had presentations that could “appropriately” delay treatment. Factors including lack of infiltrates seen on the chest radiograph were significantly associated with diagnostic uncertainty. Altered mental status was nearly twice as common among patients with diagnostic uncertainty (42.1%) than among those without it (23.7%). The findings of Metersky et al⁴ demonstrate that diagnostic uncertainty often is a barrier to diagnosis and timely but appropriate antibiotic administration. Were all patients with such presentations to receive treatment within 4 h, many actually without pneumonia would receive antibiotics, increasing antimicrobial resistance, costs, and the risk of adverse events.

How should these findings guide performance measurement? First, skepticism and reexamination must be encouraged, as they promote improved measures. However, new studies must have adequate power and populations comparable to those of the studies being challenged. The methods should reflect previous findings such as the importance of prior treatment. Randomized trials of TFAD would be ideal, but ethical concerns limit us to observational studies such as that of Meehan et al.⁸ Unfortunately, the CMS has stopped producing large, very clinically rich databases. Second, a performance measure used in public reporting or PFP should be applied only to the population from which it was derived, unless supporting evidence is clearly generalizable. A 4-h TFAD measure should thus be limited to persons aged ≥ 65 years with radiographic evidence of pneumonia and no antibiotic pretreatment. Third, as suggested by Metersky et al,⁴ public reporting and PFP should apply measures in ways that minimize consequences such as antibiotic therapy misuse. The application of inclusion criteria is one approach. For example, CMS will soon require a pneumonia diagnosis in the emergency department and radiographic evidence of pneumonia before the timing measure is applied, reducing the pressure to administer antibiotics prior to the confirmation of pneumonia (Dale Bratzler, DD; personal communication; January 16, 2006). However, this approach adds to the data abstraction burden. Another approach avoids the public reporting of the actual rates in hospitals and the use of percentile-based ranking in PFP. Reporting and rewarding hospitals by rate “band” (eg, 70 to 84% and 85 to 100%) would reduce indiscriminant antibiotic administration by eliminating the pressure to “look the very best” or be in the highest percentile of hospitals. Targeting a submaximal benchmark (eg, 80% or 85%) would allow for diagnostic uncertainty and unusual situations without defining specific exclusions. A variation would be to use a longer time (eg, 6 h) and a higher benchmark (eg, 90%).

Those responsible for quality improvement programs must address issues arising from how this or any other measures are being applied. The failure to do so will result in both increasing dissatisfaction in the same conscientious professionals who must be recruited as allies, and continued undesirable outcomes as some hospitals “game” the system. At the same time, we cannot await perfect evidence and be so hesitant to act on existing knowledge that patients fail to receive what is believed to be the most effective care. As Sir Austin Bradford Hill so aptly put it > 40 years ago, “All scientific work is incomplete… is liable to be upset or modified by advancing knowledge. That does not confer on us a freedom to ignore the knowledge that we already have or to postpone the action that it appears to demand at a given time.”¹⁰