0
Original Research: CRITICAL CARE MEDICINE |

Transferring Critically Ill Patients Out of Hospital Improves the Standardized Mortality Ratio*: A Simulation Study FREE TO VIEW

Jeremy M. Kahn, MD, MS; Andrew A. Kramer, PhD; Gordon D. Rubenfeld, MD, MSc
Author and Funding Information

*From the Division of Pulmonary & Critical Care (Drs. Kahn and Rubenfeld), Harborview Medical Center, University of Washington, Seattle WA; and Cerner Corporation (Dr. Kramer), Kansas City, MO.

Correspondence to: Jeremy M. Kahn, MD, MS, Division of Pulmonary, Allergy, and Critical Care, University of Pennsylvania School of Medicine, 874 Maloney Building, 3600 Spruce St, Philadelphia, PA 19104; e-mail: jkahn@cceb.med.upenn.edu



Chest. 2007;131(1):68-75. doi:10.1378/chest.06-0741
Text Size: A A A
Published online

Background: Transferring critically ill patients to other acute care hospitals may artificially impact benchmarking measures. We sought to quantify the effect of out-of-hospital transfers on the standardized mortality ratio (SMR), an outcome-based measure of ICU performance.

Methods: We performed a cohort study and Monte Carlo simulation using data from 85 ICUs participating in the acute physiology and chronic health evaluation (APACHE) clinical information system from 2002 to 2003. The SMR (observed divided by expected hospital mortality) was calculated for each ICU using APACHE IV risk adjustment. A set number of patients was randomly assigned to be transferred out alive rather than experience their original outcome. The SMR was recalculated, and the mean simulated SMR was compared to the original.

Results: The mean (± SD) baseline SMR was 1.06 ± 0.19. In the simulation, increasing the number of transfers by 2% and 6% over baseline decreased the SMR by 0.10 ± 0.03 and 0.14 ± 0.03, respectively. At a 2% increase, 27 ICUs had a decrease in SMR of > 0.10, and two ICUs had a decrease in SMR of > 0.20. Transferring only one additional patient per month was enough to create a bias of > 0.1 in 27 ICUs.

Conclusions: Increasing the number of acute care transfers by a small amount can significantly bias the SMR, leading to incorrect inference about ICU quality. Sensitivity to the variation in hospital discharge practices greatly limits the use of the SMR as a quality measure.

Figures in this Article

The movement toward increased accountability in medicine has prompted greater demand for valid and reliable measures of quality in the ICU.14 One potential quality measure is the standardized mortality ratio (SMR), which is the observed hospital mortality divided by the expected mortality adjusting for severity of illness and case mix.5An SMR of > 1 indicates higher than expected mortality, while an SMR of < 1 indicates lower than expected mortality. Theoretically, the SMR and other outcome-based quality measures can be used to compare outcomes across multiple ICUs or to follow temporal trends in a single ICU.68 Currently, the Joint Commission on Accreditation of Health Care Organizations (JCAHO) is considering the use of the SMR as a core measure of ICU quality in US hospitals.9

A major limitation of risk-adjusted outcome is that it can depend on factors other than the quality of care.10In addition to true variation in quality, variation in the SMR can be due to poor reliability in data collection, random error, and variation in case mix.11One additional factor may be patient discharge location. Many patients in the ICU are discharged directly to transitional care units such as skilled nursing facilities or long-term acute care centers (LTACs) or are transferred to regional medical centers for tertiary care. The outcomes of these patients are generally poor, with most studies reporting approximately a 50% in-hospital mortality rate for patients transferred to LTACs1217 and higher than expected mortality rate for those transferred to tertiary care centers.1819 ICUs that transfer a large number of patients may therefore have better than expected outcomes by shifting the mortality burden of high-risk patients.

Longitudinal data at the population level have shown that in years in which ICU transfers to nursing homes increase, the SMR tends to decrease.20 The effect of increasing out-of-hospital transfers at the individual ICU level, however, has not been directly examined. Furthermore, it is not known how many additional transfer patients are needed to produce a meaningful change in the SMR. To address this issue, we used cohort data from the acute physiology and chronic health evaluation (APACHE) IV clinical database to examine changes in the SMR at the ICU level after simulating an increase in the number of out-of-hospital transfers.

Study Design

We performed a cohort study and Monte Carlo simulation using data from the APACHE IV clinical database.21 APACHE is maintained by the Cerner Corporation (Kansas City, MO), which feeds back risk-adjusted outcome data to participating hospitals for benchmarking purposes. Detailed clinical information was prospectively collected on patients in 104 ICUs at 45 US hospitals during the calendar years 2002 and 2003. Data collection occurs on site by trained local coordinators. Although inconsistency in data collection procedures may occur, a three-phase standardized training session ensures that collection standards are uniform across sites, minimizing bias due to unreliable data collection techniques. All patients in the database were initially eligible for the study. Repeat hospital admissions and patients < 18 years of age were excluded from the study, as recommended in the JCAHO guidelines on ICU quality measures.9 Because it is difficult to accurately estimate risk-adjusted mortality when the number of patients is small, we also excluded ICUs with < 150 patients per year.

Variables and Analysis

Demographic and outcome data were summarized over each participating ICU. Original discharge location was determined by the ICU discharging nurse, and was coded as the hospital floor, telemetry, step-down unit, home, another ICU, another hospital (including skilled nursing facility or nursing home), or dead. Patients who were discharged to either another ICU or a hospital were grouped together as transfer patients for the baseline analysis. The specific location of transfer, such as nursing home, LTAC, or tertiary care center, was not available.

The SMR was calculated as the observed in-hospital mortality divided by the mean expected in-hospital mortality. The expected mortality rate for each patient was determined from the APACHE IV risk equation, which is a publicly available, validated model for predicting the outcome of critically ill hospitalized patients.21The APACHE IV mortality equation has been recently updated and includes the day 1 acute physiology score, age, and select chronic health items (unchanged from APACHE III), as well as primary diagnosis, hospital admission location (including whether a patient was transferred in from an outside hospital), pre-ICU length of stay, and whether or not a sedated patient could have their Glasgow coma score assessed, whether or not a patient received ventilation during the first ICU day, and whether the patient had undergone emergency surgery. Confidence intervals (CIs) for the SMR of each ICU were obtained using bootstrap resampling, which is a nonparametric method that produces valid CIs accounting for the variability in both observed and expected mortality.22 To perform the bootstrap, the population of each ICU was sampled with replacement 1,000 times; the SMR was calculated for each bootstrapped sample using the observed mortality and the predicted mortality from the APACHE IV equation, and the non-bias-corrected percentiles were used as the 95% CIs.

Monte Carlo Simulation

The simulation was performed by randomly sampling a set percentage of patients to be transferred out of the ICU alive rather than experience their original outcome. A separate simulation was performed for each ICU. Increases of 2% and 6% above baseline were chosen to represent a range of effects, with transfer rates that were consistent with those in the data set and were similar to those for patients receiving prolonged ICU care in other cohorts.23 The sampling algorithm was based partly on the patient’s severity of illness, such that patients with a higher probability of death were more likely to be assigned for transfer (see “Appendix”). The algorithm was chosen so that the probability of hospital death for new transfers would approximate that of the mortality of patients who actually transferred out of the ICU.,1219 An alternative strategy of sampling based on a propensity score was considered; however, many important factors related to transfer, such as bed availability at the accepting center, the goals of the treating physician, and patient and family preference, were unavailable. This made it unlikely that a propensity score could reliably identify patients who had a higher likelihood for transfer. We also considered a selection algorithm based on predicted ICU length of stay. In this cohort, however, patients were relatively equally likely to be transferred throughout their ICU stay, such that a length-of-stay algorithm would also not reliably identify patients who were at high likelihood for transfer.

Patients who were randomly selected for transfer then had their vital status at hospital discharge recoded to alive, and the SMR was recalculated based on the new observed mortality rate. The final transferred population therefore included all of those patients who actually transferred plus the 2% or 6% of patients randomly selected. The simulation was repeated 500 times with different random sets of assigned transfer patients (Fig 1 ). The mean probability of death and the mean new SMR from each simulation were recorded; transfer bias was calculated as the difference between the mean simulated SMR and the original SMR. The number of ICUs with bias outside the original 95% CI and the average number of patients transferred to create important bias were also calculated. Additionally, we performed a sensitivity analysis on the effect of transfer bias including the 19 ICUs that were originally excluded for low volume.

Statistical analyses and simulations were preformed with a statistical software package (Stata, version 8.0; Stata Corp; College Station, TX). This research was approved by the University of Washington institutional review board.

The complete cohort contained information on 131,618 admissions to 104 ICUs in 45 hospitals. We excluded 9,120 non-first admissions and 588 patients < 18 years of age. Nineteen ICUs in seven hospitals were excluded due to a volume of < 150 admissions per year (1,435 patients). The final cohort contained 120,475 patients in 85 ICUs. Unit demographics are shown Table 1 . There was significant variability in ICU type, academic status, size, and risk-adjusted outcome. ICUs transferred a median of 3% of patients out of hospital at baseline (interquartile range, 1.9 to 5.0), although this ranged as high as 17.4%.

The results of the Monte Carlo simulation are shown in Figure 2 and Table 2 . The simulations for each ICU indicate that increasing the number of out-of-hospital transfers results in a significant downward bias in the SMR at both 2% and 6% above baseline. As expected, severity of illness and APACHE IV risk of death of transfer patients increased as the number of transfers increased; however, the risk of death for transfers was still relatively low compared to that cited in the literature.1219 At an increase of 2% above baseline, nearly one third of ICUs experienced bias of > 0.1; at an increase of 6% above baseline, almost all ICUs experienced this degree of bias. In a large percentage of ICUs the new SMR was outside the original 95% CI, and many ICUs switched the direction of the quality indicator (ie, the SMR went from > 1.0 to < 1.0, such that a previously low-performing ICU appears to be high-performing). Figure 2 also shows that transfer bias can significantly affect ICU performance relative to each other (ie, rank). Were each ICU to be the only one to increase their number of transfers, they would significantly improve in rank compared to ICUs that did not increase their number of transfers. For example, the ICU with an original SMR of 1.27 (11th percentile in rank) would rise to the 28th percentile in the 2% simulation and the 37th percentile in the 6% simulation. The ICU with an original SMR of 1.05 (50th percentile in rank) would rise to the 67th percentile in the 2% simulation and the 75th percentile in the 6% simulation.

The mean (± SD) number of additional transfers per month was 1.2 ± 0.6 in the 2% simulation and 3.5 ± 1.9 in the 6% simulation. In the 2% simulation, 39 ICUs transferred only one additional patient per month or less, with an average bias of 0.10 ± 0.04. Of these, 16 still had bias > 0.1 despite the low numbers of patients. In the 6% simulation, 22 ICUs transferred ≤ 2% additional patients per month, with a mean bias of 0.16 ± 0.05. The bias was > 0.1 in all 22 of these ICUs.

Table 3 shows the mean transfer bias, stratified by the number of admissions, the baseline SMR, and the baseline percentage of transfer patients. Bias was marginally greater in smaller ICUs, low-performing ICUs, and ICUs with higher baseline transfer rates; however, in general the results were consistent across strata. Including the 19 small-volume ICUs in a sensitivity analysis did not appreciably change the size of the transfer bias in the simulation (mean transfer bias: 2% simulation, 0.09 ± 0.05; 6% simulation, 0.13 ± 0.05).

Several studies1819 have indicated that accepting critically ill transfer patients can adversely affect ICU benchmarking measures. This study documents that the reverse is true; that transferring even small numbers of critically ill patients out of the hospital can significantly improve the SMR of an ICU. In many ICUs, transferring only one additional patient per month was enough to cause significant bias. Bias could occur in ICUs of varying size, performance, and baseline number of transfer patients.

Transfer bias presents an important limitation to the SMR and other outcome-based quality measures. Although JCAHO is currently considering the use of the SMR as a core measure of hospital quality, a greater understanding of the factors affecting the SMR is needed before it should be widely used to benchmark ICU outcomes. Rather than reflecting differences in quality, the variation in the SMR could be explained in part by the variation in ICU discharge patterns. Transfer bias also introduces the possibility of “gaming” the system. Just as in some settings providers have avoided treating high-risk patients or up-coded the severity of illness to improve risk-adjusted outcome measures,2426 ICU providers could actively increase LTAC discharges to improve their SMR. Based on these results, ICU directors and accrediting organizations can monitor the number of transfer patients as a potential explanation for changes in the SMR over time. Transfer bias also has implications for pay-for-performance initiatives, in which hospitals receive bonus payments or other incentives based on the adoption of evidence-based care practices or improved risk-adjusted outcome. Pay for performance has long been advocated as a potential solution to the crisis in quality in US health care.27Some evidence suggests, however, that pay for performance tends only to reward high-performing centers without motivating improvement in low-performing centers.28 This study shows that, were a pay-for-performance scheme to be based on the SMR, hospitals could be rewarded for changes in their discharge practices rather than actual changes in quality.

Many other limitations to the SMR exist. Current risk adjustment methodology may not entirely account for differences in the case mix across providers.11,29This is true even in the ICU, where detailed physiologic measurements are plentiful and statistical models for outcome prediction have been in use for decades.30Different risk-stratification methods often give different results,3133 and even in the setting of detailed risk adjustment it is difficult to reliably identify quality outliers in the ICU.3436 Other threats to the validity of the SMR include random error and measurement error.

In light of these limitations, increased emphasis should be placed on process-based quality measures as a means of benchmarking ICU performance.7 Process measures can include rates of appropriate use of evidence-based therapies such as deep vein thrombosis prophylaxis, elevation of the head of the bed to prevent ventilator-associated pneumonia, and use of weaning and sedation protocols.37Process-based measures have the advantage of offering clear strategies for improvement when quality is low and being less influenced by variations in case mix than outcome-based measures.38Although more research is needed into the reliability and validity of these indicators, measuring and improving the process of care may represent the best way to improve the overall care of the critically ill.39

As a simulation study, this analysis is limited in that it may not accurately represent real-world transfer decisions. We could not model the complex decision-making process behind hospital discharges, and we did not factor in whether patients were transferred to LTACs or to tertiary care referral centers. Nonetheless, we based our simulation on a large multicenter database with careful risk adjustment, and have shown that the results were robust to baseline SMR, baseline transfer rate, and hospital size. Additionally, using Monte Carlo methodology introduces random variation into our estimates, such that the results are asymptotically true as different random groups of patients are selected for transfer.

The results of this analysis are also dependent on the algorithm used to randomly select patients for transfer. We chose a simple, easily understandable algorithm that resulted in newly assigned transfer patients having similar mortality rates to those reported in the literature. The resulting algorithm was actually conservative, as new transfer patients still had a predicted mortality rate of approximately 25%. Patients transferred to LTACs have reported mortality rates of 24 to 67%, with most studies reporting a > 50% mortality rate.1217 Patients transferred to tertiary care centers have reported mortality rates of 32 to 34%.1819 Thus, in practice the bias might be expected to be even larger than that we observed in this simulation. An alternative strategy that is not dependent on a simulation algorithm would have been an empiric study comparing the SMR among hospitals with varying levels of transfer or at a single hospital over time. There is no way to fully control for the multiple confounders related to transfer in such a study, including patient and physician preferences, making valid results unlikely.

This study shows that increasing the number of out-of-hospital transfers can bias the SMR, even when the number of transfer patients is small. Given the many sources of error in measurement and interpretation, caution is warranted in using the SMR as a benchmarking measure. Uncritical use of the SMR to benchmark ICU performance is likely to misinform rather than provide meaningful information about ICU quality. ICU directors and administrators can monitor the number of transfer patients as a potential explanation for a decreasing SMR.

Details of Selection Algorithm

In each iteration of the simulation, transfer patients included those who were actually transferred from the ICU as well as those randomly selected by the simulation algorithm. The selection algorithm was designed to generate a random sample of patients for simulated transfer to an outside institution with an observed mortality rate similar to that of typical transfer patients, but independent of whether the patient actually died. Since patients are frequently transferred for a higher level of care, we wanted the probability of transfer to be linked to the patient’s severity of illness as measured by their APACHE IV score for risk of death.

For each simulation, patients who were not actually transferred out of the hospital were assigned a random number (x) with a uniform distribution between 0 and 1. The random number was added to the product of their APACHE IV risk of hospital death and a scaling constant of 0.2 to select patients with an observed mortality rate of approximately 20%. This final number was between 0 (for individuals with a random number of 0 and a predicted mortality rate of 0) and 1.2 (for individuals with a random number of 0.99 and a predicted mortality rate of 0.99). Patients were ranked by this number, and the upper two or six percentiles (p) were selected for transfer.

This algorithm ensured that every patient had a possibility of transfer but that sicker patients were more likely to be transferred by a factor based on severity of illness. The predicted mortality rate of transfer patients was between 0.20 and 0.30 for each simulation, as shown in Table 2.

Abbreviations: APACHE = acute physiology and chronic health evaluation; CI = confidence interval; JCAHO = Joint Commission on Accreditation of Healthcare Organizations; LTAC = long-term acute care center; SMR = standardized mortality ratio

Dr. Kahn has no financial conflicts of interest to disclose. Dr. Kramer is an employee of the Cerner Corporation and owns shares of Cerner Corporation stock. Dr. Rubenfeld is a paid consultant for the Cerner Corporation in matters unrelated to the APACHE clinical information system.

Figure Jump LinkFigure 1. Schematic of the simulation. Large boxes indicate the entire ICU population; ovals indicate the actual and simulated transfer patients. In each simulation (SMR1), the SMR was recalculated with all transfer patients coded to survive (SMR2) and was repeated 500 times (SMR500) for each ICU. SMRbiased = outcome SMR of SMR1, SMR2, and SMR500.Grahic Jump Location
Table Graphic Jump Location
Table 1. ICU Demographics*
* 

Values are given as frequency (%), median (interquartile range), or mean ± SD.

 

The average of each ICU observed and the expected hospital mortality rate.

 

SMR is not equal to 1.00 because after exclusions (see “Materials and Methods” section) the cohort is different than the derivation cohort (21); and the mean SMR is the unweighted mean of the SMR of each hospital, rather than a global mean.

Figure Jump LinkFigure 2. Change in SMR for the 85 ICUs in the Monte Carlo simulation, ranked according to their baseline SMR. Triangles and circles denote the mean SMR in simulations with an increased number of transfers of 2% and 6% above baseline, respectively. Lines indicate 95% CIs for each ICU obtained from bootstrap resampling.Grahic Jump Location
Table Graphic Jump Location
Table 2. Bias in SMR With Increasing Numbers of Out-of-Hospital Transfers in 85 ICUs*
* 

Values are given as the mean ± SD or No. (%).

 

Monte Carlo simulation with an increased number of transfers of 2% above baseline.

 

Monte Carlo simulation with an increased number of transfers of 6% above baseline.

§ 

Based on day 1 APACHE IV hospital mortality predictions.

 

ICUs with original SMR, demonstrating worse than expected mortality, and biased SMR, demonstrating better than expected mortality.

Table Graphic Jump Location
Table 3. Transfer Bias Stratified by ICU Characteristics*
* 

Values are given as the mean ± SD, unless otherwise indicated.

 

Monte Carlo simulation with an increased number of transfers of 2% above baseline.

 

Monte Carlo simulation with an increased number of transfers of 6% above baseline.

Blumenthal, D (1996) Part 1. Quality of care: what is it?N Engl J Med335,891-894. [CrossRef] [PubMed]
 
Institute of Medicine... Crossing the quality chasm: a new health system for the 21st century. 2001; National Academy Press. Washington, DC:.
 
Garland, A Improving the ICU: part 1.Chest2005;127,2151-2164. [CrossRef] [PubMed]
 
Curtis, JR, Cook, DJ, Wall, RJ, et al Intensive care unit quality improvement: a “how-to” guide for the interdisciplinary team.Crit Care Med2006;34,211-218. [CrossRef] [PubMed]
 
Iezzoni, LI Risk adjustment for measuring health care outcomes 3rd ed.2003 Health Administration Press. Chicago, IL:
 
Knaus, WA, Wagner, DP, Zimmerman, JE, et al Variations in mortality and length of stay in intensive care units.Ann Intern Med1993;118,753-761. [PubMed]
 
Pronovost, PJ, Nolan, T, Zeger, S, et al How can clinicians measure safety and quality in acute care?Lancet2004;363,1061-1067. [CrossRef] [PubMed]
 
Angus, DC, Black, N Improving care of the critically ill: institutional and health-care system approaches.Lancet2004;363,1314-1320. [CrossRef] [PubMed]
 
Joint Commission on Accreditation of Healthcare Organizations... Specifications manual for national hospital quality measures-ICU, version 1.0. 2005; Joint Commission on Accreditation of Healthcare Organizations. Oakbrook Terrace, IL:.
 
Lilford, R, Mohammed, MA, Spiegelhalter, D, et al Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma.Lancet2004;363,1147-1154. [CrossRef] [PubMed]
 
Glance, LG, Osler, T, Shinozaki, T Effect of varying the case mix on the standardized mortality ratio and W statistic: a simulation study.Chest2000;117,1112-1117. [CrossRef] [PubMed]
 
Carson, SS, Bach, PB, Brzozowski, L, et al Outcomes after long-term acute care: an analysis of 133 mechanically ventilated patients.Am J Respir Crit Care Med1999;159,1568-1573. [PubMed]
 
Seneff, MG, Wagner, D, Thompson, D, et al The impact of long-term acute-care facilities on the outcome and cost of care for patients undergoing prolonged mechanical ventilation.Crit Care Med2000;28,342-350. [CrossRef] [PubMed]
 
Scheinhorn, DJ, Chao, DC, Stearn-Hassenpflug, M, et al Outcomes in post-ICU mechanical ventilation: a therapist-implemented weaning protocol.Chest2001;119,236-242. [CrossRef] [PubMed]
 
Schonhofer, B, Euteneuer, S, Nava, S, et al Survival of mechanically ventilated patients admitted to a specialised weaning centre.Intensive Care Med2002;28,908-916. [CrossRef] [PubMed]
 
Nasraway, SA, Button, GJ, Rand, WM, et al Survivors of catastrophic illness: outcome after direct transfer from intensive care to extended care facilities.Crit Care Med2000;28,19-25. [CrossRef] [PubMed]
 
Dematte D’Amico, JE, Donnelly, HK, Mutlu, GM, et al Risk assessment for inpatient survival in the long-term acute care setting after prolonged critical illness.Chest2003;124,1039-1045. [CrossRef] [PubMed]
 
Rosenberg, AL, Hofer, TP, Strachan, C, et al Accepting critically ill transfer patients: adverse effect on a referral center’s outcome and benchmark measures.Ann Intern Med2003;138,882-890. [PubMed]
 
Combes, A, Luyt, CE, Trouillet, JL, et al Adverse effect on a referral intensive care unit’s performance of accepting patients transferred from another intensive care unit.Crit Care Med2005;33,705-710. [CrossRef] [PubMed]
 
Sirio, CA, Shepardson, LB, Rotondi, AJ, et al Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland Health Quality Choice.Chest1999;115,793-801. [CrossRef] [PubMed]
 
Zimmerman, JE, Kramer, AA, McNair, DS, et al Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients.Crit Care Med2006;34,1297-1310. [CrossRef] [PubMed]
 
Hosmer, DW, Lemeshow, S Confidence interval estimates of an index of quality performance based on logistic regression models.Stat Med1995;14,2161-2172. [CrossRef] [PubMed]
 
Carson, SS, Bach, PB The epidemiology and costs of chronic critical illness.Crit Care Clin2002;18,461-476. [CrossRef] [PubMed]
 
Green, J, Wintfeld, N Report cards on cardiac surgeons: assessing New York State’s approach.N Engl J Med1995;332,1229-1232. [CrossRef] [PubMed]
 
Hofer, TP, Hayward, RA, Greenfield, S, et al The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease.JAMA1999;281,2098-2105. [CrossRef] [PubMed]
 
Burack, JH, Impellizzeri, P, Homel, P, et al Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons.Ann Thorac Surg1999;68,1195-1200. [CrossRef] [PubMed]
 
Rosenthal, MB, Fernandopulle, R, Song, HR, et al Paying for quality: providers’ incentives for quality improvement.Health Aff (Millwood)2004;23,127-141. [CrossRef] [PubMed]
 
Rosenthal, MB, Frank, RG, Li, Z, et al Early experience with pay-for-performance: from concept to practice.JAMA2005;294,1788-1793. [CrossRef] [PubMed]
 
Pappachan, JV, Millar, B, Bennett, ED, et al Comparison of outcome from intensive care admission after adjustment for case mix by the APACHE III prognostic system.Chest1999;115,802-810. [CrossRef] [PubMed]
 
Knaus, WA, Wagner, DP, Draper, EA, et al The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults.Chest1991;100,1619-1636. [CrossRef] [PubMed]
 
Iezzoni, LI The risks of risk adjustment.JAMA1997;278,1600-1607. [CrossRef] [PubMed]
 
Glance, LG, Osler, TM, Dick, A Rating the quality of intensive care units: is it a function of the intensive care unit scoring system?Crit Care Med2002;30,1976-1982. [CrossRef] [PubMed]
 
Glance, LG, Osler, TM, Dick, AW Identifying quality outliers in a large, multiple-institution database by using customized versions of the Simplified Acute Physiology Score II and the Mortality Probability Model II0.Crit Care Med2002;30,1995-2002. [CrossRef] [PubMed]
 
Thomas, JW, Hofer, TP Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care.Med Care1999;37,83-92. [CrossRef] [PubMed]
 
Baker, DW, Einstadter, D, Thomas, C, et al The effect of publicly reporting hospital performance on market share and risk-adjusted mortality at high-mortality hospitals.Med Care2003;41,729-740. [PubMed]
 
Sivak, ED, Rogers, MA Assessing quality of care using in-hospital mortality: does it yield informed choices?Chest1999;115,613-614. [CrossRef] [PubMed]
 
Pronovost, PJ, Berenholtz, SM, Ngo, K, et al Developing and pilot testing quality indicators in the intensive care unit.J Crit Care2003;18,145-155. [CrossRef] [PubMed]
 
Mant, J Process versus outcome indicators in the assessment of quality of health care.Int J Qual Health Care2001;13,475-480. [CrossRef] [PubMed]
 
Garland, A Improving the ICU: part 2.Chest2005;127,2165-2179. [CrossRef] [PubMed]
 

Figures

Figure Jump LinkFigure 1. Schematic of the simulation. Large boxes indicate the entire ICU population; ovals indicate the actual and simulated transfer patients. In each simulation (SMR1), the SMR was recalculated with all transfer patients coded to survive (SMR2) and was repeated 500 times (SMR500) for each ICU. SMRbiased = outcome SMR of SMR1, SMR2, and SMR500.Grahic Jump Location
Figure Jump LinkFigure 2. Change in SMR for the 85 ICUs in the Monte Carlo simulation, ranked according to their baseline SMR. Triangles and circles denote the mean SMR in simulations with an increased number of transfers of 2% and 6% above baseline, respectively. Lines indicate 95% CIs for each ICU obtained from bootstrap resampling.Grahic Jump Location

Tables

Table Graphic Jump Location
Table 1. ICU Demographics*
* 

Values are given as frequency (%), median (interquartile range), or mean ± SD.

 

The average of each ICU observed and the expected hospital mortality rate.

 

SMR is not equal to 1.00 because after exclusions (see “Materials and Methods” section) the cohort is different than the derivation cohort (21); and the mean SMR is the unweighted mean of the SMR of each hospital, rather than a global mean.

Table Graphic Jump Location
Table 2. Bias in SMR With Increasing Numbers of Out-of-Hospital Transfers in 85 ICUs*
* 

Values are given as the mean ± SD or No. (%).

 

Monte Carlo simulation with an increased number of transfers of 2% above baseline.

 

Monte Carlo simulation with an increased number of transfers of 6% above baseline.

§ 

Based on day 1 APACHE IV hospital mortality predictions.

 

ICUs with original SMR, demonstrating worse than expected mortality, and biased SMR, demonstrating better than expected mortality.

Table Graphic Jump Location
Table 3. Transfer Bias Stratified by ICU Characteristics*
* 

Values are given as the mean ± SD, unless otherwise indicated.

 

Monte Carlo simulation with an increased number of transfers of 2% above baseline.

 

Monte Carlo simulation with an increased number of transfers of 6% above baseline.

References

Blumenthal, D (1996) Part 1. Quality of care: what is it?N Engl J Med335,891-894. [CrossRef] [PubMed]
 
Institute of Medicine... Crossing the quality chasm: a new health system for the 21st century. 2001; National Academy Press. Washington, DC:.
 
Garland, A Improving the ICU: part 1.Chest2005;127,2151-2164. [CrossRef] [PubMed]
 
Curtis, JR, Cook, DJ, Wall, RJ, et al Intensive care unit quality improvement: a “how-to” guide for the interdisciplinary team.Crit Care Med2006;34,211-218. [CrossRef] [PubMed]
 
Iezzoni, LI Risk adjustment for measuring health care outcomes 3rd ed.2003 Health Administration Press. Chicago, IL:
 
Knaus, WA, Wagner, DP, Zimmerman, JE, et al Variations in mortality and length of stay in intensive care units.Ann Intern Med1993;118,753-761. [PubMed]
 
Pronovost, PJ, Nolan, T, Zeger, S, et al How can clinicians measure safety and quality in acute care?Lancet2004;363,1061-1067. [CrossRef] [PubMed]
 
Angus, DC, Black, N Improving care of the critically ill: institutional and health-care system approaches.Lancet2004;363,1314-1320. [CrossRef] [PubMed]
 
Joint Commission on Accreditation of Healthcare Organizations... Specifications manual for national hospital quality measures-ICU, version 1.0. 2005; Joint Commission on Accreditation of Healthcare Organizations. Oakbrook Terrace, IL:.
 
Lilford, R, Mohammed, MA, Spiegelhalter, D, et al Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma.Lancet2004;363,1147-1154. [CrossRef] [PubMed]
 
Glance, LG, Osler, T, Shinozaki, T Effect of varying the case mix on the standardized mortality ratio and W statistic: a simulation study.Chest2000;117,1112-1117. [CrossRef] [PubMed]
 
Carson, SS, Bach, PB, Brzozowski, L, et al Outcomes after long-term acute care: an analysis of 133 mechanically ventilated patients.Am J Respir Crit Care Med1999;159,1568-1573. [PubMed]
 
Seneff, MG, Wagner, D, Thompson, D, et al The impact of long-term acute-care facilities on the outcome and cost of care for patients undergoing prolonged mechanical ventilation.Crit Care Med2000;28,342-350. [CrossRef] [PubMed]
 
Scheinhorn, DJ, Chao, DC, Stearn-Hassenpflug, M, et al Outcomes in post-ICU mechanical ventilation: a therapist-implemented weaning protocol.Chest2001;119,236-242. [CrossRef] [PubMed]
 
Schonhofer, B, Euteneuer, S, Nava, S, et al Survival of mechanically ventilated patients admitted to a specialised weaning centre.Intensive Care Med2002;28,908-916. [CrossRef] [PubMed]
 
Nasraway, SA, Button, GJ, Rand, WM, et al Survivors of catastrophic illness: outcome after direct transfer from intensive care to extended care facilities.Crit Care Med2000;28,19-25. [CrossRef] [PubMed]
 
Dematte D’Amico, JE, Donnelly, HK, Mutlu, GM, et al Risk assessment for inpatient survival in the long-term acute care setting after prolonged critical illness.Chest2003;124,1039-1045. [CrossRef] [PubMed]
 
Rosenberg, AL, Hofer, TP, Strachan, C, et al Accepting critically ill transfer patients: adverse effect on a referral center’s outcome and benchmark measures.Ann Intern Med2003;138,882-890. [PubMed]
 
Combes, A, Luyt, CE, Trouillet, JL, et al Adverse effect on a referral intensive care unit’s performance of accepting patients transferred from another intensive care unit.Crit Care Med2005;33,705-710. [CrossRef] [PubMed]
 
Sirio, CA, Shepardson, LB, Rotondi, AJ, et al Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland Health Quality Choice.Chest1999;115,793-801. [CrossRef] [PubMed]
 
Zimmerman, JE, Kramer, AA, McNair, DS, et al Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients.Crit Care Med2006;34,1297-1310. [CrossRef] [PubMed]
 
Hosmer, DW, Lemeshow, S Confidence interval estimates of an index of quality performance based on logistic regression models.Stat Med1995;14,2161-2172. [CrossRef] [PubMed]
 
Carson, SS, Bach, PB The epidemiology and costs of chronic critical illness.Crit Care Clin2002;18,461-476. [CrossRef] [PubMed]
 
Green, J, Wintfeld, N Report cards on cardiac surgeons: assessing New York State’s approach.N Engl J Med1995;332,1229-1232. [CrossRef] [PubMed]
 
Hofer, TP, Hayward, RA, Greenfield, S, et al The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease.JAMA1999;281,2098-2105. [CrossRef] [PubMed]
 
Burack, JH, Impellizzeri, P, Homel, P, et al Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons.Ann Thorac Surg1999;68,1195-1200. [CrossRef] [PubMed]
 
Rosenthal, MB, Fernandopulle, R, Song, HR, et al Paying for quality: providers’ incentives for quality improvement.Health Aff (Millwood)2004;23,127-141. [CrossRef] [PubMed]
 
Rosenthal, MB, Frank, RG, Li, Z, et al Early experience with pay-for-performance: from concept to practice.JAMA2005;294,1788-1793. [CrossRef] [PubMed]
 
Pappachan, JV, Millar, B, Bennett, ED, et al Comparison of outcome from intensive care admission after adjustment for case mix by the APACHE III prognostic system.Chest1999;115,802-810. [CrossRef] [PubMed]
 
Knaus, WA, Wagner, DP, Draper, EA, et al The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults.Chest1991;100,1619-1636. [CrossRef] [PubMed]
 
Iezzoni, LI The risks of risk adjustment.JAMA1997;278,1600-1607. [CrossRef] [PubMed]
 
Glance, LG, Osler, TM, Dick, A Rating the quality of intensive care units: is it a function of the intensive care unit scoring system?Crit Care Med2002;30,1976-1982. [CrossRef] [PubMed]
 
Glance, LG, Osler, TM, Dick, AW Identifying quality outliers in a large, multiple-institution database by using customized versions of the Simplified Acute Physiology Score II and the Mortality Probability Model II0.Crit Care Med2002;30,1995-2002. [CrossRef] [PubMed]
 
Thomas, JW, Hofer, TP Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care.Med Care1999;37,83-92. [CrossRef] [PubMed]
 
Baker, DW, Einstadter, D, Thomas, C, et al The effect of publicly reporting hospital performance on market share and risk-adjusted mortality at high-mortality hospitals.Med Care2003;41,729-740. [PubMed]
 
Sivak, ED, Rogers, MA Assessing quality of care using in-hospital mortality: does it yield informed choices?Chest1999;115,613-614. [CrossRef] [PubMed]
 
Pronovost, PJ, Berenholtz, SM, Ngo, K, et al Developing and pilot testing quality indicators in the intensive care unit.J Crit Care2003;18,145-155. [CrossRef] [PubMed]
 
Mant, J Process versus outcome indicators in the assessment of quality of health care.Int J Qual Health Care2001;13,475-480. [CrossRef] [PubMed]
 
Garland, A Improving the ICU: part 2.Chest2005;127,2165-2179. [CrossRef] [PubMed]
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

CHEST Journal Articles
PubMed Articles
Guidelines
Diagnostic laparoscopy in the ICU. In: Diagnostic laparoscopy guidelines.
Society of American Gastrointestinal and Endoscopic Surgeons | 4/10/2009
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543