Objective: To evaluate the validity of using the
standardized mortality ratio (SMR) and the W statistic as risk-adjusted
measures of hospital mortality to judge ICU performance.
Design: APACHE (acute physiology and chronic health
evaluation) II data were collected prospectively from the surgical ICU
(SICU) at a single institution using all adult admissions (n = 6806)
over an 8-year period (excluding cardiac surgical patients, burn
patients, and patients under 16 years of age). Using a computer
simulation technique, virtual ICUs (VICUs) with mortality rates between
5% and 16% were constructed. After first dividing the original data
set into deciles of risk, each VICU was constructed by randomly
resampling between 10 and 680 patients from each decile. The SMR, W
statistic, and Z statistic were calculated for 10,000 different case
mixes.
Setting: The SICU at a 450-bed teaching
hospital.
Patients: A group of 6,806 adult patient
admissions, excluding cardiac surgical patients and burn patients.
Measurements and results: VICUs were created from a data
set of actual patients treated at one institution in order to test the
hypothesis that the SMR and W statistic would remain invariant when
applied to subsets of patients from a single institution. Instead, the
SMR and W statistic were found to be very sensitive to changes in case
mix. The SMR and W statistic were linear functions of the simulated ICU
mortality rate.
Conclusion: This simulation
demonstrates that the SMR and the W statistic based on APACHE II cannot
be used to compare outcomes of ICUs. We have proposed a revision of the
SMR that eliminates the effect of case mix and allows for more accurate
comparisons of ICU performance.