0
Original Research: Pulmonary Vascular Disease |

Prognostic Accuracy of Clinical Prediction Rules for Early Post-Pulmonary Embolism All-Cause MortalityAccuracy of Clinical Prediction Rules: A Bivariate Meta-analysis FREE TO VIEW

Christine G. Kohn, PharmD; Elizabeth S. Mearns, PharmD; Matthew W. Parker, MD; Adrian V. Hernandez, MD, PhD; Craig I. Coleman, PharmD
Author and Funding Information

From the University of Saint Joseph School of Pharmacy (Dr Kohn), Hartford, CT; the University of Connecticut/Hartford Hospital Evidence-Based Practice Center (Drs Mearns and Coleman), Hartford, CT; the University of Connecticut School of Pharmacy (Drs Mearns and Coleman), Storrs, CT; the Departments of Critical Care Medicine and Cardiology (Dr Parker), Hartford Hospital, Hartford, CT; the Department of Quantitative Health Sciences (Dr Hernandez), Cleveland Clinic Lerner Research Institute, Cleveland, OH; and the Unidad de Análisis y Gestión de Evidencias en Salud Pública (Dr Hernandez), Instituto Nacional de Salud, Lima, Peru.

CORRESPONDENCE TO: Craig I. Coleman, PharmD, University of Connecticut/Hartford Hospital, Evidence-Based Practice Center, 80 Seymour St, Hartford, CT 06102-5037; e-mail: craig.coleman@hhchealth.org


FUNDING/SUPPORT: This project was supported by a grant from Janssen Scientific Affairs, LLC, Raritan, NJ.

Reproduction of this article is prohibited without written permission from the American College of Chest Physicians. See online for more details.


Chest. 2015;147(4):1043-1062. doi:10.1378/chest.14-1888
Text Size: A A A
Published online

BACKGROUND:  Studies suggest outpatient treatment or early discharge of patients with acute pulmonary embolism (aPE) is reasonable for those deemed to be at low risk of early mortality. We sought to determine clinical prediction rule accuracy for identifying patients with aPE at low risk for mortality.

METHODS:  We performed a literature search of Medline and Embase from January 2000 to March 2014, along with a manual search of references. We included studies deriving/validating a clinical prediction rule for early post-aPE all-cause mortality and providing mortality data over at least the index aPE hospitalization but ≤ 90 days. A bivariate model was used to pool sensitivity and specificity estimates using a random-effects approach. Traditional random-effects meta-analysis was performed to estimate the weighted proportion of patients deemed at low risk for early mortality and their ORs for death compared with high-risk patients.

RESULTS:  Forty studies (52 cohort-clinical prediction rule analyses) reporting on 11 clinical prediction rules were included. The highest sensitivities were observed with the Global Registry of Acute Coronary Events (0.99, 95% CI = 0.89-1.00), Aujesky 2006 (0.97, 95% CI = 0.95-0.99), simplified Pulmonary Embolism Severity Index (0.92, 95% CI = 0.89-0.94), Pulmonary Embolism Severity Index (0.89, 95% CI = 0.87-0.90), and European Society of Cardiology (0.88, 95% CI = 0.77-0.94) tools, with remaining clinical prediction rule sensitivities ranging from 0.41 to 0.82. Of these five clinical prediction rules with the highest sensitivities, none had a specificity > 0.48. They suggested anywhere from 22% to 45% of patients with aPE were at low risk and that low-risk patients had a 77% to 97% lower odds of death compared with those at high risk.

CONCLUSIONS:  Numerous clinical prediction rules for prognosticating early mortality in patients with aPE are available, but not all demonstrate the high sensitivity needed to reassure clinicians.

Figures in this Article

Acute pulmonary embolism (aPE) is common, with an estimated annual incidence of 69 cases per 100,000.14 aPE often leads to hospitalization for monitoring and initiation of a parenteral anticoagulant as a bridge to vitamin K antagonist (VKA) therapy.3,4 However, the management of aPE with a VKA carries the need for frequent laboratory monitoring and dosage adjustments, which can significantly delay hospital discharge.5 The newer oral anticoagulants (chiefly rivaroxaban and apixaban, which do not require pretreatment with a heparin) provide the potential for cost-effective management of aPE by allowing for shorter aPE-related hospital stays or, in some patients, reducing the need for a hospital admission altogether.

Multiple studies6 suggest outpatient treatment of symptomatic aPE is reasonable for patients at low risk of early post-aPE all-cause mortality. However, there is currently no consensus for what criteria or clinical prediction rules to use to categorize patients with aPE into lower- or higher-risk groups.746 Thus, we performed a systematic review and meta-analysis to (1) identify published clinical prediction rules that use a combination of multiple prognostic factors for determining the risk of early all-cause mortality in patients suffering an aPE, (2) determine the proportion of patients with aPE deemed to be at low (generally regarded as suitable for outpatient treatment or early hospital discharge) or high risk of early mortality according to these prediction rules and the relative odds of early mortality between these groups, and (3) assess the prognostic accuracy of clinical prediction rules for identifying patients with aPE at low risk for early mortality and, thus, suitable for outpatient treatment or early hospital discharge.

Study Selection

We performed a systematic literature search of the Medline and Embase computerized bibliographic databases from January 1, 2000, through March 17, 2014. The searches began at the year 2000 to limit the identification of studies not using modern aPE diagnostic and treatment practices (ie, not following evidence-based guidelines for diagnosis, risk stratification, use of interventions, and duration of anticoagulation created after the performance of well-done heparin and VKA randomized trials).4749 For our search, we combined Medical Subject Heading terms and key words for aPE with previously validated search filters for prognostic studies.50 Our Medline search strategy is provided in e-Appendix 1. Manual backward citation tracking of references from identified studies and review articles was also performed to identify additional relevant studies.

Two investigators (C. G. K., C. I. C.) independently scanned titles and abstracts for initial inclusion, with disagreements resolved by discussion. Potentially eligible articles were then reviewed in depth by two investigators (C. G. K., C. I. C.) for inclusion, with disagreements resolved by discussion. To be included in this analysis, studies had to meet the following inclusion criteria: (1) evaluate a cohort of patients experiencing an aPE, (2) be a prognostic study designed to derive and/or validate a clinical prediction rule consisting of a combination of multiple prognostic factors for early post-aPE all-cause mortality, (3) provide data on early all-cause mortality (the reference standard) over at least the index aPE hospital admission but not longer than at 90 days, and (4) be published in English language full text. Our base-case analysis only included studies enrolling patients with aPE regardless of hemodynamic stability at admission. However, we did identify studies limited to hemodynamically stable patients only and used these studies in sensitivity analysis.

Validity Assessment

Two investigators (C. G. K., C. I. C.) independently assessed validity for each included study. We adapted the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool, which assesses bias and applicability over multiple domains (eg, patient selection, index test [clinical prediction rule], reference test [early all-cause mortality], and flow and timing)51 to assess the quality of each cohort-clinical prediction rule analysis as having low, high, or unclear risk of bias and concerns regarding applicability (e-Table 1). In addition, we used the Hierarchy of Evidence for Clinical Decision (Prediction) Rules described by McGinn and colleagues52 to classify the overall body of evidence for each clinical prediction rule into one of four levels (level 1 = one or more prospective validation in a different population and one impact analysis demonstrating change in clinician behavior with beneficial consequences; level 2 = demonstrated accuracy in either one large prospective study including a broad spectrum of patients and clinicians or validated in several smaller settings that differ from one another; level 3 = validated in only one narrow prospective sample; level 4 = derived but not validated or validated only in split samples, large retrospective databases, or by statistical techniques).

Data Extraction

Two investigators (C. G. K., E. S. M.), through the use of a standardized tool, independently extracted all data, with disagreements resolved by a third investigator (C. I. C.). Data collected included: study/cohort identifier and year of publication; geographic location; sample size; study design (prospective vs retrospective); study inclusion/exclusion criteria; sampling technique (consecutive patients, random, or convenience sample); patient characteristics (age, proportion of patients with cancer), methods for diagnosing aPE (clinical signs and symptoms, pulmonary angiography, CT scan, ventilation-perfusion lung scan, medical records, billing codes); hemodynamic status of patients at admission; loss to follow-up, method of mortality determination, clinical prediction rule scoring, and patient-level 2 × 2 data (proportions dying in both the low- and high-risk groups) needed to calculate true and false positives and negatives for the occurrence of early all-cause mortality (the latter used to calculate sensitivity, specificity, and other accuracy statistics for clinical prediction rule prognostication). Some studies reported all-cause mortality data at various time points. For the purposes of this meta-analysis we preferentially used 30-day mortality data, followed by 90-day, 7-day, and, finally, in-hospital data. In cases of missing 2 × 2 data, we attempted to contact the corresponding authors by e-mail. If we did not receive an answer after sending a reminder, we excluded the study. Studies evaluating the same clinical prediction rule in the same or overlapping populations were identified, and the largest of the studies was kept for analysis.

Statistical Analysis

We used a bivariate statistical model to pool logit transformed sensitivity and specificity estimates.5355 The bivariate approach simultaneously models pairs of (logit transformed) sensitivity and specificity from individual studies and, in so doing, incorporates any correlation that may exist between these measures. The model uses a random-effects approach to incorporate heterogeneity resulting from clinical and methodological differences between studies. Summary estimates of sensitivity and specificity were then computed by inverse transformation of logit estimates to the original receiver-operating characteristic (ROC) scale. For this meta-analysis, sensitivity was our a priori selected primary end point, since it was assumed that clinicians would be most concerned about mistakenly not admitting or discharging early a patient with aPE who was at high risk of early all-cause mortality (desire to minimize false negatives). We tested for significant differences between clinical prediction rule summary sensitivity and specificity estimates (with the Pulmonary Embolism Severity Index [PESI] model as referent because of its frequent study) using the methods described by Altman and Bland and statistical significance defined as P < .05.56 A bivariate summary ROC (sROC) for each of the post-aPE early all-cause mortality clinical prediction rules with summary operating points and 95% confidence regions was generated. Summary positive and negative likelihood ratios (LR+ and LR−), diagnostic ORs (DORs), and area under the curve estimates were also calculated as part of the bivariate analysis.

In addition to the bivariate meta-analysis, we also performed traditional meta-analysis for each clinical prediction rule, whereby the proportion of each cohort’s total population deemed to be at low risk for early mortality was determined and used to calculate weighted summary proportions and accompanying 95% CIs. Moreover, the proportion of patients experiencing early mortality in the low- and high-risk groups of each study was determined and used to calculate ORs with accompanying 95% CIs for each clinical prediction rule. Finally, DORs (calculated as [true positives/false negative]/[false positive/true negative]) were calculated for each cohort and used to compute a summary estimate with 95% CIs. Each of these traditional meta-analytic analyses was conducted using a random-effects approach.

In all analyses, statistical heterogeneity was assessed using the I2 statistics (a value ≥ 50% deemed substantial). To assess the presence of publication bias, we created and tested funnel plots (with SE on the vertical axis and the log of the DOR on the horizontal axis) for asymmetry using Eggers test (a P value < .05 deemed statistically significant).57 This was done for all clinical prediction rules with more than one evaluable cohort. We performed prespecified sensitivity analyses to evaluate whether the inclusion of studies/cohorts evaluating only hemodynamically stable patients, removing poorer-quality studies (those deemed to have high or unclear risk of bias on QUADAS-2), and excluding studies/cohorts following patients for > 30 days for mortality altered our meta-analysis’ results.

R version 3.0.2 (The R Foundation for Statistical Computing) and the mada (Meta-Analysis of Diagnostic Accuracy) package58 were used to perform bivariate meta-analysis and produce sROCs. StatsDirect, version 2.7.8 (StatsDirect Ltd, Cheshire, United Kingdom) was used to perform traditional meta-analysis and to generate funnel plots and Eggers weighted regression statistic P values. This report was written to conform to the reporting standards described in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.59

Study Characteristics

Our initial search yielded a total of 5,726 nonduplicate citations. Reasons for exclusion during title and abstract and full-text article review are described in Figure 1. Of the 259 full-text eligible articles reviewed, 40 studies (reporting results of 52 unique patient cohort-clinical prediction rule analyses) reported mortality data for both low- and high-risk classified patients with aPE and were included in the meta-analysis746,60 (Table 1). In total, we identified 11 different clinical prediction rules,7,8,17,19,21,23,27,35,6164 with 36 studies reporting on one clinical prediction rule and four studies reporting data on two or more. e-Appendix 2 details the clinical parameters and scoring methods to determine prognosis in patients with aPE for each of the identified clinical prediction rules, along with a comparison of the parameters used in each tool. A total of 19 patient cohorts reported 2 × 2 all-cause mortality data according to PESI; nine cohorts used the simplified PESI (sPESI); seven used the European Society of Cardiology (ESC) tool; seven used the Geneva tool; four used the tool described by Aujesky et al17 in 2006; and one cohort each used the Global Registry of Acute Coronary Events (GRACE), Hestia, Prognosis in Pulmonary Embolism, Davies 2007,23 Low-Risk Pulmonary Embolism Decision (LR-PED), and Spanish score tools. Most (86%) of the identified cohorts included patients regardless of hemodynamic stability; however, two PESI, two sPESI, one Geneva, and the lone LR-PED cohort included hemodynamically stable patients only and therefore were excluded from the base-case analysis. Of the 11 unique clinical prediction rules, none could be rated a McGinn level 1; four met criteria for McGinn level 2, including PESI, sPESI, ESC, and Geneva; and the remainder were categorized as McGinn level 3 or higher. Included cohort-clinical prediction rule analyses were published between 2004 and 2013 and were performed in various geographic regions, but mainly Europe (70% of cohorts, N = 21,995) and the United States (21% of cohorts, N = 34,358). Mean patient ages in included cohort-clinical prediction rule analyses ranged from 56 to 71 years, and malignancy was observed in 5% to 42% of patients. A majority of cohort-clinical prediction rule analyses used objective, guideline-recommended measures to diagnose an aPE (91%), and 81% used rigorous methods (eg, patient, proxy, or physician interview along with medical record review) to determine survival status during follow-up. We used 30-day mortality (73%) for the majority of included cohort-clinical prediction rule analyses, and 90-day, 7-day, and in-hospital mortality for 17%, 0%, and 10% of cohort-clinical prediction rule analyses, respectively (e-Table 2). Using the QUADAS-2 tool, 45% of cohort-clinical prediction rule analyses were found to have both a low risk of bias and low concerns regarding applicability. The most common reasons for a rating of high risk of bias were issues with patient selection as well as patient flow and timing (Fig 2). Few concerns regarding the applicability of included cohort-clinical prediction rule analyses were noted. Assessment of each QUADAS-2 domain for bias and applicability of each cohort-clinical prediction rule analysis are summarized in e-Figures 1 and 2.

Figure Jump LinkFigure 1 –  Results of the literature search and disposition of citations/articles screened for inclusion. *Low-Risk Pulmonary Embolism Decision (LR-PED) study was performed in hemodynamically stable patients only and was included only in the appropriate sensitivity analysis. aPE = acute pulmonary embolism; CPR = clinical prediction rule.Grahic Jump Location
Table Graphic Jump Location
TABLE 1 ]  Characteristics of Included Studies

CUS = compression ultrasonography; CTPA = CT pulmonary angiography; ESC = European Society of Cardiology; GRACE = Global Registry of Acute Coronary Events; ICD-9 = International Classification of Diseases, Ninth Revision; LR-PED = Low-Risk Pulmonary Embolism Decision; MRA = magnetic resonance angiogram; NR = not reported; P = prospective; PA = pulmonary angiography; PE = pulmonary embolism; PESI = Pulmonary Embolism Severity Index; PREP = prognosis in pulmonary embolism; QUADAS = Quality Assessment of Studies of Diagnostic Accuracy; R = retrospective; RV = right ventricular; sPESI = Simplified Pulmonary Embolism Severity Index; V. /Q.  = ventilation-perfusion

a 

Studies restricting enrollment to hemodynamically stable patients were only included in a sensitivity analysis.

b 

N = 485 for ESC and N = 463 for PESI.

c 

Counted as unique studies even though they were in a single population because multiple unique analyses were performed in multiple unique patient populations.

d 

Included data from four prior studies, three of which excluded patients with shock, and, therefore, this study was excluded from the base-case analysis.

Figure Jump LinkFigure 2 –  A, The proportion of cohort-clinical prediction rule analyses with low, high, or unclear risk of bias for each of the QUADAS-2 domains. B, The proportion of cohort-clinical prediction rule analyses with low, high, or unclear concerns regarding applicability for the “reference standard,” “index test,” and “patient selection” QUADAS-2 domains. By convention, the “flow and timing” domain is not assessed for applicability. QUADAS = Quality Assessment of Diagnostic Accuracy Studies.Grahic Jump Location
Data Synthesis

Per-cohort per-clinical prediction rule measures of accuracy for prognosticating early all-cause mortality are reported in Table 2, and the main results of our bivariate meta-analysis are summarized in Table 3. In addition, e-Figures 3-7 depict supplemental graphical depictions of our results, including weighted crosshair plots and sROCs. The highest sensitivities were observed with the GRACE (0.99, 95% CI = 0.89-1.00), Aujesky 200617 (0.97, 95% CI = 0.95-0.99), sPESI (0.92, 95% CI = 0.89-0.94), PESI (0.89, 95% CI = 0.87-0.90), and ESC (0.88, 95% CI = 0.77-0.94) tools, with the remaining tools having sensitivity values ranging between 0.41 and 0.82 (I2 > 57% for all clinical prediction rules with the exception of PESI [I2 = 6%] (Table 4). When compared with PESI, the sPESI, Aujesky 2006,17 and GRACE tools were found to have a 3% to 11% better sensitivity (P = .04 for sPESI, P < .001 for remaining), whereas the Geneva tool was found to have a 48% (P < .001) and ESC a 1% (P = .83) poorer sensitivity. Of the abovementioned clinical prediction rules with the highest sensitivity, none had a specificity > 48% (I2 > 85% for all). PESI had the highest specificity (0.48, 95% CI = 0.43-0.53), with the sPESI, ESC, Aujesky 2006,17 and GRACE tools demonstrating anywhere from 10% to 24% poorer specificity compared with PESI (P < .02 for all except ESC, P = .13). The probability of publication bias was deemed to be low for all analyses, as suggested by funnel plots and Eggers weight regression statistic P values (P ≥ .22 for all) (e-Figs 8-12).57 Upon performing sensitivity analysis, we did not observe any important changes in conclusions from our base-case analyses.

Table Graphic Jump Location
TABLE 2 ]  Risk of All-Cause Mortality and Clinical Prediction Rule Sensitivity and Specificity

DOR = diagnostic OR; H = in hospital; LR+ = positive likelihood ratio; LR− = negative likelihood ratio; LR-PED = Low Risk Pulmonary Embolism Decision. See Table 1 legend for expansion of other abbreviations.

a 

Represents the hierarchical use of all-cause mortality data at select time points in the meta-analysis (30-d first, followed by 90-d, 7-d, and finally, in-hospital).

b 

Studies restricting enrollment to hemodynamically stable patients were only included in a sensitivity analysis.

Table Graphic Jump Location
TABLE 3 ]  Results of Bivariate and Traditional Meta-analysis

AUC = area under the curve; NC = not calculable. See Table 1 and 2 legends for expansion of other abbreviations.

a 

Pooled using a random-effects approach.

b 

High risk used as referent group.

c 

I2 > 50% (only sensitivity, specificity, DOR, proportion low risk, and OR for death were assessed for statistical heterogeneity).

d 

P value ≤ .05.

e 

P value ≤ .001.

Table Graphic Jump Location
TABLE 4 ]  Results of Sensitivity Analyses

See Table 1 for expansion of abbreviations.

Only meta-analyzed clinical prediction rules with at least four cohorts. Results of LR-PED are also presented in the “including cohorts of stable only patients” as it was the only clinical prediction rule not included in the base-case analysis (only studies in hemodynamically stable patients).

Traditional random-effects meta-analysis suggested up to 82% of patients could be classified as low risk for early all-cause mortality depending on the clinical prediction rule used; however, when the Geneva tool was excluded (because of its poor sensitivity), this value was reduced to 69%. When only the five tools with the highest sensitivity were considered, our meta-analysis suggested 22% to 45% of patients with aPE were at low risk of early all-cause mortality (I2 > 90% for all). Moreover, data based upon cohorts using these same five tools suggest patients deemed to be at low risk had a 77% to 97% lower odds of death compared with those at high risk (I2 = 0% for PESI and ESC, I2 = 43% for sPESI, I2 ≥ 50% for Geneva and Aujesky 200617; all other clinical prediction rules had only one study/cohort).

Our systematic review found 40 studies (52 cohort-clinical prediction rule analyses) reporting on 11 clinical prediction rules for prognosticating early all-cause mortality following an aPE. Upon bivariate meta-analysis, we identified five clinical prediction rules with sensitivities (or true-positive rates) at or near 90% (GRACE, Aujesky 2006,17 sPESI, PESI, and ESC tools) for predicting early all-cause mortality. These tools have high sensitivity and accurately classify essentially all patients who ultimately die following aPE into the high-risk group. However, none demonstrated specificity > 48%, meaning that many patients classified as high risk actually survived the respective study period. Although a perfect clinical prediction rule would be both 100% sensitive and specific, there is an inherent trade-off between the two measures. Given the clinical goal to “do no harm,” high sensitivity when making a decision to discharge early from the hospital or treat on an outpatient basis is preferable to high specificity. This is particularly true, given our meta-analysis’ finding that high-risk patients had at least a fourfold increased odds of early mortality compared with low-risk patients. Finally, despite likely underestimation because of low specificity, the five clinical prediction rules with the highest sensitivities still estimated a rather robust proportion of all-comer patients with aPE (up to 45%) were at low risk of early all-cause mortality.

The American College of Chest Physicians (CHEST)guidelines suggest that in low-risk patients with PE with adequate home conditions (ie, well-maintained living conditions, strong support network, phone access, patient feeling well enough to go home, and the ability to be promptly rehospitalized), early discharge of patients should be considered.3 In addition, guidelines from the ESC61 indicate that prognostic risk scores can be used to identify patients with aPE at low risk of early mortality and that these patients should be considered for early discharge or outpatient treatment. These recommendations are supported by a meta-analysis of randomized trials and observational studies performed by Zondag and colleagues6 demonstrating the pooled incidences of recurrent VTE, major bleeding, or all-cause mortality in low-risk patients with aPE treated at home or discharged early (within 3 days) were not significantly different from those of similar patients treated in-hospital (undergoing similar anticoagulation). A study by Aujesky and colleagues65 that randomized 344 patients with aPE at low risk for early mortality (using the PESI score) to receive outpatient (discharged within 24 h) or inpatient treatment with enoxaparin 1 mg/kg followed by a VKA demonstrated outpatient treatment could effectively and safety be used, as evidenced by the low number of overall adverse clinical outcomes observed in both groups. Moreover, this same study also demonstrated outpatient treatment could reduce the mean length of hospital stay by 3.4 days, which would likely translate to significant cost savings.65,66

Although guidelines encourage the use of clinical prediction rules to identify low-risk patients suitable for outpatient treatment or early discharge, at present, there is no consensus on which tool to use. Although high sensitivity should certainly be a priority, other characteristics of the tools need to be considered as well. Regarding the strength of the total body of evidence supporting the validity of each clinical prediction rule (size, quality, external validity), only the PESI, sPESI, ESC, and Geneva tools meet the criteria for categorization as McGinn level 2 tools (no tool meets the criteria of level 1).52 However, given the low sensitivity of the Geneva tool, its primary use is not advisable. Also of note, only the PESI score has been used to select patients with aPE for home treatment in a randomized clinical trial.65 Ease of use is another factor that merits consideration, since these tools will need to be operationalized in busy ED settings. Many of the identified clinical prediction rules, such as the PESI and LR-PED tools, are more difficult to score (eg, PESI incorporates 11 variables worth varying point values, LR-PED requires a calculator or application to compute its complex formula) when compared with tools such as sPESI and ESC.

Although all the evaluated clinical prediction rules in our meta-analysis are suitable for use in clinical practice or as part of a randomized trial, none are likely useful in performing a claims database analysis of outpatient or early discharge of aPE because of their need for clinical characteristics and test results (ie, systolic BP, heart rate, respiratory rate, temperature, oxygen saturation, echocardiography, troponin, C-reactive protein and serum creatinine levels, and so on) not typically found in such databases. Although not considered in our meta-analysis, the Charlson Comorbidity Index (CCI), a score measuring disease burden based on the number of comorbidities present, was shown in a study of 1,023 patients with aPE to exhibit good sensitivity in predicting early all-cause mortality (30-day mortality was 1 of 351 in those with a CCI = 0 and 40 of 672 in those with a CCI ≥ 1; sensitivity = 0.98, specificity = 0.36).67

Current aPE treatment guidelines recommend the initiation of heparin with a VKA overlap for a minimum of 5 days and until a therapeutic international normalized ratio has been achieved for at least 24 h,3 and it is likely the difficulty in achieving stable therapeutic international normalized ratios early on in patients with VTE68 that prevents many clinicians from treating them as outpatients or discharging them early. There are now three novel oral anticoagulants (rivaroxaban, apixaban, dabigatran) that have received Food and Drug Administration approval for the treatment of VTE including aPE and prevention of recurrent events. Because of their rapid onset of action, predictable pharmacokinetics negating the need for monitoring, and lack of need for heparin bridging therapy, these agents should simplify anticoagulation for patients with aPE by making outpatient or early discharge more feasible. In fact, in the EINSTEIN-Pulmonary Embolism trial, patients receiving rivaroxaban were discharged a median of 1 day earlier than those receiving enoxaparin followed by a VKA (median 7 days vs 6 days, P = .0001), even though the study protocol did not mandate or encourage early discharge based upon the medication to which subjects were randomized.69

Our meta-analysis has a number of limitations worth discussion. First, our meta-analysis assessed early all-cause mortality as the end point of interest, and not recurrent VTE, major bleeding, or other adverse events. Early mortality was chosen as the primary end point because the majority of clinical prediction rules were initially derived and validated on this end point. Unfortunately, far fewer studies reported data on recurrent VTE and major bleeding, and although the reporting of a composite end point of adverse events was somewhat common in studies, the constituents of the composite were often heterogeneous between studies. Second, our systematic review was limited to publications in English because our research group did not have multiple members capable of reading languages other than English. Analyses have suggested “language bias” resulting from excluding non-English language studies generally has minimal effect on summary estimates in meta-analyses.70,71 In addition, the probability of publication bias was deemed to be low for all analyses in our meta-analysis, as suggested by funnel plots and Eggers weight-regression statistics. Third, the comprehensive search strategy used in this meta-analysis started in the year 2000. This was done to limit the inclusion of studies not using modern aPE diagnostic and treatment modalities. Although the year 2000 could be viewed as somewhat arbitrary, we chose it because it marked the time at which evidence-based guidelines48,49 incorporating the first, to our knowledge, well-done randomized trials of heparin and VKAs became available. The year 2000 cutoff is further supported by a study by Kobberøe Søgaard and colleagues72 that suggested early aPE mortality has decreased significantly over time, with early mortality rate ratios declining from 81.97 (95% CI = 72.27-92.98) in 1990 to 1999 to 36.08 (95% CI = 32.65-39.87) in 2000 to 2011. Next, it is important to note that other clinical prediction rules for assessing post-aPE prognosis exist that were not included in this meta-analysis. Some clinical prediction rules were not included because they did not report data on early all-cause mortality as an end point or in a useable fashion,64,7375 because they focused on patients with aPE with cancer only76,77 or because they were presented in abstract form only.78,79 These clinical prediction rules may, however, have merit, and we hope that future studies to validate them will be performed. A final limitation of our meta-analysis is our inclusion of in-hospital through 90-day all-cause mortality data in our definition of “early” mortality. Although the ESC defines early mortality as ≤ 30 days61 (as this timing is consistent with the majority of recurrent events80), our sensitivity analysis suggests the inclusion of 90-day mortality did not substantially alter our results.

Numerous clinical prediction rules for prognosticating early all-cause mortality in patients with aPE are available, but not all demonstrate the high sensitivity needed to reassure clinicians. Given their higher sensitivities and stronger literature base, the PESI, sPESI, and ESC tools appear the most reliable for identifying low-risk patients suitable for outpatient treatment or early discharge. However, ease of use should also be considered, suggesting the sPESI or ESC tools may be best suited for use in everyday clinical practice.

Author contributions: C. G. K. and C. I. C. had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis and contributed equally to the performance and publication of the meta-analysis. C. G. K. contributed to study concept and design, acquisition, analysis, or interpretation of data, drafting of the manuscript, statistical analysis, and critical revision of the manuscript for important intellectual content; E. S. M. and M. W. P. contributed to acquisition, analysis, or interpretation of data and critical revision of the manuscript for important intellectual content; A. V. H. contributed to acquisition, analysis, or interpretation of data, statistical analysis, and critical revision of the manuscript for important intellectual content; and C. I. C. contributed to obtaining funding, study supervision, study concept and design, acquisition, analysis, or interpretation of data, drafting of the manuscript, statistical analysis, and critical revision of the manuscript for important intellectual content.

Financial/nonfinancial disclosures: The authors have reported to CHEST the following conflicts of interest: Dr Coleman has received funding from Janssen Scientific Affairs, LLC, and is a member of their Speakers’ Bureau for Xarelto. Drs Kohn, Mearns, Parker, and Hernandez have reported that no potential conflicts of interest exist with any companies/organizations whose products or services may be discussed in this article.

Role of sponsors: Janssen Scientific Affairs, LLC had no role in the design and conduct of the study; collection, management, analysis, interpretation of the data; preparation or approval of the manuscript; or decision to submit the manuscript for publication. Janssen Scientific Affairs, LLC did review the manuscript prior to submission but made no revision.

Other contributions: We thank Savaş Özsu, MD, for providing unpublished data for use in this meta-analysis.

Additional information: The e-Appendix, e-Figures, and e-Tables can be found in the Supplemental Materials section of the online article.

aPE

acute pulmonary embolism

CCI

Charlson Comorbidity Index

DOR

diagnostic OR

ESC

European Society of Cardiology

GRACE

Global Registry of Acute Coronary Events

LR

likelihood ratio

LR-PED

Low-Risk Pulmonary Embolism Decision

PESI

Pulmonary Embolism Severity Index

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

QUADAS

Quality Assessment of Diagnostic Accuracy Studies

ROC

receiver operator curve

sPESI

Simplified Pulmonary Embolism Severity Index

sROC

summary receiver operator curve

VKA

vitamin K antagonist

Silverstein MD, Heit JA, Mohr DN, Petterson TM, O’Fallon WM, Melton LJ III. Trends in the incidence of deep vein thrombosis and pulmonary embolism: a 25-year population-based study. Arch Intern Med. 1998;158(6):585-593. [CrossRef] [PubMed]
 
Oger E. Incidence of venous thromboembolism: a community-based study in Western France. EPI-GETBP Study Group. Groupe d’Etude de la Thrombose de Bretagne Occidentale. Thromb Haemost. 2000;83(5):657-660. [PubMed]
 
Kearon C, Akl EA, Comerota AJ, et al. Antithrombotic therapy for VTE disease: antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2012;141(2_suppl):e419S-e494S. [CrossRef] [PubMed]
 
Douketis JD, Kearon C, Bates S, Duku EK, Ginsberg JS. Risk of fatal pulmonary embolism in patients with treated venous thromboembolism. JAMA. 1998;279(6):458-462. [CrossRef] [PubMed]
 
Van Bellen B, Prins M, Bamber L, Wang M, Lensing AWA. Reduction in initial length of stay with rivaroxaban single-drug regimen versus LMWH-VKA standard of care: findings from the EINSTEIN trial program. Presented at 54th Annual American Society of Hematology Meeting; December 10, 2012; Atlanta, GA.
 
Zondag W, Kooiman J, Klok FA, Dekkers OM, Huisman MV. Outpatient versus inpatient treatment in patients with pulmonary embolism: a meta-analysis. Eur Respir J. 2013;42(1):134-144. [CrossRef] [PubMed]
 
Aujesky D, Obrosky DS, Stone RA, et al. Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med. 2005;172(8):1041-1046. [CrossRef] [PubMed]
 
Jiménez D, Aujesky D, Moores L, et al; RIETE Investigators. Simplification of the pulmonary embolism severity index for prognostication in patients with acute symptomatic pulmonary embolism. Arch Intern Med. 2010;170(15):1383-1389. [CrossRef] [PubMed]
 
Zondag W, den Exter PL, Crobach MJ, et al; Hestia Study Investigators. Comparison of two methods for selection of out of hospital treatment in patients with acute pulmonary embolism. Thromb Haemost. 2013;109(1):47-52. [CrossRef] [PubMed]
 
Ozsu S, Abul Y, Orem A, et al. Predictive value of troponins and simplified pulmonary embolism severity index in patients with normotensive pulmonary embolism. Multidiscip Respir Med. 2013;8(1):34. [CrossRef] [PubMed]
 
Lankeit M, Jiménez D, Kostrubiec M, et al. Predictive value of the high-sensitivity troponin T assay and the simplified Pulmonary Embolism Severity Index in hemodynamically stable patients with acute pulmonary embolism: a prospective validation study. Circulation. 2011;124(24):2716-2724. [CrossRef] [PubMed]
 
Nordenholz K, Ryan J, Atwood B, Heard K. Pulmonary embolism risk stratification: pulse oximetry and pulmonary embolism severity index. J Emerg Med. 2011;40(1):95-102. [CrossRef] [PubMed]
 
Sam A, Sánchez D, Gómez V, et al. The shock index and the simplified PESI for identification of low-risk patients with acute pulmonary embolism. Eur Respir J. 2011;37(4):762-766. [CrossRef] [PubMed]
 
Sánchez D, De Miguel J, Sam A, et al. The effects of cause of death classification on prognostic assessment of patients with pulmonary embolism. J Thromb Haemost. 2011;9(11):2201-2207. [CrossRef] [PubMed]
 
Chan CM, Woods C, Shorr AF. The validation and reproducibility of the pulmonary embolism severity index. J Thromb Haemost. 2010;8(7):1509-1514. [CrossRef] [PubMed]
 
Vanni S, Nazerian P, Pepe G, et al. Comparison of two prognostic models for acute pulmonary embolism: clinical vs right ventricular dysfunction-guided approach. J Thromb Haemost. 2011;9(10):1916-1923. [CrossRef] [PubMed]
 
Aujesky D, Obrosky DS, Stone RA, et al. A prediction rule to identify low-risk patients with pulmonary embolism. Arch Intern Med. 2006;166(2):169-175. [CrossRef] [PubMed]
 
Aujesky D, Perrier A, Roy PM, et al. Validation of a clinical prognostic model to identify low-risk patients with pulmonary embolism. J Intern Med. 2007;261(6):597-604. [CrossRef] [PubMed]
 
Barra S, Paiva L, Providência R, Fernandes A, Nascimento J, Marques AL. LR-PED rule: low risk pulmonary embolism decision rule - a new decision score for low risk pulmonary embolism. Thromb Res. 2012;130(3):327-333. [CrossRef] [PubMed]
 
Bova C, Pesavento R, Marchiori A, et al; TELESIO Study Group. Risk stratification and outcomes in hemodynamically stable patients with acute pulmonary embolism: a prospective, multicentre, cohort study with three months of follow-up. J Thromb Haemost. 2009;7(6):938-944. [CrossRef] [PubMed]
 
Chan CM, Woods CJ, Shorr AF. Comparing the pulmonary embolism severity index and the prognosis in pulmonary embolism scores as risk stratification tools. J Hosp Med. 2012;7(1):22-27. [CrossRef] [PubMed]
 
Choi WH, Kwon SU, Jwa YJ, et al. The pulmonary embolism severity index in predicting the prognosis of patients with pulmonary embolism. Korean J Intern Med. 2009;24(2):123-127. [CrossRef] [PubMed]
 
Davies CW, Wimperis J, Green ES, et al. Early discharge of patients with pulmonary embolism: a two-phase observational study. Eur Respir J. 2007;30(4):708-714. [CrossRef] [PubMed]
 
Dentali F, Riva N, Turato S, et al. Pulmonary embolism severity index accurately predicts long-term mortality rate in patients hospitalized for acute pulmonary embolism. J Thromb Haemost. 2013;11(12):2103-2110. [CrossRef] [PubMed]
 
Donzé J, Le Gal G, Fine MJ, et al. Prospective validation of the Pulmonary Embolism Severity Index. A clinical prognostic model for pulmonary embolism. Thromb Haemost. 2008;100(5):943-948. [PubMed]
 
Erkens PM, Gandara E, Wells PS, et al. Does the Pulmonary Embolism Severity Index accurately identify low risk patients eligible for outpatient treatment? Thromb Res. 2012;129(6):710-714. [CrossRef] [PubMed]
 
Ferrer M, Morillo R, Elías T, et al. Validation of two clinical prognostic models in patients with acute symptomatic pulmonary embolism. Arch Bronconeumol. 2013;49(10):427-431. [CrossRef] [PubMed]
 
Jakobsson C, Jiménez D, Gómez V, Zamarro C, Méan M, Aujesky D. Validation of a clinical algorithm to identify low-risk patients with pulmonary embolism. J Thromb Haemost. 2010;8(6):1242-1247. [CrossRef] [PubMed]
 
Jiménez D, Yusen RD, Otero R, et al. Prognostic models for selecting patients with acute pulmonary embolism for initial outpatient therapy. Chest. 2007;132(1):24-30. [CrossRef] [PubMed]
 
Kukla P, Bryniarski L, Długopolski R, et al. Acute pulmonary embolism registry in the Małopolska region - clinical course. Kardiol Pol. 2009;67(7):735-741. [PubMed]
 
Labyk A, Ciurzyński M, Jankowski K, et al. Acute pulmonary embolism: analysis of consecutive 353 patients hospitalised in a single centre. A 3-year experience. Kardiol Pol. 2012;70(1):15-22. [PubMed]
 
Lankeit M, Gómez V, Wagner C, et al; on behalf of the Instituto Ramón y Cajal de Investigación Sanitaria Pulmonary Embolism Study Group. A strategy combining imaging and laboratory biomarkers in comparison with a simplified clinical score for risk stratification of patients with acute pulmonary embolism. Chest. 2012;141(4):916-922. [CrossRef] [PubMed]
 
Nendaz MR, Bandelier P, Aujesky D, et al. Validation of a risk score identifying patients with acute pulmonary embolism, who are at low risk of clinical adverse outcome. Thromb Haemost. 2004;91(6):1232-1236. [PubMed]
 
Palmieri V, Gallotta G, Rendina D, et al. Troponin I and right ventricular dysfunction for risk assessment in patients with nonmassive pulmonary embolism in the Emergency Department in combination with clinically based risk score. Intern Emerg Med. 2008;3(2):131-138. [CrossRef] [PubMed]
 
Paiva LV, Providencia RC, Barra SN, Faustino AC, Botelho AM, Marques AL. Cardiovascular risk assessment of pulmonary embolism with the GRACE risk score. Am J Cardiol. 2013;111(3):425-431. [CrossRef] [PubMed]
 
Pollack CV, Schreiber D, Goldhaber SZ, et al. Clinical characteristics, management, and outcomes of patients diagnosed with acute pulmonary embolism in the emergency department: initial report of EMPEROR (Multicenter Emergency Medicine Pulmonary Embolism in the Real World Registry). J Am Coll Cardiol. 2011;57(6):700-706. [CrossRef] [PubMed]
 
Post F, Mertens D, Sinning C, Peetz D, Münzel T. Decision for aggressive therapy in acute pulmonary embolism: implication of elevated troponin T. Clin Res Cardiol. 2009;98(6):401-408. [CrossRef] [PubMed]
 
Sanchez O, Trinquart L, Planquette B, et al. Echocardiography and pulmonary embolism severity index have independent prognostic roles in pulmonary embolism. Eur Respir J. 2013;42(3):681-688. [CrossRef] [PubMed]
 
Singanayagam A, Chalmers JD, Scally C, et al. Right ventricular dilation on CT pulmonary angiogram independently predicts mortality in pulmonary embolism. Respir Med. 2010;104(7):1057-1062. [CrossRef] [PubMed]
 
Soares TH, de Bastos M, de Carvalho BV, et al. Prognostic value of computed tomographic pulmonary angiography and the pulmonary embolism severity index in patients with acute pulmonary embolism. Blood Coagul Fibrinolysis. 2013;24(1):64-70. [CrossRef] [PubMed]
 
Spencer FA, Goldberg RJ, Lessard D, et al. Factors associated with adverse outcomes in outpatients presenting with pulmonary embolism: the Worcester Venous Thromboembolism Study. Circ Cardiovasc Qual Outcomes. 2010;3(4):390-394. [CrossRef] [PubMed]
 
Spirk D, Aujesky D, Husmann M, et al. Cardiac troponin testing and the simplified Pulmonary Embolism Severity Index. The SWIss Venous ThromboEmbolism Registry (SWIVTER). Thromb Haemost. 2011;106(5):978-984. [CrossRef] [PubMed]
 
Subramaniam RM, Mandrekar J, Blair D, Peller PJ, Karalus N. The Geneva prognostic score and mortality in patients diagnosed with pulmonary embolism by CT pulmonary angiogram. J Med Imaging Radiat Oncol. 2009;53(4):361-365. [CrossRef] [PubMed]
 
Venetz C, Jiménez D, Mean M, Aujesky D. A comparison of the original and simplified Pulmonary Embolism Severity Index. Thromb Haemost. 2011;106(3):423-428. [CrossRef] [PubMed]
 
Zondag W, Vingerhoets LM, Durian MF, et al; Hestia Study Investigators. Hestia criteria can safely select patients with pulmonary embolism for outpatient treatment irrespective of right ventricular function. J Thromb Haemost. 2013;11(4):686-692. [CrossRef] [PubMed]
 
Zwierzina D, Limacher A, Méan M, et al. Prospective comparison of clinical prognostic scores in elder patients with a pulmonary embolism. J Thromb Haemost. 2012;10(11):2270-2276. [CrossRef] [PubMed]
 
Goldhaber SZ. Modern treatment of pulmonary embolism. Eur Respir J Suppl. 2002;35:22s-27s. [CrossRef] [PubMed]
 
Torbicki A, van Beek EJR, Charbonnier B, et al. Guidelines on diagnosis and management of acute pulmonary embolism. Task Force on Pulmonary Embolism, European Society of Cardiology. Eur Heart J. 2000;21(16):1301-1336. [CrossRef] [PubMed]
 
Dalen JE, Hirsh J. Introduction: antithrombotic therapy—the evolving consensus: 1985 to 1998. Chest. 1998;114(5_suppl):439S-440S. [CrossRef] [PubMed]
 
Geersing GJ, Bouwmeester W, Zuithoff P, Spijker R, Leeflang M, Moons KG. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS ONE. 2012;7(2):e32844. [CrossRef] [PubMed]
 
Whiting PF, Rutjes AWS, Westwood ME, et al; QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. [CrossRef] [PubMed]
 
McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS; Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. JAMA. 2000;284(1):79-84. [CrossRef] [PubMed]
 
Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990. [CrossRef] [PubMed]
 
Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Chapter 10: analysing and presenting results.. In:Deeks JJ, Bossuyt PM, Gatsonis C., eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Version 1.0. The Cochrane Collaboration; 2010. http://srdta.cochrane.org/. Accessed October 2014.
 
Trikalinos TA, Balion CM, Coleman CI, et al. Chapter 8: meta-analysis of test performance when there is a “gold standard.” J Gen Intern Med. 2012;27(suppl 1):S56-S66. [CrossRef] [PubMed]
 
Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ. 2003;326(7382):219. [CrossRef] [PubMed]
 
Higgins JPT, Green S., eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. http://srdta.cochrane.org/. Accessed October 2014.
 
Doebler P, Holling H. Meta-analysis of diagnostic accuracy (mada). R package version 0.5.4. http://CRAN.R-project.org/package=mada. Published 2013. Accessed September 2014.
 
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264-269. [CrossRef] [PubMed]
 
Aujesky D, Roy PM, Le Manach CP, et al. Validation of a model to predict adverse outcomes in patients with pulmonary embolism. Eur Heart J. 2006;27(4):476-481. [CrossRef] [PubMed]
 
Torbicki A, Perrier A, Konstantinides S, et al; ESC Committee for Practice Guidelines (CPG). Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J. 2008;29(18):2276-2315. [CrossRef] [PubMed]
 
Wicki J, Perrier A, Perneger TV, Bounameaux H, Junod AF. Predicting adverse outcome in patients with acute pulmonary embolism: a risk score. Thromb Haemost. 2000;84(4):548-552. [PubMed]
 
Zondag W, Mos IC, Creemers-Schild D, et al; Hestia Study Investigators. Outpatient treatment in patients with acute pulmonary embolism: the Hestia Study. J Thromb Haemost. 2011;9(8):1500-1507. [CrossRef] [PubMed]
 
Uresandi F, Otero R, Cayuela A, et al. A clinical prediction rule for identifying short-term risk of adverse events in patients with pulmonary thromboembolism [in Spanish]. Arch Bronconeumol. 2007;43(11):617-622. [CrossRef] [PubMed]
 
Aujesky D, Roy PM, Verschuren F, et al. Outpatient versus inpatient treatment for patients with acute pulmonary embolism: an international, open-label, randomised, non-inferiority trial. Lancet. 2011;378(9785):41-48. [CrossRef] [PubMed]
 
Aujesky D, Smith KJ, Cornuz J, Roberts MS. Cost-effectiveness of low-molecular-weight heparin for treatment of pulmonary embolism. Chest. 2005;128(3):1601-1610. [CrossRef] [PubMed]
 
Ng AC, Chow V, Yong AS, Chung T, Kritharides L. Prognostic impact of the Charlson comorbidity index on mortality following acute pulmonary embolism. Respiration. 2013;85(5):408-416. [CrossRef] [PubMed]
 
Erkens PM, ten Cate H, Büller HR, Prins MH. Benchmark for time in therapeutic range in venous thromboembolism: a systematic review and meta-analysis. PLoS ONE. 2012;7(9):e42269. [CrossRef] [PubMed]
 
van Bellen B, Bamber L, Correa de Carvalho F, Prins M, Wang M, Lensing AW. Reduction in the length of stay with rivaroxaban as a single-drug regimen for the treatment of deep vein thrombosis and pulmonary embolism. Curr Med Res Opin. 2014;30(5):829-837. [CrossRef] [PubMed]
 
Moher D, Pham B, Klassen TP, et al. What contributions do languages other than English make on the results of meta-analyses? J Clin Epidemiol. 2000;53(9):964-972. [CrossRef] [PubMed]
 
Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002;31(1):115-123. [CrossRef] [PubMed]
 
Kobberøe Søgaard K, Schmidt M, Pedersen L, Horváth-Puhó E, Toft Sørensen H. 30-year mortality after venous thromboembolism: a population-based cohort study. Circulation. 2014;130(10):829-836. [CrossRef] [PubMed]
 
Yamaki T, Nozaki M, Sakurai H, Takeuchi M, Soejima K, Kono T. Presence of lower limb deep vein thrombosis and prognosis in patients with symptomatic pulmonary embolism: preliminary report. Eur J Vasc Endovasc Surg. 2009;37(2):225-231. [CrossRef] [PubMed]
 
Volschan A, Albuquerque D, Tura BR, et al; EMEP (Estudo Multicêntrico de Embolia Pulmonar) investigators. Predictors of hospital mortality in hemodynamically stable patients with pulmonary embolism. Arq Bras Cardiol. 2009;93(2):135-140. [CrossRef] [PubMed]
 
Agterof MJ, Schutgens RE, Moumli N, et al. A prognostic model for short term adverse events in normotensive patients with pulmonary embolism. Am J Hematol. 2011;86(8):646-649. [CrossRef] [PubMed]
 
den Exter PL, Gómez V, Jiménez D, et al; Registro Informatizado de la Enfermedad TromboEmbólica (RIETE) Investigators. A clinical prognostic model for the identification of low-risk patients with acute symptomatic pulmonary embolism and active cancer. Chest. 2013;143(1):138-145. [CrossRef] [PubMed]
 
Kline JA, Roy PM, Than MP, et al. Derivation and validation of a multivariate model to predict mortality from pulmonary embolism with cancer: The POMPE-C tool. Thromb Res. 2012;129(5):e194-e199. [CrossRef] [PubMed]
 
Leal SRN, Dores H, Rosario I, et al. Acute pulmonary embolism: development of early and long term morbidity and mortality risk score. Eur Heart J. 2010;31(suppl 1):625. [CrossRef] [PubMed]
 
Murugappan M, Johnson JA, Gage BF, et al. HOme Management Exclusion (HOME) criteria for initial treatment of acute pulmonary embolism [abstract]. Am J Respir Crit Care Med. 2008;177:A182.
 
Limone BL, Hernandez AV, Michalak D, Bookhart BK, Coleman CI. Timing of recurrent venous thromboembolism early after the index event: a meta-analysis of randomized controlled trials. Thromb Res. 2013;132(4):420-426. [CrossRef] [PubMed]
 

Figures

Figure Jump LinkFigure 1 –  Results of the literature search and disposition of citations/articles screened for inclusion. *Low-Risk Pulmonary Embolism Decision (LR-PED) study was performed in hemodynamically stable patients only and was included only in the appropriate sensitivity analysis. aPE = acute pulmonary embolism; CPR = clinical prediction rule.Grahic Jump Location
Figure Jump LinkFigure 2 –  A, The proportion of cohort-clinical prediction rule analyses with low, high, or unclear risk of bias for each of the QUADAS-2 domains. B, The proportion of cohort-clinical prediction rule analyses with low, high, or unclear concerns regarding applicability for the “reference standard,” “index test,” and “patient selection” QUADAS-2 domains. By convention, the “flow and timing” domain is not assessed for applicability. QUADAS = Quality Assessment of Diagnostic Accuracy Studies.Grahic Jump Location

Tables

Table Graphic Jump Location
TABLE 1 ]  Characteristics of Included Studies

CUS = compression ultrasonography; CTPA = CT pulmonary angiography; ESC = European Society of Cardiology; GRACE = Global Registry of Acute Coronary Events; ICD-9 = International Classification of Diseases, Ninth Revision; LR-PED = Low-Risk Pulmonary Embolism Decision; MRA = magnetic resonance angiogram; NR = not reported; P = prospective; PA = pulmonary angiography; PE = pulmonary embolism; PESI = Pulmonary Embolism Severity Index; PREP = prognosis in pulmonary embolism; QUADAS = Quality Assessment of Studies of Diagnostic Accuracy; R = retrospective; RV = right ventricular; sPESI = Simplified Pulmonary Embolism Severity Index; V. /Q.  = ventilation-perfusion

a 

Studies restricting enrollment to hemodynamically stable patients were only included in a sensitivity analysis.

b 

N = 485 for ESC and N = 463 for PESI.

c 

Counted as unique studies even though they were in a single population because multiple unique analyses were performed in multiple unique patient populations.

d 

Included data from four prior studies, three of which excluded patients with shock, and, therefore, this study was excluded from the base-case analysis.

Table Graphic Jump Location
TABLE 2 ]  Risk of All-Cause Mortality and Clinical Prediction Rule Sensitivity and Specificity

DOR = diagnostic OR; H = in hospital; LR+ = positive likelihood ratio; LR− = negative likelihood ratio; LR-PED = Low Risk Pulmonary Embolism Decision. See Table 1 legend for expansion of other abbreviations.

a 

Represents the hierarchical use of all-cause mortality data at select time points in the meta-analysis (30-d first, followed by 90-d, 7-d, and finally, in-hospital).

b 

Studies restricting enrollment to hemodynamically stable patients were only included in a sensitivity analysis.

Table Graphic Jump Location
TABLE 3 ]  Results of Bivariate and Traditional Meta-analysis

AUC = area under the curve; NC = not calculable. See Table 1 and 2 legends for expansion of other abbreviations.

a 

Pooled using a random-effects approach.

b 

High risk used as referent group.

c 

I2 > 50% (only sensitivity, specificity, DOR, proportion low risk, and OR for death were assessed for statistical heterogeneity).

d 

P value ≤ .05.

e 

P value ≤ .001.

Table Graphic Jump Location
TABLE 4 ]  Results of Sensitivity Analyses

See Table 1 for expansion of abbreviations.

Only meta-analyzed clinical prediction rules with at least four cohorts. Results of LR-PED are also presented in the “including cohorts of stable only patients” as it was the only clinical prediction rule not included in the base-case analysis (only studies in hemodynamically stable patients).

References

Silverstein MD, Heit JA, Mohr DN, Petterson TM, O’Fallon WM, Melton LJ III. Trends in the incidence of deep vein thrombosis and pulmonary embolism: a 25-year population-based study. Arch Intern Med. 1998;158(6):585-593. [CrossRef] [PubMed]
 
Oger E. Incidence of venous thromboembolism: a community-based study in Western France. EPI-GETBP Study Group. Groupe d’Etude de la Thrombose de Bretagne Occidentale. Thromb Haemost. 2000;83(5):657-660. [PubMed]
 
Kearon C, Akl EA, Comerota AJ, et al. Antithrombotic therapy for VTE disease: antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2012;141(2_suppl):e419S-e494S. [CrossRef] [PubMed]
 
Douketis JD, Kearon C, Bates S, Duku EK, Ginsberg JS. Risk of fatal pulmonary embolism in patients with treated venous thromboembolism. JAMA. 1998;279(6):458-462. [CrossRef] [PubMed]
 
Van Bellen B, Prins M, Bamber L, Wang M, Lensing AWA. Reduction in initial length of stay with rivaroxaban single-drug regimen versus LMWH-VKA standard of care: findings from the EINSTEIN trial program. Presented at 54th Annual American Society of Hematology Meeting; December 10, 2012; Atlanta, GA.
 
Zondag W, Kooiman J, Klok FA, Dekkers OM, Huisman MV. Outpatient versus inpatient treatment in patients with pulmonary embolism: a meta-analysis. Eur Respir J. 2013;42(1):134-144. [CrossRef] [PubMed]
 
Aujesky D, Obrosky DS, Stone RA, et al. Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med. 2005;172(8):1041-1046. [CrossRef] [PubMed]
 
Jiménez D, Aujesky D, Moores L, et al; RIETE Investigators. Simplification of the pulmonary embolism severity index for prognostication in patients with acute symptomatic pulmonary embolism. Arch Intern Med. 2010;170(15):1383-1389. [CrossRef] [PubMed]
 
Zondag W, den Exter PL, Crobach MJ, et al; Hestia Study Investigators. Comparison of two methods for selection of out of hospital treatment in patients with acute pulmonary embolism. Thromb Haemost. 2013;109(1):47-52. [CrossRef] [PubMed]
 
Ozsu S, Abul Y, Orem A, et al. Predictive value of troponins and simplified pulmonary embolism severity index in patients with normotensive pulmonary embolism. Multidiscip Respir Med. 2013;8(1):34. [CrossRef] [PubMed]
 
Lankeit M, Jiménez D, Kostrubiec M, et al. Predictive value of the high-sensitivity troponin T assay and the simplified Pulmonary Embolism Severity Index in hemodynamically stable patients with acute pulmonary embolism: a prospective validation study. Circulation. 2011;124(24):2716-2724. [CrossRef] [PubMed]
 
Nordenholz K, Ryan J, Atwood B, Heard K. Pulmonary embolism risk stratification: pulse oximetry and pulmonary embolism severity index. J Emerg Med. 2011;40(1):95-102. [CrossRef] [PubMed]
 
Sam A, Sánchez D, Gómez V, et al. The shock index and the simplified PESI for identification of low-risk patients with acute pulmonary embolism. Eur Respir J. 2011;37(4):762-766. [CrossRef] [PubMed]
 
Sánchez D, De Miguel J, Sam A, et al. The effects of cause of death classification on prognostic assessment of patients with pulmonary embolism. J Thromb Haemost. 2011;9(11):2201-2207. [CrossRef] [PubMed]
 
Chan CM, Woods C, Shorr AF. The validation and reproducibility of the pulmonary embolism severity index. J Thromb Haemost. 2010;8(7):1509-1514. [CrossRef] [PubMed]
 
Vanni S, Nazerian P, Pepe G, et al. Comparison of two prognostic models for acute pulmonary embolism: clinical vs right ventricular dysfunction-guided approach. J Thromb Haemost. 2011;9(10):1916-1923. [CrossRef] [PubMed]
 
Aujesky D, Obrosky DS, Stone RA, et al. A prediction rule to identify low-risk patients with pulmonary embolism. Arch Intern Med. 2006;166(2):169-175. [CrossRef] [PubMed]
 
Aujesky D, Perrier A, Roy PM, et al. Validation of a clinical prognostic model to identify low-risk patients with pulmonary embolism. J Intern Med. 2007;261(6):597-604. [CrossRef] [PubMed]
 
Barra S, Paiva L, Providência R, Fernandes A, Nascimento J, Marques AL. LR-PED rule: low risk pulmonary embolism decision rule - a new decision score for low risk pulmonary embolism. Thromb Res. 2012;130(3):327-333. [CrossRef] [PubMed]
 
Bova C, Pesavento R, Marchiori A, et al; TELESIO Study Group. Risk stratification and outcomes in hemodynamically stable patients with acute pulmonary embolism: a prospective, multicentre, cohort study with three months of follow-up. J Thromb Haemost. 2009;7(6):938-944. [CrossRef] [PubMed]
 
Chan CM, Woods CJ, Shorr AF. Comparing the pulmonary embolism severity index and the prognosis in pulmonary embolism scores as risk stratification tools. J Hosp Med. 2012;7(1):22-27. [CrossRef] [PubMed]
 
Choi WH, Kwon SU, Jwa YJ, et al. The pulmonary embolism severity index in predicting the prognosis of patients with pulmonary embolism. Korean J Intern Med. 2009;24(2):123-127. [CrossRef] [PubMed]
 
Davies CW, Wimperis J, Green ES, et al. Early discharge of patients with pulmonary embolism: a two-phase observational study. Eur Respir J. 2007;30(4):708-714. [CrossRef] [PubMed]
 
Dentali F, Riva N, Turato S, et al. Pulmonary embolism severity index accurately predicts long-term mortality rate in patients hospitalized for acute pulmonary embolism. J Thromb Haemost. 2013;11(12):2103-2110. [CrossRef] [PubMed]
 
Donzé J, Le Gal G, Fine MJ, et al. Prospective validation of the Pulmonary Embolism Severity Index. A clinical prognostic model for pulmonary embolism. Thromb Haemost. 2008;100(5):943-948. [PubMed]
 
Erkens PM, Gandara E, Wells PS, et al. Does the Pulmonary Embolism Severity Index accurately identify low risk patients eligible for outpatient treatment? Thromb Res. 2012;129(6):710-714. [CrossRef] [PubMed]
 
Ferrer M, Morillo R, Elías T, et al. Validation of two clinical prognostic models in patients with acute symptomatic pulmonary embolism. Arch Bronconeumol. 2013;49(10):427-431. [CrossRef] [PubMed]
 
Jakobsson C, Jiménez D, Gómez V, Zamarro C, Méan M, Aujesky D. Validation of a clinical algorithm to identify low-risk patients with pulmonary embolism. J Thromb Haemost. 2010;8(6):1242-1247. [CrossRef] [PubMed]
 
Jiménez D, Yusen RD, Otero R, et al. Prognostic models for selecting patients with acute pulmonary embolism for initial outpatient therapy. Chest. 2007;132(1):24-30. [CrossRef] [PubMed]
 
Kukla P, Bryniarski L, Długopolski R, et al. Acute pulmonary embolism registry in the Małopolska region - clinical course. Kardiol Pol. 2009;67(7):735-741. [PubMed]
 
Labyk A, Ciurzyński M, Jankowski K, et al. Acute pulmonary embolism: analysis of consecutive 353 patients hospitalised in a single centre. A 3-year experience. Kardiol Pol. 2012;70(1):15-22. [PubMed]
 
Lankeit M, Gómez V, Wagner C, et al; on behalf of the Instituto Ramón y Cajal de Investigación Sanitaria Pulmonary Embolism Study Group. A strategy combining imaging and laboratory biomarkers in comparison with a simplified clinical score for risk stratification of patients with acute pulmonary embolism. Chest. 2012;141(4):916-922. [CrossRef] [PubMed]
 
Nendaz MR, Bandelier P, Aujesky D, et al. Validation of a risk score identifying patients with acute pulmonary embolism, who are at low risk of clinical adverse outcome. Thromb Haemost. 2004;91(6):1232-1236. [PubMed]
 
Palmieri V, Gallotta G, Rendina D, et al. Troponin I and right ventricular dysfunction for risk assessment in patients with nonmassive pulmonary embolism in the Emergency Department in combination with clinically based risk score. Intern Emerg Med. 2008;3(2):131-138. [CrossRef] [PubMed]
 
Paiva LV, Providencia RC, Barra SN, Faustino AC, Botelho AM, Marques AL. Cardiovascular risk assessment of pulmonary embolism with the GRACE risk score. Am J Cardiol. 2013;111(3):425-431. [CrossRef] [PubMed]
 
Pollack CV, Schreiber D, Goldhaber SZ, et al. Clinical characteristics, management, and outcomes of patients diagnosed with acute pulmonary embolism in the emergency department: initial report of EMPEROR (Multicenter Emergency Medicine Pulmonary Embolism in the Real World Registry). J Am Coll Cardiol. 2011;57(6):700-706. [CrossRef] [PubMed]
 
Post F, Mertens D, Sinning C, Peetz D, Münzel T. Decision for aggressive therapy in acute pulmonary embolism: implication of elevated troponin T. Clin Res Cardiol. 2009;98(6):401-408. [CrossRef] [PubMed]
 
Sanchez O, Trinquart L, Planquette B, et al. Echocardiography and pulmonary embolism severity index have independent prognostic roles in pulmonary embolism. Eur Respir J. 2013;42(3):681-688. [CrossRef] [PubMed]
 
Singanayagam A, Chalmers JD, Scally C, et al. Right ventricular dilation on CT pulmonary angiogram independently predicts mortality in pulmonary embolism. Respir Med. 2010;104(7):1057-1062. [CrossRef] [PubMed]
 
Soares TH, de Bastos M, de Carvalho BV, et al. Prognostic value of computed tomographic pulmonary angiography and the pulmonary embolism severity index in patients with acute pulmonary embolism. Blood Coagul Fibrinolysis. 2013;24(1):64-70. [CrossRef] [PubMed]
 
Spencer FA, Goldberg RJ, Lessard D, et al. Factors associated with adverse outcomes in outpatients presenting with pulmonary embolism: the Worcester Venous Thromboembolism Study. Circ Cardiovasc Qual Outcomes. 2010;3(4):390-394. [CrossRef] [PubMed]
 
Spirk D, Aujesky D, Husmann M, et al. Cardiac troponin testing and the simplified Pulmonary Embolism Severity Index. The SWIss Venous ThromboEmbolism Registry (SWIVTER). Thromb Haemost. 2011;106(5):978-984. [CrossRef] [PubMed]
 
Subramaniam RM, Mandrekar J, Blair D, Peller PJ, Karalus N. The Geneva prognostic score and mortality in patients diagnosed with pulmonary embolism by CT pulmonary angiogram. J Med Imaging Radiat Oncol. 2009;53(4):361-365. [CrossRef] [PubMed]
 
Venetz C, Jiménez D, Mean M, Aujesky D. A comparison of the original and simplified Pulmonary Embolism Severity Index. Thromb Haemost. 2011;106(3):423-428. [CrossRef] [PubMed]
 
Zondag W, Vingerhoets LM, Durian MF, et al; Hestia Study Investigators. Hestia criteria can safely select patients with pulmonary embolism for outpatient treatment irrespective of right ventricular function. J Thromb Haemost. 2013;11(4):686-692. [CrossRef] [PubMed]
 
Zwierzina D, Limacher A, Méan M, et al. Prospective comparison of clinical prognostic scores in elder patients with a pulmonary embolism. J Thromb Haemost. 2012;10(11):2270-2276. [CrossRef] [PubMed]
 
Goldhaber SZ. Modern treatment of pulmonary embolism. Eur Respir J Suppl. 2002;35:22s-27s. [CrossRef] [PubMed]
 
Torbicki A, van Beek EJR, Charbonnier B, et al. Guidelines on diagnosis and management of acute pulmonary embolism. Task Force on Pulmonary Embolism, European Society of Cardiology. Eur Heart J. 2000;21(16):1301-1336. [CrossRef] [PubMed]
 
Dalen JE, Hirsh J. Introduction: antithrombotic therapy—the evolving consensus: 1985 to 1998. Chest. 1998;114(5_suppl):439S-440S. [CrossRef] [PubMed]
 
Geersing GJ, Bouwmeester W, Zuithoff P, Spijker R, Leeflang M, Moons KG. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS ONE. 2012;7(2):e32844. [CrossRef] [PubMed]
 
Whiting PF, Rutjes AWS, Westwood ME, et al; QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. [CrossRef] [PubMed]
 
McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS; Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. JAMA. 2000;284(1):79-84. [CrossRef] [PubMed]
 
Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990. [CrossRef] [PubMed]
 
Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Chapter 10: analysing and presenting results.. In:Deeks JJ, Bossuyt PM, Gatsonis C., eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Version 1.0. The Cochrane Collaboration; 2010. http://srdta.cochrane.org/. Accessed October 2014.
 
Trikalinos TA, Balion CM, Coleman CI, et al. Chapter 8: meta-analysis of test performance when there is a “gold standard.” J Gen Intern Med. 2012;27(suppl 1):S56-S66. [CrossRef] [PubMed]
 
Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ. 2003;326(7382):219. [CrossRef] [PubMed]
 
Higgins JPT, Green S., eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. http://srdta.cochrane.org/. Accessed October 2014.
 
Doebler P, Holling H. Meta-analysis of diagnostic accuracy (mada). R package version 0.5.4. http://CRAN.R-project.org/package=mada. Published 2013. Accessed September 2014.
 
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264-269. [CrossRef] [PubMed]
 
Aujesky D, Roy PM, Le Manach CP, et al. Validation of a model to predict adverse outcomes in patients with pulmonary embolism. Eur Heart J. 2006;27(4):476-481. [CrossRef] [PubMed]
 
Torbicki A, Perrier A, Konstantinides S, et al; ESC Committee for Practice Guidelines (CPG). Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J. 2008;29(18):2276-2315. [CrossRef] [PubMed]
 
Wicki J, Perrier A, Perneger TV, Bounameaux H, Junod AF. Predicting adverse outcome in patients with acute pulmonary embolism: a risk score. Thromb Haemost. 2000;84(4):548-552. [PubMed]
 
Zondag W, Mos IC, Creemers-Schild D, et al; Hestia Study Investigators. Outpatient treatment in patients with acute pulmonary embolism: the Hestia Study. J Thromb Haemost. 2011;9(8):1500-1507. [CrossRef] [PubMed]
 
Uresandi F, Otero R, Cayuela A, et al. A clinical prediction rule for identifying short-term risk of adverse events in patients with pulmonary thromboembolism [in Spanish]. Arch Bronconeumol. 2007;43(11):617-622. [CrossRef] [PubMed]
 
Aujesky D, Roy PM, Verschuren F, et al. Outpatient versus inpatient treatment for patients with acute pulmonary embolism: an international, open-label, randomised, non-inferiority trial. Lancet. 2011;378(9785):41-48. [CrossRef] [PubMed]
 
Aujesky D, Smith KJ, Cornuz J, Roberts MS. Cost-effectiveness of low-molecular-weight heparin for treatment of pulmonary embolism. Chest. 2005;128(3):1601-1610. [CrossRef] [PubMed]
 
Ng AC, Chow V, Yong AS, Chung T, Kritharides L. Prognostic impact of the Charlson comorbidity index on mortality following acute pulmonary embolism. Respiration. 2013;85(5):408-416. [CrossRef] [PubMed]
 
Erkens PM, ten Cate H, Büller HR, Prins MH. Benchmark for time in therapeutic range in venous thromboembolism: a systematic review and meta-analysis. PLoS ONE. 2012;7(9):e42269. [CrossRef] [PubMed]
 
van Bellen B, Bamber L, Correa de Carvalho F, Prins M, Wang M, Lensing AW. Reduction in the length of stay with rivaroxaban as a single-drug regimen for the treatment of deep vein thrombosis and pulmonary embolism. Curr Med Res Opin. 2014;30(5):829-837. [CrossRef] [PubMed]
 
Moher D, Pham B, Klassen TP, et al. What contributions do languages other than English make on the results of meta-analyses? J Clin Epidemiol. 2000;53(9):964-972. [CrossRef] [PubMed]
 
Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002;31(1):115-123. [CrossRef] [PubMed]
 
Kobberøe Søgaard K, Schmidt M, Pedersen L, Horváth-Puhó E, Toft Sørensen H. 30-year mortality after venous thromboembolism: a population-based cohort study. Circulation. 2014;130(10):829-836. [CrossRef] [PubMed]
 
Yamaki T, Nozaki M, Sakurai H, Takeuchi M, Soejima K, Kono T. Presence of lower limb deep vein thrombosis and prognosis in patients with symptomatic pulmonary embolism: preliminary report. Eur J Vasc Endovasc Surg. 2009;37(2):225-231. [CrossRef] [PubMed]
 
Volschan A, Albuquerque D, Tura BR, et al; EMEP (Estudo Multicêntrico de Embolia Pulmonar) investigators. Predictors of hospital mortality in hemodynamically stable patients with pulmonary embolism. Arq Bras Cardiol. 2009;93(2):135-140. [CrossRef] [PubMed]
 
Agterof MJ, Schutgens RE, Moumli N, et al. A prognostic model for short term adverse events in normotensive patients with pulmonary embolism. Am J Hematol. 2011;86(8):646-649. [CrossRef] [PubMed]
 
den Exter PL, Gómez V, Jiménez D, et al; Registro Informatizado de la Enfermedad TromboEmbólica (RIETE) Investigators. A clinical prognostic model for the identification of low-risk patients with acute symptomatic pulmonary embolism and active cancer. Chest. 2013;143(1):138-145. [CrossRef] [PubMed]
 
Kline JA, Roy PM, Than MP, et al. Derivation and validation of a multivariate model to predict mortality from pulmonary embolism with cancer: The POMPE-C tool. Thromb Res. 2012;129(5):e194-e199. [CrossRef] [PubMed]
 
Leal SRN, Dores H, Rosario I, et al. Acute pulmonary embolism: development of early and long term morbidity and mortality risk score. Eur Heart J. 2010;31(suppl 1):625. [CrossRef] [PubMed]
 
Murugappan M, Johnson JA, Gage BF, et al. HOme Management Exclusion (HOME) criteria for initial treatment of acute pulmonary embolism [abstract]. Am J Respir Crit Care Med. 2008;177:A182.
 
Limone BL, Hernandez AV, Michalak D, Bookhart BK, Coleman CI. Timing of recurrent venous thromboembolism early after the index event: a meta-analysis of randomized controlled trials. Thromb Res. 2013;132(4):420-426. [CrossRef] [PubMed]
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Supporting Data

Online Supplement

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Find Similar Articles
CHEST Journal Articles
PubMed Articles
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543