0
Clinical Investigations in Critical Care |

Impact of a Disease Management Program Upon Caregivers of Chronically Critically Ill Patients* FREE TO VIEW

Sara L. Douglas, PhD, CRNP; Barbara J. Daly, PhD, RN; Carol Genet Kelley, PhD, RN; Elizabeth O’Toole, MD; Hugo Montenegro, MD
Author and Funding Information

*From Case Western Reserve University, Cleveland, OH.

Correspondence to: Sara L. Douglas, PhD, CRNP, School of Nursing, Case Western Reserve University, 10900 Euclid Ave, Cleveland, OH 44106-4904; e-mail: SLD4@case.edu



Chest. 2005;128(6):3925-3936. doi:10.1378/chest.128.6.3925
Text Size: A A A
Published online

Objectives: Few studies have examined the effects of caregiving on the caregivers of chronically critically ill (CCI) patients, and no one has examined the impact of a disease management program (DMP) on physical and psychological outcomes for the caregivers of CCI patients. The purposes of this study of caregivers of CCI patients were as follows: (1) to describe the characteristics of CCI patients and caregivers and to examine the frequency of depression, subjective burden, and physical health; (2) to examine factors related to depression after hospital discharge; and (3) to examine the effects of a DMP on the physical health, depression, and burden of caregivers 2 months post-hospital discharge.

Design: Prospective experimental design.

Setting and participants: Caregivers of 290 patients who had received > 3 days of mechanical ventilation while in the ICU of a university medical center.

Measurements: Sociodemographics, caregiver burden, physical health status, and depression were measured using established tools.

Results: Interviews of caregivers were conducted at hospital discharge and 2 months later. Seventy-three percent of patients survived, completed the study period, and required caregiving 2 months later. Caregivers of patients residing in an institution reported higher depression (p = 0.0001), higher burden (ie, disrupted schedule, p = 0.0001; lack of family support, p = 0.036), and greater health problem scores (p = 0.0001) than did caregivers of patients residing at home. The DMP did not have a statistically significant impact on any of the outcome variables. However, by 2 months, 54% of caregivers in the experimental group had no depression or mild depression compared with 34.5% of the control group.

Conclusion: Two months after hospital discharge, approximately 25% of caregivers were classified as depressed with 16.7% of the depressed group classified as moderately or severely depressed. The caregivers of CCI patients are at risk for post-hospital discharge depression, and the caregivers of institutionalized CCI patients are at highest risk of long-term negative effects from caregiving.

Figures in this Article

Approximately 5.7 million adult patients receive treatment in ICUs in the United States.1Of those patients, approximately 6% will require > 3 days of mechanical ventilation and extended ICU stays. These patients, labeled as “chronically critically ill”2(CCI), although small in number, consume 25 to 30% of all ICU resources.34 While in-hospital outcomes are poor, research56 has identified the fact that post-hospital discharge outcomes continue to carry an increased risk of mortality, morbidity, and poor quality of life. In addition, it has been noted that many of the caregivers of these patients demonstrate a high degree of depression that lasts up to 6 months.78

To date, caregiving research has focused primarily on caregiving associated with several specific populations, as follows: infirm elderly9; Alzheimer disease1012; cancer1315; and in-home mechanical ventilation.1618 The characteristics of the caregivers of these patients as well as the risks for caregiver depression and/or burden have remained relatively consistent across these disease groups. Such characteristics include the following: female gender; relative of the care receiver; being employed; and being younger in age.9,12,1920 The risks for poor caregiver outcomes have included poor caregiver physical health, lack of support, poor patient functional status, and cognitive impairment of the patient.5,19,21 In addition, caregivers have reported a sense of burden that increases over time and is cumulative.20 Subjective burden has been described as a primary stressor arising from caregiving that has an impact on mental health outcomes and depressive symptoms in caregivers.22

Some research has focused on the caregivers of those receiving mechanical ventilation at home.1618 These studies have been descriptive and have identified the fact that the caregivers of these patients provided a substantial amount of informal care (ie, 7 to 8 h/d) and received very little additional formal or informal assistance.,16,18

To date, there have been very few studies that have focused on the caregivers of patients who are CCI. Douglas and Daly7studied the physical and psychological outcomes of caregiving for the caregivers of patients who had received > 4 days of continuous in-hospital mechanical ventilation. They reported that 6 months after hospital discharge caregivers had a significant decline in physical health, with 36.4% reporting symptoms consistent with some degree of depression, and that impaired caregiver physical health and high emotional overload were significantly related to caregiver depression. Im et al8 described caregiver outcomes 2 months post-hospital discharge for patients who had received ≥ 48 h of mechanical ventilation in the ICU. They found that over one third of caregivers reported symptoms consistent with some degree of depression. In addition, they reported that higher depression scores were related to more daily hours of caregiving assistance with patients’ activities of daily living (ADLs).

Caregiver intervention studies have focused primarily on interventions for caregivers of cancer patients1415,23 and Alzheimer disease patients.21,2425 These studies have reported a variety of findings, most notably that coping and social support concentrated on specific areas can be beneficial to caregivers,15 that respite interventions and psychosocial interventions have a moderating effect on caregiver distress,21 and that individual psychosocial interventions have some effect on the outcomes of caregiver burden and distress.21 Despite the observations of Knight et al21 that supportive interventions could have an effect on caregiver distress (eg, burden and depression), Tennstedt et al,9 reported that caregiver depressive symptoms were primarily related to the health status of the care receiver and that informal or formal support of the caregiver was not associated with lower degree of depression symptoms. As a result, Tennstedt et al9 recommended that caregiver depression be treated directly with medication and/or psychotherapy.

Thus, while preliminary work has documented a rate of depression for the caregivers of patients who have received prolonged mechanical ventilation in the ICU that is comparable to the rates of depression reported for caregivers of patients with Alzheimer disease and cancer, no one has examined the effect of an intervention that has been specifically designed for the needs of caregivers of CCI patients. Therefore, the purposes of the present study were as follows: (1) to describe the characteristics of and to document the incidence of caregiver depression, burden, and physical health in caregivers of CCI patients; (2) to examine the relationship of selected variables to caregiver depression and burden 2 months after hospital discharge; and (3) to examine the effects of an 8-week post-hospital supportive caregiver intervention on the outcomes of caregiver depression, burden, and physical health.

This study was a randomized trial of a disease management program for long-term ventilator patients and their caregivers following hospitalization. Long-term mechanical ventilation was defined as continuous mechanical ventilation beyond 72 h while in the hospital. While the focus of this study was on outcomes after hospital discharge, the eligibility criterion of > 72 h of continuous mechanical ventilation while in the ICU was established while the patient was still in the hospital. Although there has been very little consensus about what constitutes “long-term” ventilation (range, 48 h to 10 days),4,78 we chose 72 h in order to exclude patients who were simply slow to wean from ventilation while still capturing those whose clinical problems were likely to entail a high risk of continued mortality and morbidity.

For our study, all people who identified themselves as principal caregivers or whom the patient identified as a primary caregiver (caregiver being defined as the person who gave a majority of needed care) were deemed to be the caregiver of the identified patient. This definition has been utilized in several other studies78 involving the impact of serious illness on caregivers.

The study site was a university medical center located in Cleveland, OH. The study was approved prior to data collection and was reviewed annually by the institutional review boards of the University Hospitals of Cleveland and Case Western Reserve University. Patient eligibility criteria needed to be met and patient consent obtained before the caregiver could be identified. Patient eligibility criteria were as follows: age ≥ 18 years; the ability to speak and understand English; hospital discharge location within 80 miles of the study site; and > 72 h of continuous mechanical ventilation while in the ICU. Patients who had received organ transplants and received case management from the transplant team, and hospice patients who received case management from the hospice team were excluded from the study. All diagnoses were accepted. Patients who required mechanical ventilation at home before becoming hospitalized were not eligible for enrollment. Caregiver eligibility criteria were as follows: age ≥ 18 years; the ability to speak and understand English; and the person designated by the patient as the principal caregiver (or if patient is unable to speak, the person who identified themselves as the principal caregiver).

All patients who met the eligibility criteria during their hospitalization in any adult ICU of the study hospital were enrolled. Patients and family members were approached for consent to participate when it was clear that hospital discharge was expected within the next few days.

Between March 2001 and December 2004, research nurses screened all patients admitted to all ICUs at the study site to determine patient eligibility. The research nurses tracked eligible patients during their hospital stay, and patients or their proxies were approached for written informed consent once the patient was close to hospital discharge. After written consent was obtained, the caregiver was identified and approached for written informed consent. At hospital discharge, patients and their caregivers were randomly assigned to either the experimental group or control group using a computerized randomization program. Patients and their caregivers in both groups were followed for the 2-month post-hospital discharge study period.

Prior to data collection, research nurses were trained in the use and administration of all interview tools. Interrater reliabilities between the research nurses was assessed; acceptable reliabilities of 80% agreement, as well as a Pearson correlations of at least 0.80 (for continuous variables) and a κ statistics of 0.70 (for categoric variables) were established before data collection proceeded.2627 Every 4 months throughout the data collection period, ongoing interrater reliability was also assessed and retraining occurred if reliabilities fell below acceptable levels.

Caregivers were interviewed twice. The first interview was conducted within 2 weeks of the patient’s discharge from the hospital, and the second interview was conducted 2 months later. The purpose of the interviews was to assess the caregiver’s depression, burden, and physical health status.

Instruments

The instruments used for data collection were as follows: The Center for Epidemiologic Studies depression scale (CES-D) was used to measure symptoms of depression. The CES-D focuses on distress symptoms prevalent among nonpsychiatric populations and is a measure of depressed mood.28 The CES-D has been tested with samples of patients receiving long-term ventilation (LTV).78 It is a 20-item tool that uses a 4-point Likert-type summative scale. Scores range from 0 to 60, with higher scores indicating more depressed mood. A score of > 15 has been identified as indicating depressive symptoms.28 Test-retest reliability and the validity of the tool have been well-established.28

Caregiver (subjective) burden was defined as “psychological, social and emotional impact caregivers may experience from the objective burden of caregiving”.29 The caregiver reaction assessment was used to measure subjective burden in our caregiver sample. The caregiver reaction assessment assesses informal caregivers’ experiences with providing care on 24 items, all of which use a 5-point Likert scale. Four negative subscales (ie, disrupted schedule, financial problems, lack of family support, and health problems) and one positive subscale (self-esteem) are obtained. For the negative subscales, higher scores indicate a higher level of burden in that particular domain. For the positive subscale, higher scores indicate a lower level of burden in the area of caregivers’ sense of self-esteem. Reliability and concurrent validity have been established.,22

Caregiver health status was measured using the Medical Outcomes Study eight-item short form (SF-8) tool.30 It consists of eight items, each of which addresses a different dimension of quality of life, and yields two subscale scores (physical and mental). Scores on the physical subscale range from 14.1 to 64.0, with higher scores indicating a better physical health status. Scores on the mental subscale range from 6.8 to 70.4, with higher scores indicting a better mental health status. Reliability has been well-established for the 1-week recall version.30

Some additional measures were utilized to assess patient (ie, care-receiver) variables of interest. As with the caregiver, the patient also completed the SF-8 at hospital discharge, and 2 months later to assess health status (physical and emotional). Two portions of the Outcomes and Assessment Information Set were used to assess patients’ functional status. The Outcomes and Assessment Information Set is a 79-item tool that is commonly used by home care agencies to assess care needs and outcome measures.31 The 14 items assessing ADLs and instrumental activities of daily living (IADLs) provided a total score ranging from 0 to 66, with higher scores indicating increased dependency or the need for help in performing these activities. Interrater reliability and construct validity have been reported.31

Intervention

The disease management program intervention was structured in order to provide emotional support as well as instrumental support, and provided individualized case management services from an advanced practice nurse (APN) who had access to a pulmonologist, geriatrician, and bioethicist for guidance and collaboration. The use of coping and social/emotional support had been supported by several intervention studies in the caregiving literature.21 Specifically, the 8-week intervention provided the following:

1. Emotional support through discussion, referrals, and reassurance; and

2. Instrumental support through care coordination, education, and communication.

Intervention Procedure

Prior to hospital discharge, the APN met with the patient and caregiver to perform a baseline assessment of both patient and caregiver, to discuss hospital discharge plans, and to establish a plan of care. The APN consulted with the hospital care team about the patient’s care needs and completed a hospital discharge summary that included the post-hospital discharge plan of care for the patient, patient goals, the presence of advance directives, and an assessment of caregiver needs and current status of coping. This summary was sent on the day of hospital discharge to all relevant out-of-hospital health-care providers (eg, nursing home staff, family physician, consulting specialist, or home care agency).

Patients and caregivers who lived > 30 miles from the study site were contacted by telephone rather than with in-person visits. All contacts with patient, family, and health-care providers were documented using standardized data collection forms.

During subsequent in-person meetings (for those who lived < 30 miles away), the APNs assessed both patient and caregiver needs for assistance and then, through an individualized plan of care, provided assistance that was needed. Typical APN activities included attending team meetings at extended care facilities, helping caregivers prepare for the patient’s eventual return home, providing emotional support for caregivers, counseling caregivers about end-of-life options, providing referrals for support (physical and/or emotional) to caregivers, coordinating services among multiple providers, arranging follow-up care from specialists, and monitoring the patient’s condition and medications. APNs often served as advocates for the patients and the caregivers, and made phone calls to physicians on behalf of the patients or caregivers in order to facilitate the treatment plan, answer questions, or expedite care.

Caregivers of patients in the usual-care group (ie, control group) were interviewed within 2 weeks of hospital discharge for the completion of study instruments, then 2 months later. Interviews were conducted in person at hospital discharge and by telephone after hospital discharge. When caregivers in the usual-care group asked for advice or information from the interviewers, they were referred back to their primary care provider, extended care facility staff, or home care agency.

Statistical Analysis

Comparisons between experimental and control groups were done using repeated measures analysis of variance (ANOVA) for nonskewed continuous variables, Mann-Whitney U test for skewed continuous variables (two-group comparisons), and χ2 test for categoric variables. Multiple linear regression was used to examine a model for predicting depression post-hospital discharge. Sample size was calculated using power analysis that incorporated the following assumptions: α = 0.05; nondirectional hypotheses; medium effect size; and a desired power of 90%. Based on these assumptions, a sample size of 256 was needed.32

Figure 1 shows the distribution of the total sample, the percentage of eligible patients, refusals, and in-hospital deaths. The 28.5% refusal rate is consistent with that found in previous work with this population and was most often related to the patient’s or family member’s report of feeling “overwhelmed” and being reluctant to take on any additional burden of research.6 Forty-four patients who were randomized (experimental group, 20 patients; control group, 24 patients) did not have primary caregivers and thus were excluded from enrollment into the study. A total of 290 caregivers were approached, and all gave written consent to participate in the study.

Caregiver Characteristics

Table 1 summarizes caregiver demographic characteristics. In general, the caregivers of CCI patients were predominantly female, middle-aged, employed, and a relative of the patient, such as a spouse or child. A majority of caregivers were living at home, with slightly more than half having lived with the patient prior to this current hospitalization. Caregivers were in good health, according to the SF-8 physical health subscale scores. There were no statistically significant differences between caregivers in the experimental and control groups for any of the demographic variables.

Patient Characteristics

A description of patient characteristics (n = 290) is presented in Table 2 . In general, the median hospital length of stay was 20 days (range, 6 to 107 days), and the median ICU stay was 13 days (range, 3 to 101 days). Prior to the index hospitalization, 90.7% of patients were living at home. A majority of patients (n = 224) who survived hospitalization were discharged to an institutional setting as follows: to a nursing home, 34.1%; to a rehabilitation center, 22.1%; and to a long-term acute care facility, 21%. Only 12.1% (n = 66) were able to be discharged directly to home. There were no statistically significant differences between experimental and control patients concerning any of these variables. At hospital discharge, patients had low physical functioning as evidenced by low SF-8 physical function subscale and high ADL scores.

Patient Mortality and Disposition 2 Months After Hospital Discharge

During the 2-month post-hospital discharge study period, 52 of the patients (17.9%) who were alive at hospital discharge died and 16 (5.5%) patients dropped out or were unavailable for follow-up. Patients who were initially discharged to home had a significantly lower rate of posthospital mortality (1.5%) than did those who were discharged to an institution (22.3%) [relative risk (RR), 14.73; 95% confidence interval (CI), 2.07 to 104.63; p = 0.0001]. In addition, there was no significant difference in post-hospital discharge mortality rate between the experimental and control groups (RR, 1.09; 95% CI, 0.61 to 1.94; p = 0.76). Finally, the association between the initial hospital discharge location and the location 2 months later was statistically significant (91.8% of those discharged to home were alive and still residing at home 2 months later; and 55.2% of those discharged to an institution were alive and still residing in an institution 2 months later [RR, 6.73; 95% CI, 2.87 to 15.76; p = 0.0001]).

Effects of Caregiving

First, we examined whether the effects of caregiving varied according to the relationship of the caregiver to the patient. In our study, while spouses had higher odds of being depressed (CES-D score, > 15) 2 months post-hospital discharge (odds ratio [OR], 1.19) than did children (OR, 0.83) or other caregivers (OR, 0.89), the differences were not statistically significant (p = 0.72). We then examined whether there were differences in burden subscale scores as determined by the relationship of the caregiver to the patient. The distribution of all of the subscale scores was skewed, and the Kruskal-Wallis test was used to examine the differences. Of the five burden subscale scores, the subscale “lack of family support” demonstrated a statistically significant difference among the three groups (ie, spouse, child, and other) [χ2 test, 12.7; p = 0.002]. Caregivers who were children of the patient reported the highest burden due to lack of family support (mean, 2.4), while spouses reported the lowest burden due to lack of family support (mean, 1.8).

Another interest of the study was in whether the effects of caregiving were different depending on the disposition of the patient (ie, home vs institution). Prior research67 examining this outcome in caregivers of LTV patients had demonstrated various results. Using repeated-measures ANOVA, we examined the changes in CES-D score, burden, and physical health scores between hospital discharge and 2 months later for the caregivers of patients residing in an institution (eg, nursing home, rehabilitation center, or long-term acute care facility) vs the caregivers of patients residing at home. The caregivers of patients residing in an institution reported worse 2-month CES-D and burden scores than did caregivers of patients residing at home. On average, 2-month CES-D scores for the caregivers of patients residing in an institution were higher (mean, 15.5; SD, 12.26; 95% CI, 12.9 to 18.1) than for the caregivers of patients residing at home (mean, 9.63; SD, 10.23; 95% CI, 7.8 to 11.5), and these differences were significant (p = 0.0001). Differences in three burden subscales between the caregivers of patients residing in an institution vs those residing at home were also statistically significant. Those subscales were as follows: disrupted schedule (p = 0.001); lack of family support (p = 0.026); and health problems (p = 0.0001).

Finally, we were interested in the effect of caregiving on caregiver depression. At hospital discharge, 50.2% of caregivers reported symptoms that were consistent with some degree of depression (CES-D score, > 15), while only 31.1% of caregivers were classified as depressed 2 months post-hospital discharge. In addition, at hospital discharge 36.5% of caregivers were classified as having symptoms that were consistent with moderate or severe depression, while 21.8% of caregivers were classified as being moderately or severely depressed 2 months later. Looking specifically at those patients who reported being depressed at hospital discharge, we found that more than half (52.8%) were still depressed 2 months later. The association between being depressed at discharge and being depressed 2 months later was statistically significant (p = 0.0001).

We also examined the relationship between caregiver burden and depression among the caregivers of CCI patients. Significant correlations had been reported between burden and depression in other caregiving populations,11 but, to date, no one had reported such findings for the CCI caregiver population. Bivariate correlations between burden subscale scores and CES-D scores were conducted; all five subscale scores had a statistically significant correlation with CES-D scores. The highest correlation was between CES-D scores and physical health problems (r = 0.535; p = 0.0001), and the lowest correlation was between CES-D score and self-esteem (r = −0.175; p = 0.011).

Impact of Intervention on Outcomes

Mean scores for all outcome variables are reported in Table 3 . Using repeated-measures ANOVA, we examined changes in physical health scores, burden subscale scores, and depression scores over time between experimental and control patients, and found that there was no statistically significant difference between experimental and control groups over time for any of these variables.

While repeated-measures ANOVA had shown no difference in depression scores over time between the experimental and control groups, we were interested in further examining the outcome of depression. Specifically, we were interested in examining the trend in depression before and after the intervention. Given the fact that 17.9% of patients died during the study period, we were concerned that the caregivers of patients with better health status would be overrepresented in our analyses. Therefore, we used established methods33 used in longitudinal studies that involved creating a category for death in order to provide a more comprehensive picture of the trajectory of the group over time. Given the fact that when the patient died the caregiver was no longer included in the study, we decided to utilize this approach to describe the trajectory over time for caregivers in our study. In keeping with this conservative approach, we also assigned all cases that represented subject drop-out or subjects who were lost-to-follow-up to the death category as well.

As seen in Figures 2 and 3 , 69.2% of caregivers in the control group and 61.7% of those in the experimental group were classified as having mild or no depression at the time of the patient’s discharge from the hospital. With patient death and caregiver drop-out accounted for, 53.5% of the control group and 61.2% of the experimental group were classified as having mild or no depression by 2 months. A χ2 goodness-of-fit test was performed to see whether the percentage of subjects in the experimental group who had no or mild depression was better than expected (control group) outcome. We found a statistically significant improvement in the percentage of subjects who had no or mild depression compared to what was expected (χ2 = 5.30; p < 0.05).

By contrast, when death and drop-out were not accounted for, we found that by 2 months post-hospital discharge 78% of caregivers in the experimental group and 79.2% of those in the control group were classified as having mild or no depression. A χ2 goodness-of-fit was not statistically significant (χ2 = 0.07; p > 0.05).

In addition, we subtracted the 2-month CES-D score from the hospital discharge (baseline) score in order to assess the direction of change in depression scores over the intervention period. As seen in Figure 4 , the experimental group had a larger percentage of caregivers who improved or stayed the same (55.6%) over the 2-month period than did the control group (40.8%).

Next, multiple linear regression was used to examine the relationship of specific variables (shown to relate to caregiver depression in LTV as well as other patient populations) to caregiver depression 2 months post-hospital discharge. For the purposes of our analyses, we included the following as independent variables: caregiver age; caregiver physical health rating; caregiver gender; location of the patient (ie, institution or home); caregiver burden (five subscales); physical status of the patient (number of dependencies on ADLs and IADLs); and baseline caregiver depression. The 11 independent variables were regressed on the 2-month CES-D scores with an R2 of 0.363. The standardized coefficients and corresponding significance are reported in Table 4 . Only one variable was statistically significant and made a statistically significant unique contribution with all variables in the equation (caregiver depression at discharge; p = 0.001). Adding the intervention variable into the model essentially made no difference in the R2 (change in R2 = 0.0001).

Intervention Dose-Response Effects on Depression Scores

Finally, we were interested in testing for intervention dose-response effects on depression scores. For subjects in the experimental group, there was a minimum of 8 APN-initiated contacts that constituted the intervention. Additional contacts were initiated by the patient, caregiver, health-care professional, or APN. The median number of contacts during the 2-month study period was 30 for those in the experimental group and 5 for those in the control group (Z = −12.39; p = 0.0001). A “typical” contact was initiated by a registered nurse caring for the patient (or the APN following the patient) and was made for the purpose of monitoring, assessing, or supporting the patient.

To examine the dose-response effect, caregiver depression at baseline and a count of the number of nursing contacts were regressed on caregiver depression at 2 months post-hospital discharge. Caregiver depression at 2 months was positively and significantly related to caregiver depression at baseline (β = 0.56; SE, 0.05; p = 0.0001). In the model, a nonsignificant inverse relationship was demonstrated between the number of nursing contacts and the caregivers’ level of depression at 2 months post-hospital discharge (β = −0.01; SE, 0.02; p = 0.84). Thus, as the number of nursing contacts increased, the CES-D scores decreased.

Overall, the caregivers of CCI patients demonstrated characteristics that were similar to those found in other groups of caregivers. When examining outcomes, some findings are worth noting. First, as in previous research with this caregiving population, we found that the caregivers of patients residing in an institution reported higher depression and burden than the caregivers of patients residing at home. Specifically, this study is the first to report on specific areas of burden for the caregivers of CCI patients residing in an institution. These caregivers felt more of a sense of disrupted schedule, a higher degree burden (specifically a lack of family support), and a higher degree of health problems than did their counterparts caring for patients residing at home. In addition, we found that while spouses were at the highest risk for depression (compared to children or other caregivers), they reported the lowest sense of burden from a lack of family support. This finding is in contrast to the work of others11 who reported that spouses had the highest risk for burden compared to children or other caregivers.

To our knowledge, this is the first study to report the specific sources of burden for this caregiving population. When comparing the mean burden subscale scores to those of other caregiving groups, we found that the caregivers in our sample had higher self-esteem scores (more positive impact of caregiving) than those reported for other caregiving groups. Conversely, caregivers in our sample had scores that indicated a more negative impact of caregiving on their daily schedule and their health than for the caregivers of cancer or stroke patients. We also found a relationship between caregiver burden and caregiver depression that was similar to relationships reported for other caregiving populations.34

With regard to the reports of depression among caregivers in our sample, we found that the average CES-D score at 2 months post-hospital discharge (mean, 12.12; SD, 11.2) was in the higher range of depression scores reported for other caregiver groups (mean CES-D scores range, 6.1 to 16.0).1920 Our sample had higher CES-D scores than those reported for the caregivers of patients with cancer, dementia, and eldercare, and were slightly lower than the scores reported for the caregivers of patients with spinal cord injuries.20 The prevalence of depression reported in this study is comparable to that reported for other long-term ventilator caregiving groups.78 However, unlike earlier findings in which 15.6% of caregivers of LTV patients were reported to be severely depressed at 6 months post-hospital discharge,7 we found that only 9% of caregivers in our current sample reported symptoms that were consistent with severe depression.

Finally, the intervention appeared to have no impact on any of our outcome variables (ie, depression, burden, and physical health). This may be due to several factors. First, others have reported23 that when conducting intervention studies the goals of which are to impact depression, the effects did not become statistically significant until 8 months after entry into the study. Our work, like that of Kozachik et al,15 found no statistically significant differences between the experimental and control groups but did find some results that might indicate that the intervention was beginning to effect a positive change. Like Kozachik et al,15 we found a nonsignificant inverse relationship between the number of nursing contacts and depressive symptoms at 2 months after entry into the study. In addition, we found that a larger percentage of subjects in the experimental group had CES-D scores that improved or stayed the same compared to subjects in the control group. And finally, we found that a larger percentage of caregivers in the experimental group had no or mild depression compared to those in the control group and that the experimental group had a statistically higher than expected percentage of subjects with mild or no depression after the 2-month intervention. Again, while most of these results were not statistically significant, they may indicate that the intervention was, at the 2-month point, managing to keep some caregivers from progressing to moderate or severe depression.

There are several limitations to the study. First, our attrition rate was high. We lost 23.4% of our sample to patient mortality and caregiver attrition. Thus, the generalizability of the study results is limited and the power of the test statistics were low. A second limitation is the high false-negative rate (36.4 to 40%) previously reported for the CES-D scale.3536 In essence, in more than a third of the cases, caregivers who were identified as “not depressed” by cutoff scores of the CES-D scale were identified as being depressed by more clinically specific measures. Therefore, we have, most likely, underreported the number of caregivers who were depressed. Finally, we did not include as outcome variables measures that at 2 months after entry into the study may have been more sensitive to the impact of the intervention on caregivers. For example, the inclusion of tools that measured other areas that may be affected by caregiving such as overload, distress, or sadness1921,37 might have yielded different results.

Caregivers of patients who have received prolonged mechanical ventilation while in the hospital may be at risk for depressive symptoms, burden, and poor physical health post-hospital discharge. Of note is that caregiver (rather than patient) variables were the best predictors of depression post-hospital discharge. In addition, the caregivers of patients residing in an institutional setting are at particular risk for burden and depression, and may need special attention and support. Some possible sources of their risk for depression could be the more debilitated nature of institutionalized patients, possible guilt because the patient is not being cared for at home, and possible frustration by caregivers who have less control over their loved one’s care in an institutional setting. In addition, patients who are discharged to an institution are at high risk for continued institutionalization and death, a fact that should be communicated to their caregivers. Such information could be helpful to caregivers in terms of planning, and in setting realistic expectations and goals, and might contribute to minimizing depression over time. While the present study confirms earlier findings related to the incidence of depression found in this specific group of caregivers, it is the first to report on the sources of burden for these caregivers. Again, the caregivers of patients in institutions are at particular risk for feeling an increased burden due to disrupted schedules, lack of family support, and health problems.

Approximately one third of caregivers who were still caring for patients 2 months post-hospital discharge reported some degree of depressive symptoms, and it is these families that have ongoing needs for support and assistance. In addition, while the caregivers of CCI patients reported higher burden scores (disrupted schedule) than the caregivers of cancer and stroke patients, and depression scores that were comparable to those of other recognized caregiver groups, caregivers in our sample continued to remain an unrecognized group to the health-care system. While our intervention did not show statistically significant results, this may be due, in part, to the fact that the intervention was too short in length or that our final observation time needed to be further from hospital discharge. However, the clinical significance of the intervention may be that empathetic listening, support in planning and decision making, and referrals for support may have been instrumental in minimizing depression in some caregivers. Nonetheless, health-care providers need to recognize that the caregivers of patients who receive prolonged in-hospital mechanical ventilation represent a distinct group of caregivers who are at risk for depression, burden, and poor health outcomes. Health-care providers should be more alert to recognizing the symptoms of depression early (ie, at patient hospital discharge) in this caregiver population and should refer caregivers to mental health professionals for further evaluation and treatment. Medical management with psychopharmacologic agents along with support and psychological therapy may be needed to treat depression in these caregivers.9,15

Finally, further research needs to be conducted on the testing of interventions that are aimed at reducing depression through referral to mental health professionals for therapy and/or medication, as well as through the provision of ongoing emotional and structural support for this caregiver population. In addition to trying to tailor interventions to this distinct group of caregivers, future interventions should pay special attention to meeting the unique needs of the caregivers of patients residing in an institutional setting. To date, the role of the caregiver in an institutional setting has been underappreciated in research as well as in clinical practice. As has been shown in this work as well as in the work of others, the negative effects of caregiving are not unique to those who provide care at home. The early identification and referral of those exhibiting symptoms of depression, as well as the development and testing of specific interventions, must occur if we are to meet the mental health needs of this vulnerable population of caregivers.

Abbreviations: ADL = activity of daily living; ANOVA = analysis of variance; APN = advanced practice nurse; CCI = chronically critically ill; CES-D = Center for Epidemiologic Studies depression scale; CI = confidence interval; IADL = instrumental activity of daily living; LTV = long-term ventilation; OR = odds ratio; RR = relative risk; SF-8 = Medical Outcomes Study eight-item short form

This study was supported by a grant (No. RO1-NR0–0527) from the National Institute of Nursing Research.

Table Graphic Jump Location
Table 1. Comparison of Demographic Variables Between Experimental and Control Group Caregivers at Patients’ Hospital Discharge (n = 290)
* 

Forty-seven of the 211 experimental caregivers (22.3%) received telephone follow-up rather than in-person visits because they lived > 30 miles from the study site. There were no significant differences in outcomes between these caregivers and those who received in-person visits, and therefore the groups were combined for purposes of analysis.

 

Values given as the mean (SD).

 

Values given as No. (%).

Table Graphic Jump Location
Table 2. Comparison of Demographic and Clinical Variables Between Experimental and Control Group Patients at Hospital Discharge (n = 290)
* 

Values given as the mean (SD).

 

Proxy data were not included in SF-8 analyses. Group sizes for these analyses were as follows: experimental group, 141 patients; control group, 57 subjects.

 

ADL and IADL values are derived from the OASIS tool. Higher numbers indicate a higher degree of dependence in performing ADLs and IADLs.

§ 

Values given as No. (%).

 

Diagnoses taken from the International Classification of Diseases, ninth revision.

 

GI disease and cancer, for example.

# 

RR, 1.09; 95% CI, 0.62 to 1.94.

Table Graphic Jump Location
Table 3. Caregiver Outcomes at 2 Months for Experimental and Control Groups (n = 211)*
* 

In the interest of brevity, only 2-month postintervention scores are reported.

Figure Jump LinkFigure 2. Categories of caregiver depression (CES-D score) at hospital discharge (n = 206) and 2 months later (n = 166) for caregivers in the experimental group. Solid black represents percent of patients who died or dropped out and whose caregivers did not provide data.Grahic Jump Location
Figure Jump LinkFigure 3. Categories of caregiver depression (CES-D score) at hospital discharge (n = 79) and 2 months later for caregivers in the control group (n = 56). Solid black represents percent of patients who died or dropped out and whose caregivers did not provide data.Grahic Jump Location
Figure Jump LinkFigure 4. Changes in levels of caregiver depression (CES-D score) over time for the experimental group (n = 205) and control group (n = 71). Solid black represents the percentage of patients who died or dropped out and of those whose caregivers did not provide data.Grahic Jump Location
Table Graphic Jump Location
Table 4. Standardized Estimates From Ordinary Least-Squares Regressions of Caregiver Depression on Caregiver and Care-Receiver Variables (n = 211)
* 

Values are reported with the intervention variable out of the model.

 

For all subscales except self-esteem, higher numbers indicate a greater negative impact of the dimension upon caregiving. For self-esteem, higher numbers indicate a greater positive impact of that dimension upon caregiving.

Nelson, JE, Meier, DE, Litke, A, et al (2004) The symptom burden of chronic critical illness.Crit Care Med32,1527-1534. [CrossRef] [PubMed]
 
Girard, K, Raffin, TA The chronically critically ill: to save or let die.Respir Care1985;30,339-347. [PubMed]
 
Gracey, DR, Gillespie, D, Nobrega, F, et al Financial implications of prolonged ventilator care of Medicare patients under the prospective payment system.Chest1987;91,424-427. [CrossRef] [PubMed]
 
Douglas, P, Rosen, RL, Butler, PW, et al DRG payment for long-term ventilator patients.Chest1987;91,413-417. [CrossRef] [PubMed]
 
Carson, SS, Bach, PB, Brzozowski, L, et al Outcomes after long-term acute care.Am J Respir Crit Care Med1999;159,1568-1573. [PubMed]
 
Douglas, SL, Daly, BJ, Gordon, N, et al Survival and quality of life: short-term versus long-term ventilator patients.Crit Care Med2002;30,2655-2662. [CrossRef] [PubMed]
 
Douglas, SL, Daly, BJ Caregivers of long-term ventilator patients: physical and psychological outcomes.Chest2003;123,1073-1081. [CrossRef] [PubMed]
 
Im, K, Belle, SH, Schulz, R, et al Prevalence and outcomes of caregiving after prolonged (≥ 48 hours) mechanical ventilation in the ICU.Chest2004;125,597-606. [CrossRef] [PubMed]
 
Tennstedt, S, Cafferata, GL, Sullivan, L Depression among caregivers of impaired elders.J Aging Health1992;4,58-76. [CrossRef] [PubMed]
 
Stevenson, JP Family stress related to home care of Alzheimer’s disease patients and implications for support.J Neurosci Nurs1990;22,179-187. [CrossRef] [PubMed]
 
Gallicchio, L, Siddiqi, N, Langenberg, P, et al Gender differences in burden and depression among informal caregivers of demented elders in the community.Int J Geriatr Psychiatry2002;17,154-163. [CrossRef] [PubMed]
 
Covinskky, KE, Newcomer, R, Fox, P, et al Patient and caregiver characteristics associated with depression in caregivers of patients with dementia.J Gen Intern Med2003;18,1006-1014. [CrossRef] [PubMed]
 
Nijboer, C, Triemstra, M, Tempelaar, R, et al Determinants of caregiving experiences and mental health of partners of cancer patients.Cancer1999;86,577-588. [CrossRef] [PubMed]
 
Walsh, SM, Schmidt, LA Telephone supports for caregivers of patients with cancer.Cancer Nurs2003;26,448-453. [PubMed]
 
Kozachik, SL, Given, CW, Given, BA, et al Improving depressive symptoms among caregivers of patients with cancer: results of a randomized clinical trial.Oncol Nurs Forum2001;28,1149-1157. [PubMed]
 
Sevick, MA, Sereika, S, Matthews, JT, et al Home-based ventilator-dependent patients: measurement of the emotional aspects of home caregiving.Heart Lung1994;23,269-278. [PubMed]
 
Sevick, MA, Bradham, DD Economic value of caregiver effort in maintaining long-term ventilator-assisted individuals at home.Heart Lung1997;26,148-157. [CrossRef] [PubMed]
 
Smith, CE, Mayer, LS, Perkins, SB, et al Caregiver learning needs and reactions to managing home mechanical ventilation.Heart Lung1994;23,157-163. [PubMed]
 
Miller, B, Townsend, A, Carpenter, E, et al Social support and caregiver distress: a replication analysis.J Gerontol B Psychol Sci Soc Sci2001;56,S249-S256. [CrossRef] [PubMed]
 
Weitzenkakmp, DA, Gerhart, KA, Charlifue, MW, et al Spouses of spinal cord injury survivors: the added impact of caregiving.Arch Phys Med Rehabil1997;78,822-827. [CrossRef] [PubMed]
 
Knight, BG, Lutzky, SM, Macofsky-Urban, F A meta-analytic review of interventions for caregiver distress: recommendations for future research.Gerontologist1993;33,240-248. [CrossRef] [PubMed]
 
Stommel, M, Want, S, Given, C, et al Confirmatory factor analysis as a method to assess measurement equivalence.Res Nurs Health1992;15,399-405. [CrossRef] [PubMed]
 
Blanchard, CG, Toseland, RW, McCallion, P The effects of a problem-solving intervention with spouses of cancer patients.J Psychosoc Oncol1996;14,1-21
 
Mittleman, MS, Ferris, S, Shulman, E, et al A comprehensive support program: effect of depression in spouse-caregivers of AD patients.Gerontologist1995;35,792-802. [CrossRef] [PubMed]
 
Vitaliano, PP, Zhang, J, Scanlan, JM Is caregiving hazardous to one’s physical health? A meta-analysis.Psychol Bull2003;129,946-972. [CrossRef] [PubMed]
 
Bland, J, Altman, D Statistical methods for assessing agreement between two methods of clinical measurement.Lancet1988;8,307-310
 
Landis, J, Koch, L The measurement of observer agreement for categorical data.Biometrics1977;33,159-174. [CrossRef] [PubMed]
 
Ensel, WM Measuring depression: the CES-D scale. Lin, N Dean, A Ensel, W eds.Social support, life events, and depression1986,51-70 Academic Press. New York, NY:
 
van Exel, NJ, Scholte op Reimer, WJ, Brouwer, WB, et al Instruments for assessing the burden of informal caregiving for stroke patients in clinical practice: a comparison of CSI, CRA, SCQ and self-rated burden.Clin Rehabil2004;18,203-214. [CrossRef] [PubMed]
 
Ware, JE, Kosinski, M Interpreting SF-36 summary health measures: a response.Qual Life Res2001;10,405-413. [CrossRef] [PubMed]
 
Madigan, EA, Fortinsky, RH Interrater reliability of the outcomes and assessment information set: results from the field.Gerontologist2004;44,689-692. [CrossRef] [PubMed]
 
Elashoff, J. nQuery advisor. 2002; Statistical Solutions Ltd. Boston, MA:.
 
Diehr, P, Patrick, DL Trajectories of health for older adults over time: accounting fully for death.Ann Intern Med2003;139,416-421. [PubMed]
 
Clyburn, LD, Stones, MJ, Hadjistavropoulos, T, et al Predicting caregiver burden and depression in Alzheimer’s disease.J Gerontol2002;55B,S2-S13
 
Meyers, JK, Weissman, MM Use of a self-report symptom scale to detect depression in a community sample.Am J Psychiatry1980;137,1081-1083. [PubMed]
 
Roberts, RE, Vernon, SW The Center for Epidemiologic Studies depression scale: its use in a community sample.Am J Psychiatry1983;140,41-46. [PubMed]
 
Cousins, R, Davies, AD, Turnbull, CJ, et al Assessing caregiving distress: a conceptual analysis and a brief scale.Br J Clin Psychol2002;41,387-403. [CrossRef] [PubMed]
 

Figures

Figure Jump LinkFigure 2. Categories of caregiver depression (CES-D score) at hospital discharge (n = 206) and 2 months later (n = 166) for caregivers in the experimental group. Solid black represents percent of patients who died or dropped out and whose caregivers did not provide data.Grahic Jump Location
Figure Jump LinkFigure 3. Categories of caregiver depression (CES-D score) at hospital discharge (n = 79) and 2 months later for caregivers in the control group (n = 56). Solid black represents percent of patients who died or dropped out and whose caregivers did not provide data.Grahic Jump Location
Figure Jump LinkFigure 4. Changes in levels of caregiver depression (CES-D score) over time for the experimental group (n = 205) and control group (n = 71). Solid black represents the percentage of patients who died or dropped out and of those whose caregivers did not provide data.Grahic Jump Location

Tables

Table Graphic Jump Location
Table 1. Comparison of Demographic Variables Between Experimental and Control Group Caregivers at Patients’ Hospital Discharge (n = 290)
* 

Forty-seven of the 211 experimental caregivers (22.3%) received telephone follow-up rather than in-person visits because they lived > 30 miles from the study site. There were no significant differences in outcomes between these caregivers and those who received in-person visits, and therefore the groups were combined for purposes of analysis.

 

Values given as the mean (SD).

 

Values given as No. (%).

Table Graphic Jump Location
Table 2. Comparison of Demographic and Clinical Variables Between Experimental and Control Group Patients at Hospital Discharge (n = 290)
* 

Values given as the mean (SD).

 

Proxy data were not included in SF-8 analyses. Group sizes for these analyses were as follows: experimental group, 141 patients; control group, 57 subjects.

 

ADL and IADL values are derived from the OASIS tool. Higher numbers indicate a higher degree of dependence in performing ADLs and IADLs.

§ 

Values given as No. (%).

 

Diagnoses taken from the International Classification of Diseases, ninth revision.

 

GI disease and cancer, for example.

# 

RR, 1.09; 95% CI, 0.62 to 1.94.

Table Graphic Jump Location
Table 3. Caregiver Outcomes at 2 Months for Experimental and Control Groups (n = 211)*
* 

In the interest of brevity, only 2-month postintervention scores are reported.

Table Graphic Jump Location
Table 4. Standardized Estimates From Ordinary Least-Squares Regressions of Caregiver Depression on Caregiver and Care-Receiver Variables (n = 211)
* 

Values are reported with the intervention variable out of the model.

 

For all subscales except self-esteem, higher numbers indicate a greater negative impact of the dimension upon caregiving. For self-esteem, higher numbers indicate a greater positive impact of that dimension upon caregiving.

References

Nelson, JE, Meier, DE, Litke, A, et al (2004) The symptom burden of chronic critical illness.Crit Care Med32,1527-1534. [CrossRef] [PubMed]
 
Girard, K, Raffin, TA The chronically critically ill: to save or let die.Respir Care1985;30,339-347. [PubMed]
 
Gracey, DR, Gillespie, D, Nobrega, F, et al Financial implications of prolonged ventilator care of Medicare patients under the prospective payment system.Chest1987;91,424-427. [CrossRef] [PubMed]
 
Douglas, P, Rosen, RL, Butler, PW, et al DRG payment for long-term ventilator patients.Chest1987;91,413-417. [CrossRef] [PubMed]
 
Carson, SS, Bach, PB, Brzozowski, L, et al Outcomes after long-term acute care.Am J Respir Crit Care Med1999;159,1568-1573. [PubMed]
 
Douglas, SL, Daly, BJ, Gordon, N, et al Survival and quality of life: short-term versus long-term ventilator patients.Crit Care Med2002;30,2655-2662. [CrossRef] [PubMed]
 
Douglas, SL, Daly, BJ Caregivers of long-term ventilator patients: physical and psychological outcomes.Chest2003;123,1073-1081. [CrossRef] [PubMed]
 
Im, K, Belle, SH, Schulz, R, et al Prevalence and outcomes of caregiving after prolonged (≥ 48 hours) mechanical ventilation in the ICU.Chest2004;125,597-606. [CrossRef] [PubMed]
 
Tennstedt, S, Cafferata, GL, Sullivan, L Depression among caregivers of impaired elders.J Aging Health1992;4,58-76. [CrossRef] [PubMed]
 
Stevenson, JP Family stress related to home care of Alzheimer’s disease patients and implications for support.J Neurosci Nurs1990;22,179-187. [CrossRef] [PubMed]
 
Gallicchio, L, Siddiqi, N, Langenberg, P, et al Gender differences in burden and depression among informal caregivers of demented elders in the community.Int J Geriatr Psychiatry2002;17,154-163. [CrossRef] [PubMed]
 
Covinskky, KE, Newcomer, R, Fox, P, et al Patient and caregiver characteristics associated with depression in caregivers of patients with dementia.J Gen Intern Med2003;18,1006-1014. [CrossRef] [PubMed]
 
Nijboer, C, Triemstra, M, Tempelaar, R, et al Determinants of caregiving experiences and mental health of partners of cancer patients.Cancer1999;86,577-588. [CrossRef] [PubMed]
 
Walsh, SM, Schmidt, LA Telephone supports for caregivers of patients with cancer.Cancer Nurs2003;26,448-453. [PubMed]
 
Kozachik, SL, Given, CW, Given, BA, et al Improving depressive symptoms among caregivers of patients with cancer: results of a randomized clinical trial.Oncol Nurs Forum2001;28,1149-1157. [PubMed]
 
Sevick, MA, Sereika, S, Matthews, JT, et al Home-based ventilator-dependent patients: measurement of the emotional aspects of home caregiving.Heart Lung1994;23,269-278. [PubMed]
 
Sevick, MA, Bradham, DD Economic value of caregiver effort in maintaining long-term ventilator-assisted individuals at home.Heart Lung1997;26,148-157. [CrossRef] [PubMed]
 
Smith, CE, Mayer, LS, Perkins, SB, et al Caregiver learning needs and reactions to managing home mechanical ventilation.Heart Lung1994;23,157-163. [PubMed]
 
Miller, B, Townsend, A, Carpenter, E, et al Social support and caregiver distress: a replication analysis.J Gerontol B Psychol Sci Soc Sci2001;56,S249-S256. [CrossRef] [PubMed]
 
Weitzenkakmp, DA, Gerhart, KA, Charlifue, MW, et al Spouses of spinal cord injury survivors: the added impact of caregiving.Arch Phys Med Rehabil1997;78,822-827. [CrossRef] [PubMed]
 
Knight, BG, Lutzky, SM, Macofsky-Urban, F A meta-analytic review of interventions for caregiver distress: recommendations for future research.Gerontologist1993;33,240-248. [CrossRef] [PubMed]
 
Stommel, M, Want, S, Given, C, et al Confirmatory factor analysis as a method to assess measurement equivalence.Res Nurs Health1992;15,399-405. [CrossRef] [PubMed]
 
Blanchard, CG, Toseland, RW, McCallion, P The effects of a problem-solving intervention with spouses of cancer patients.J Psychosoc Oncol1996;14,1-21
 
Mittleman, MS, Ferris, S, Shulman, E, et al A comprehensive support program: effect of depression in spouse-caregivers of AD patients.Gerontologist1995;35,792-802. [CrossRef] [PubMed]
 
Vitaliano, PP, Zhang, J, Scanlan, JM Is caregiving hazardous to one’s physical health? A meta-analysis.Psychol Bull2003;129,946-972. [CrossRef] [PubMed]
 
Bland, J, Altman, D Statistical methods for assessing agreement between two methods of clinical measurement.Lancet1988;8,307-310
 
Landis, J, Koch, L The measurement of observer agreement for categorical data.Biometrics1977;33,159-174. [CrossRef] [PubMed]
 
Ensel, WM Measuring depression: the CES-D scale. Lin, N Dean, A Ensel, W eds.Social support, life events, and depression1986,51-70 Academic Press. New York, NY:
 
van Exel, NJ, Scholte op Reimer, WJ, Brouwer, WB, et al Instruments for assessing the burden of informal caregiving for stroke patients in clinical practice: a comparison of CSI, CRA, SCQ and self-rated burden.Clin Rehabil2004;18,203-214. [CrossRef] [PubMed]
 
Ware, JE, Kosinski, M Interpreting SF-36 summary health measures: a response.Qual Life Res2001;10,405-413. [CrossRef] [PubMed]
 
Madigan, EA, Fortinsky, RH Interrater reliability of the outcomes and assessment information set: results from the field.Gerontologist2004;44,689-692. [CrossRef] [PubMed]
 
Elashoff, J. nQuery advisor. 2002; Statistical Solutions Ltd. Boston, MA:.
 
Diehr, P, Patrick, DL Trajectories of health for older adults over time: accounting fully for death.Ann Intern Med2003;139,416-421. [PubMed]
 
Clyburn, LD, Stones, MJ, Hadjistavropoulos, T, et al Predicting caregiver burden and depression in Alzheimer’s disease.J Gerontol2002;55B,S2-S13
 
Meyers, JK, Weissman, MM Use of a self-report symptom scale to detect depression in a community sample.Am J Psychiatry1980;137,1081-1083. [PubMed]
 
Roberts, RE, Vernon, SW The Center for Epidemiologic Studies depression scale: its use in a community sample.Am J Psychiatry1983;140,41-46. [PubMed]
 
Cousins, R, Davies, AD, Turnbull, CJ, et al Assessing caregiving distress: a conceptual analysis and a brief scale.Br J Clin Psychol2002;41,387-403. [CrossRef] [PubMed]
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

CHEST Journal Articles
PubMed Articles
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543