The validity of the Edmonton Symptom Assessment System–Depression item for screening for depression in individuals with cancer pain: A cross-sectional study

Abstract

Context

Depression is common in individuals with cancer and pain, negatively impacts quality of life, treatment adherence, tumor progression, and survival.

Objectives

The primary aims of this study were to (1) evaluate the validity of the Edmonton Symptom Assessment System’s depression (ESAS-D) for detecting major depressive disorder (MDD) as diagnosed by a psychiatrist and (2) identify the best cutoff for this purpose in a sample of cancer pain individuals. The secondary aim was to compare ESAS-D with another commonly used screening measure (Patient Health Questionnaire-2 [PHQ-2]) for classifying individuals as meeting or not meeting Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria for depression.

Methods

49 cancer pain individuals completed the ESAS-D and PHQ-2 Within 2 weeks, a psychiatrist interviewed the participants and determined whether or not they met criteria for MDD based on the DSM-5.

Results

The ESAS-D demonstrated acceptable accuracy and validity for classifying MDD. A cutoff of ≥2 was identified as being best able to balance sensitivity (85%) and specificity (76%) and had an overall accuracy of 79%. A receiver operating characteristic curve analysis showed an area under the curve (AUC) of 0.81 (95% confidence interval [CI]: 0.68–0.94). The ESAS-D also compared favorably with the modified Thai PHQ-2 (sensitivity, 75%; specificity, 72%; overall accuracy, 73%; AUC, 0.74 [95% CI: 0.59–0.88]) for identifying MDD individuals.

Conclusions

The ESAS-D showed acceptable sensitivity, specificity, and overall accuracy for screening for MDD in cancer and pain. It could therefore be used to screen for probable depression in this population.

Introduction

Cancer is the second leading cause of death worldwide, with approximately 19.3 million new cases and 10 million deaths reported in 2020 (Sung et al. 2021). Moreover, cancer-related pain has been reported to be experienced by 50% to 70% of individuals with cancer, with a high prevalence at advanced disease stages (66%) (van den Beuken-van Everdingen et al. 2016). People with cancer face a variety of challenges beyond the experience of pain that have negative effects on their emotions and quality of life.

Depression is a common condition affecting 121 million people worldwide (Massie 2004), including individuals with cancer. The overall prevalence of depression in individuals with cancer has been reported to range from 8% to 32% (mean, 21%) (Riedl and Schuessler 2021), with 30% in the Thai cancer population (Maneeton et al. 2012). Moreover, there is evidence that depression can have a negative impact on tumor progression (Bortolato et al. 2017) and adversely impacts both treatment adherence and survival (Watson et al. 1999).

Research has shown that individuals who have both cancer and pain are more likely to have depression and anxiety than those who have cancer without pain (Li et al. 2017). A systematic review found an average prevalence of depression in individuals with cancer and pain of 37% (range, 22%–49%). The review also reported a significant positive association between pain intensity and depression (Laird et al. 2009). Although a multidisciplinary approach has been implemented in our clinic for cancer pain management, its efficacy is only 66% (Wangnamthip et al. 2021). This low level of efficacy may be due, at least in part, to the inadequate assessment and treatment of depression in these patients.

Major depressive disorder (MDD) is usually diagnosed via interviews conducted by psychologists or psychiatrists using the diagnostic criteria listed in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (American Psychiatric Association and Association 2013). However, because such interviews take time and require the availability of a psychologist or psychiatrist, neither of which may be readily available when needed. An alternative approach to assessing depression is by using one of several self-report measures, such as the Hospital Anxiety and Depression Scale (Zigmond and Snaith 1983).

One of the most commonly used measures for assessing cancer-related symptoms is the 11-item Edmonton Symptom Assessment System (ESAS) (Onitilo et al. 2006). This measure includes items that assess pain, nausea, tiredness, anxiety, drowsiness, appetite, well being, and shortness of breath, as well as a single item that assesses depression severity (ESAS-D). The ESAS can also be used to monitor the severity of these symptoms over time (Onitilo et al. 2006). Given the frequency with which the ESAS is used in cancer patient populations, and given that the measure includes an item assessing depressive symptom severity, it would be useful to evaluate the extent to which a patient’s response to this item might be used to screen for MDD. Two studies have been published that have examined this issue, including one study with individuals with hematological malignancies (Ripamonti et al. 2014) and a second with individuals with terminal cancer (Brenne et al. 2016). However, the findings from these studies do not necessarily generalize to individuals with both cancer and pain; research is needed to study this latter population.

Given these considerations, the primary aims of this study were to (1) evaluate the diagnostic utility of the ESAS-D for classifying individuals who do and do not meet criteria for having an MDD and (2) identify the best cutoff for this purpose in individuals with cancer-related pain. Our goal was to identify a cutoff what would have at least adequate sensitivity, specificity, and overall accuracy (i.e., all ≥70%). A secondary aim was to compare the ESAS depression item to another well-established screening measure for depression commonly used in Thailand, the modified Thai Patient Health Questionnaire-2 (PHQ-2) (Thailand 2014).

Materials and methods

Overall study design

Figure 1 presents a graphic representation of the study design, as well as participant flow through the study procedures. As can be seen, patients with cancer seen at the Siriraj Pain Clinic in Bangkok, Thailand, were contacted and screened for possible study inclusion, and those who were eligible and interested were asked to read and sign a consent form. All participants received standard care from the pain specialists at the clinic, using the analgesic guidelines of the World Health Organization and a multidisciplinary approach. Demographic data were collected, and the ESAS-D and a modified Thai PHQ-2 were administered to all of the eligible participants who signed the informed consent form. Within 2 weeks of enrollment, participants underwent a clinical interview with a psychiatrist to determine whether they met the DSM-5 criteria for MDD. The psychiatrist conducting the interview was blinded to the results of the ESAS-D and modified Thai PHQ-2 assessments. The Siriraj Institutional Review Board reviewed and approved the study’s protocol (Si 276/2018) prior to the initiation of the study procedures.

DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; ESAS-D, Edmonton Symptom Assessment System–Depression; MDD, major depressive disorder; PHQ-2, Patient Health Questionnaire-2.

Figure 1. STARD flow diagram.

Participants

In order to participate in the study, potential participants needed to have a cancer diagnosis, have significant pain, and be 18 years old or older. We excluded potential participants who were unable to communicate in Thai or who were otherwise unable to understand the evaluation forms and questionnaires used in the study, such as individuals with severe cognitive impairments. Based on the power analysis conducted to determine the sample sizes needed in each of the study groups (i.e., with vs. without depression, see below), we enrolled participants to achieve a specific sample size for each group; n = 20 for those who met criteria for MDD and n = 29 for those who did not.

Measures

Demographic and cancer history variables

Demographic data on age, sex assigned at birth, height and weight (for computing body mass index), marital status, history of smoking, alcohol use, and drug abuse and education level were collected at the first visit. Information about the participants’ cancer stage, cancer site, and history of cancer treatments was also extracted from the medical record at this time.

Depression item from the Edmonton Symptom Assessment System

The depression item from the Thai version of the ESAS (Chinda et al. 2011) was administered as a part of the 9-item ESAS to all study participants. The ESAS is widely used to evaluate the severity of 9 specific symptoms in patients with cancer. With the ESAS, respondents are asked to rate the severity of each symptom, including depression (specific endpoints, “No depression” and “Worst possible depression”) on 0 to 10 numerical rating scales. We used only the depression item (ESAS-D) in the current analyses.

Thai modified Patient Health Questionnaire-2

The Thai modified PHQ-2 is a commonly used self-report tool for screening for depression in Thailand, which is distributed by the Ministry of Public Health (Thailand 2014). It consists of the first 2 questions of the full 9-item Patient Health Questionnaire (PHQ-9) (Kongsuk et al. 2018; Lotrakul et al. 2008). The 2 items ask about whether the respondent has (1) felt depressed or (2) lost interest in things or activities they would typically enjoy doing in the preceding 2 weeks, respectively. With the modified version of the measure, respondents provide a dichotomous (yes or not) response to each of the 2 items. Respondents who provide a positive response to one or both of these questions are classified as being at risk for meeting criteria for having an MDD.

Psychiatrist interviews to classify participants

The study psychiatrist used DSM-5 criteria to classify all participants as meeting or not meeting criteria for MDD. The DSM-5 is used to diagnosis depression and other psychiatric conditions worldwide. According to the DSM-5 criteria, a diagnosis of MDD requires 5 or more symptoms to be present within the past 2 weeks. At least one of the symptoms should be a depressed mood or anhedonia (loss of interest or pleasure). The other possible symptoms of MDD include appetite or weight changes, sleep difficulties, psychomotor agitation or retardation, fatigue or loss of energy, diminished ability to think or concentrate, feelings of worthlessness or excessive guilt, and significant suicidal ideation. Each of these symptoms is rated by the interviewer as present or absent. To meet criteria, a symptom in question cannot be due to the physiological effects of a substance or other medical condition. Additionally, an occurrence of MDD should not be better explained by the individual meeting criteria for a schizoaffective, schizophreniform, or delusional disorder; schizophrenia; any other schizophrenia spectrum disorder; or any other psychotic disorder. Finally, there must also not be a history of one or more manic or hypomanic episodes (American Psychiatric Association D and Association AP 2013).

Sample size considerations

Ripamonti and colleagues reported that when screening for significant depression, a cutoff of 3 (i.e., ≥3) on the ESAS-D corresponded to a cutoff of 11 on the Hospital Anxiety and Depression Scale (with 87% sensitivity and 90% specificity) (Ripamonti et al. 2014). The sample size calculations for the present study were conducted using nQuery Advisor (V.6); a confidence interval (CI) for proportion with normal approximation (n large) was calculated. To estimate the infinite depressed (“MDD”) and non-depressed (“non-MDD”) group sizes, we chose proportions of 0.87 and 0.90 and errors of 0.15 and 0.11, respectively, and an alpha of 0.05 (95% CI) based on DSM-5 as a gold standard. The required sample sizes were calculated to be at least 20 for the MDD group and 29 for the non-MDD group (Daniel and Cross 2018; Ngamjarus 2016).

Statistical analyses

To describe the study sample, we computed the numbers and percentages (categorical variables), means and standard deviations (normally distributed continuous variables), and medians and interquartile ranges (IQR) (non-normally distributed continuous variables) for the demographic and cancer history variables. Next, we planned to compare the participants who met criteria for MDD with those who did not meet criteria for MDD using the independent t-test or Mann–Whitney U tests for the continuous descriptive variables, including the ESAS-D, that were normally and non-normally distributed, respectively. Between-group comparisons of categorical variables were conducted using Pearson’s chi-square or Fisher’s exact test. An alpha (α) value of <0.05 was deemed significant.

Next, to test the first study hypothesis, we compared the ability of the ESAS-D item to classify participants as meeting criteria for MDD or not for each possible EAS-D cutoff. We then identified the cutoff that had the best balance between sensitivity and specificity. We then conducted an area under the receiver operating characteristic curve (AuROC) analysis to evaluate the accuracy of the ESAS-D item responses for classifying the participants as having an MDD using the best cutoff identified, and compared this (as well as sensitivity and specificity) with the PHQ-2 using a positive cutoff for at least 1 of 2 questions. All analyses were performed using the SPSS (V.18.0) software package.

Results

Patient demographics and cancer history information

Ninety-six individuals were recruited for the study and 49 of these provided complete data. Of these, 20 participants were identified as meeting DSM-5 criteria for having an MDD by the study psychiatrist. Descriptive information for the sample as a whole and the 2 subsamples is presented in Table 1. As can be seen, the average age of the study participants was 58.9 ± 12.3. There were no significant between-group differences in any of the descriptive variables between the MDD and non-MDD groups. As would be expected, the median ESAS-D rating was significantly higher in the MDD group (median = 5.0, IQR = 3.0–6.0) than in the non-MDD group (median = 0.0, IQR = 0.0–1.0; p < 0.001). Moreover, the median of the total ESAS score was significantly higher in the MDD than the non-MDD participants, as shown in Table 2. However, there was no statistical difference in the median values of the ESAS pain intensity item between the MDD and non-MDD participants (median = 5.0 [4.0–7.5] vs. 4.0 [3.0–6.0], p = 0.138). More detailed information regarding the demographic details of the sample is presented in Tables 1 and 2

Table 1. Demographic variables for the whole sample and as a function of MDD group

p < 0.05 indicates statistical significance using the independent t-test, chi-square or Fisher’s exact test, and Mann–Whitney U test.

BMI, body mass index; ENT, ear, nose and throat; ESAS-D, Edmonton Symptom Assessment System–Depression; MDD, major depressive disorder.

Table 2. Comparing ESAS score between MDD and Non-MDD group

* p < 0.05 indicates statistical significance using Mann–Whitney U test.

The data are presented as median (IQR).

ESAS; Edmonton Symptom Assessment System, MDD; Major Depressive Disorder.

.

The validity of ESAS-D for screening for MDD

Table 3 compares the screening and diagnostic performance of each ESAS-D cutoff score for classifying someone as meeting or not meeting DSM-5 criteria for a MDD. The optimum ESAS-D cutoff that balanced sensitivity (85%) and specificity (76%) for classifying the participant meeting DSM-5 criteria for MDD depression was ≥2.

Table 3. Sensitivity, specificity, positive predictive value, negative predictive value, accuracy, and likelihood ratios for classifying participants as meeting criteria for MDD based on different ESAS-D scores

DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; ESAS-D, Edmonton Symptom Assessment System–Depression; LR, likelihood ratio; NPV, negative predictive value; PPV, positive predictive value; TP, true positive; FP, false positive; TN, true negative; FN, false negative.

Screening abilities of the ESAS-D and modified Thai PHQ-2

Table 4 and Figure 2 present the results comparing the success of the ESAS-D and modified Thai PHQ-2 to screen for MDD. As can be seen, the number (percent) of true and false positives of participants were 17 (85%) vs. 7 (24%); p < 0.001 when using an ESAS-D cutoff of ≥2 and 15 (75%) vs. 8 (28%); p = 0.001 when using a modified Thai PHQ-2 cutoff of ≥1 (i.e., at least one positive answer to the 2 PHQ-2 questions). Also, using a cutoff of ≥2, the ESAS-D had an AuROC of 0.81 (95% CI: 0.68–0.94). Using a cutoff of ≥1 for the PHQ-2 (Thailand 2014), the modified Thai PHQ-2 evidenced a sensitivity of 75%, specificity of 72%, and AuROC of 0.74 (95% CI: 0.59–0.88) for classifying the participants as meeting DSM-5 criteria for MDD in the current sample.

Table 4. Sensitivity, specificity, positive predictive value, negative predictive value, accuracy, likelihood ratios, and area under the receiving operator curve for classifying participants as meeting criteria for MDD based on DSM-5

AuROC, area under receiving operator characteristic curve; DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; ESAS-D, Edmonton Symptom Assessment System–Depression; NPV, negative predictive value; PHQ-2, Patient Questionnaire–2; PPV, positive predictive value; TP, true positive; FP, false positive; TN, true negative; FN, false negative.

DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; ESAS-D, Edmonton Symptom Assessment System–Depression; PHQ-2, Patient Questionnaire.

Figure 2. Receiver operating characteristic (ROC) curves associated with use of the ESAS-D and the modified Thai PHQ-2 for classifying participants as having or not having MDD.

Discussion

The findings support the validity of the ESAS-D for screening for depression in cancer patients who also have significant pain. The ESAS-D cutoff that best balanced specificity and sensitivity for this purpose was ≥2. This result contrasts with a systematic review and meta-analysis, which concluded that an ESAS-D cutoff score of ≥4 is best for detecting depression (sensitivity 53% [95% CI: 38%–67%] and specificity 90% [95% CI: 82%–94%]) (Boonyathee et al. 2018). This discrepancy may be due to the possibility that the best cutoff may be population specific. In our study, the participants were individuals with cancer experiencing pain. Such individuals may have symptoms that are more burdensome than those with cancer who do not experience pain and may therefore be more prone to psychological distress. Consistent with this possibility, Yamaguchi et al. (2016) also concluded that an ESAS-D score of ≥2 was the most valid cutoff for detecting depression in a sample of individuals with cancer, although their validity criterion was the Quick Inventory of Depressive Symptomatology-Self-Report and not an actual diagnosis of MDD made by a psychiatrist.

Another important study finding was that the single-item ESAS-D compared favorably with the two-item interview as a screening tool for meeting criteria for having MDD. The PHQ-2 is used worldwide to screen for MDD. Lee and colleagues found that the sensitivity and specificity of the standard PHQ-2 in screening for depression in elderly using a cutoff of 2 were 80% and 73%, respectively (Lee and Dajpratham 2017). However, our study revealed that the modified Thai PHQ-2 had a markedly lower specificity (72%) than that reported by Lee and colleagues. Again, the inconsistency in findings may be due to important differences in the study samples and how depression was assessed. Thus, it is possible that the modified Thai PHQ-2 may not perform well as a screening tool for detecting depression in some populations, such as those with cancer and pain. In such cases, the ESAS may be a better tool, not only because the depression item evidenced good validity as a screener for depression but also because the measure assesses numerous additional common cancer symptoms and can therefore provide an estimate of a patient’s overall pain and psychological function.

The utilization of a single item ESAS-D to screen for depression in individuals receiving palliative care or who are in the advanced stages of cancer holds significant importance. Chochinov et al. (1997), in a study cited earlier, found that a single-item screening question, specifically asking “Are you depressed?,” had high accuracy in detecting MDD in patients with terminal cancer as compared to either a two-item short-form Beck Depression Inventory administered via interview or a Visual Analog Scale (Chochinov et al. 1997). Therefore, the implementation of a single-item 0–10 scale to screen for depression would appear to be highly useful for use in individuals with cancer and accompanying pain.

Limitations

This study has a number of limitations that should be taken into account when interpreting the results. First, the participants were recruited from a population of patients being treated in a single-center tertiary setting within one country, thereby limiting the generalizability of our findings. Additionally, we did not collect information about the participants’ specific pain syndromes or diagnoses related to cancer pain. Not knowing these clinical details makes it challenging to be confident about the generalizability of the findings. Future researchers should assess and provide more details about the study samples if possible. Such details could be provided by measures such as the Edmonton Classification System for Cancer Pain (Fainsinger et al. 2019). Relatedly, the participants in this study all resided in a single country. We were therefore unable to determine if country of residence had a moderating impact on the findings. This provides further support for the need to replicate the current findings in other populations, including individuals who live in different countries. Third, although the power analysis conducted indicated that a sample size of 49 would be adequate for estimating the sensitivity and specificity of the ESAS-D item in the current study, this sample size is lower than that used by many other studies in this area, which have sample sizes that can range from 146 to as many as 969 (Boonyathee et al. 2018). Another limitation pertains to the relatively high dropout rate, which was largely due to functional decline stemming from natural prognosis of cancer, which prevented full adherence to the protocol. Also, to our knowledge, this is the first time that the sensitivity and specificity of the ESAS-D as a screening tool for depression has been evaluated in a sample of individuals with cancer and pain, using DSM-5 as a gold standard. Again, additional research is needed ‒ ideally with larger sample sizes ‒ to determine the reliability of the current findings.

Summary and conclusions

Despite the study’s limitations, the findings provide important new information regarding the potential utility and validity of the ESAS-D item as a screening tool for depression in individuals with cancer and pain. Given that the prevalence of depression is high in individuals with cancer and that depression may influence tumor progression and cancer therapy adherence, early detection and effective treatment of significant depression in outpatient pain clinics are important. The ESAS is widely used to assess common cancer symptoms, including depression. This makes the ESAS-D very practical as a screening measure in this setting. The current findings support this item’s use for this purpose and indicate that a cutoff score of ≥2 is the most accurate cutoff, balancing the need for both sensitivity and specificity. Future research is needed to evaluate the reliability and generalizability of the current findings by studying additional samples of individuals with cancer and pain in different settings.