The RAS-24: Development and validation of an adherence-to-medication scale for severe mental illness patients

Abstract

Introduction:

Several studies have found that most patients with severe mental illness (SMI) and comorbid (physical) conditions are partially or wholly nonadherent to their medication regimens. Nonadherence to treatment is a serious concern, affecting the successful management of patients with SMIs. Psychiatric disorders tend to worsen and persist in nonadherent patients, worsening their overall health. The study described herein aimed to develop and validate a scale (the Ralat Adherence Scale) to measure nonadherence behaviors in a culturally sensitive way.

Materials and Methods:

Guided by a previous study that explored the primary reasons for nonadherence in Puerto Rican patients, we developed a pool of 147 items linked to the concept of adherence. Nine experts reviewed the meaning, content, clarity, and relevance of the individual items, and a content validity ratio was calculated for each one. Forty items remained in the scale’s first version. This version was administered to 160 patients (21–60 years old). All the participants had a diagnosis of bipolar disorder, major depressive disorder, or schizoaffective disorder. The STROBE checklist was used as the reporting guideline.

Results:

The scale had very good internal consistency (Cronbach’s alpha = 0.812). After a factor analysis, the scale was reduced to 24 items; the new scale had a Cronbach’s alpha of 0.900.

Conclusions:

This adherence scale is a self-administered instrument with very good psychometric properties; it has yielded important information about nonadherence behaviors. The scale can help health professionals and researchers to assess patient adherence or nonadherence to a medication regimen.

Introduction

Nonadherence to treatment is a serious concern that affects the positive management and prognosis of patients with severe mental illness (SMI). Pharmacotherapy is essential for the successful management of these patients. For a person with a psychiatric illness, keeping to a prescribed regimen of medication is critical. The illness of a patient who fails to adhere will inevitably worsen and persist, which will lead to a deterioration of the patient’s overall health; such nonadherence carries a substantial economic burden, as well [1, 2]. This is a major problem for patients around the world. The World Health Organization (2020) has reported that the proportion of medication nonadherence is 50% in patients with chronic diseases [3]. Although researchers have reported a nonadherence rate of 20% to 60%, several studies have found that 40% to 60% of psychiatric patients are partially or wholly nonadherent to their prescription medications [4–6]. Lack of medication adherence is a common and potent (though modifiable) risk factor for poor outcomes [7]. Hispanics are more likely to be nonadherent to psychiatric medication and other treatments [8–10].

Individuals with SMIs are also susceptible to a variety of physical conditions considered to be risk factors for CVD, such as hypertension, metabolic syndrome, hyperlipidemia, type 2 diabetes, and abdominal obesity, among others [11, 12]. These conditions tend to be poorly treated, also exerting detrimental effects on SMI patients [13]. Not only CVD prevalence rates but also the prevalence rates for its risk factors are about twice as high in SMI patients (for example, in those with affective disorders) as they are in the general population [14, 15].

We could not find an instrument in Puerto Rico that assesses nonadherence or poor adherence to medication; while some surveys have been translated in other countries, none have been translated and validated for our population.

For that reason, we decided to develop a valid instrument for measuring treatment adherence/nonadherence in SMI patients with one or more of the following CVD risk factors: hypertension, obesity or abdominal obesity, and/or a generally unhealthy lifestyle (e.g., eats poorly, smokes cigarettes, is physically inactive). We included patients with these comorbidities because the leading cause of premature mortality in this population is related to CVD and CVD risk factors [16–19]. Up to now, no gold standard for measuring treatment adherence in Puerto Rican SMI patients with CVD-related comorbidities exists. Furthermore, the questionnaires that are available are Spanish translations of English-language instruments that have never been psychometrically validated for our population. There is a lack of appropriate and validated instruments that can be used to measure adherence–nonadherence behaviors, especially in Puerto Rico. We designed and tested just such an instrument that measures just such behaviors.

Instrument development requires a sample that has those attributes being assessed [20]. For that reason, we used a qualitative approach to provide social and cultural context in the construction of our instrument [21]. We used a psychometric approach to test the instrument for internal consistency and content validity. To create our instrument, we followed the eight-step process devised by DeVellis [16] for scale development. Our preliminary published data established that nonadherence behaviors are a complex phenomenon with a variety of patterns [21]. The categories of our study were used to create this scale. There are barriers to adherence to treatment that are related to the medications for a given individual’s mental illness and barriers to adherence to treatment for those diseases and conditions that are themselves risk factors for CVD. Patients named stigma (toward those with a mental illness), patient- and medicine-related issues, poor family support, and factors related to the patient–provider relationship as barriers adhering to the drug regimen prescribed for their psychiatric condition. For the category of barriers to adherence to medications intended to reduce CVD risk, the participants revealed having certain patient-related reasons, mentioned the fact that healthcare personnel does not always provide adequate follow-up care, and named stigma and the lack of support as additional factors.

The aim of this research was to develop and validate a scale to measure adherence and nonadherence behaviors in a culturally sensitive way, taking into consideration the barriers that prevent a patient from taking his or her medication for the treatment of a psychiatric disorder with a comorbid physical condition. This new scale for SMI patients captures the adherence barriers that were determined in our previous study [21] and can be used by healthcare professionals in targeting interventions that encourage treatment adherence by considering the needs and characteristics of the individual patient.

Methods

Scale Development and Validation

The Institutional Review Board of the University of Puerto Rico, Medical Sciences Campus, approved the study. Informed consent was obtained from all the subjects involved in the study. The STROBE checklist was used as the reporting guideline. (See Supplementary Table 1). This study consisted of two phases. The scale was in Spanish. The first phase consisted of applying the first five of the eight steps of scale development that are recommended by DeVellis [16]. To that end, we first determined what we wanted to measure, as was previously detailed in our earlier article [21].

Second, we generated a set of test items forming a pool of 147 items, all related to the concepts of adherence and nonadherence, as established by the literature [6, 7, 21]. The items were linked to the categories related to the barriers mentioned in the literature and explored by previous research done by the main author.

Third, we determined the format of the measurement (i.e., a Likert scale). The options offered in that scale consist of “totally disagree,” “partially disagree,” “partially agree,” and “totally agree.”

Fourth, we had the items in the initial item pool reviewed (in terms of content) by a group of nine experts. Each expert rated these items according to meaning, content, clarity, and relevance. Three psychiatrists, one cardiologist, two clinical psychologists, and three clinical social workers comprised the experts. In addition to being acknowledged experts in their individual fields, the members of our panel had multiple years working with the issue of adherence (Fig. 1). A standard review guide that included the definitions of adherence and nonadherence was developed. Our experts rated the relevant of each item in terms of what we intended to measure. They examined the content and face validity of the scale and then gave feedback about each item.

Fig. 1. Experts: Years in practice and years working with the issue of adherence.

Fifth, we included in the preliminary scale the items selected by the experts after determining that each one’s calculated content validity ratio (CVR) was acceptable [22]. The number of persons making up the panel of experts corresponded to the minimum value of the CVR for each. In this case, a given item needed a ratio of 0.78 to 1 for it to be retained. This procedure was essential to our maximizing the content validity of the scale. A content validity index (0.83) was calculated for the whole test after the items to be included in the first version had been identified; that index indicated that 83% of the items that were included in the instrument were acceptable. Our preliminary scale had 40 items; the CVRs of those items ranged from 0.78 to 1.0, which, according to Lawshe [22], is adequate.

The sixth through eighth steps were part of the second phase.

In that second phase, then, we, sixth, administered the scale to a sample. Seventh, we evaluated the items using factor analysis. Eighth, we choose the items of the final scale and obtained a Cronbach’s alpha for that scale.

Participants

The first version of the scale was administered to 160 patients who had been recruited from a clinical psychology practice associated with a private academic institution in San Juan and from the regional branches of an outpatient-serving governmental health agency in several cities in Puerto Rico. A social worker or clinical psychologist at each facility invited a given possible candidate to participate in the study. After the initial approach, the PI was notified that she should contact the candidate. The participants answered a questionnaire that elicited sociodemographic information and questions intended to gather mental and physical health data. Data collection was conducted from February 2017 to December 2019. The participants had bipolar disorder (BD), type I, type II, or unspecified; major depressive disorder (MDD); or schizoaffective disorder (SD). This was a convenience sample. All subjects signed written informed consent.

The inclusion criteria were the following: To participate, the individual 1) had to be taking medication for both physical and mental illnesses; 2) could have one or more risk factors for hypertension, obesity, or abdominal obesity (i.e., a body mass index of 30 kgm² or more), diabetes, or high cholesterol or triglycerides; 3) could have high LDL levels; and 4) could have a generally unhealthy lifestyle that included having a poor diet, smoking cigarettes, and/or being physically inactive. People with substance abuse problems at the moment of the interview or who were in the midst of a suicidal crisis were excluded from the study. We used the mini-mental state examination, version 2 (MMSE-2) to rule out dementia and severe cognitive deterioration as part of the exclusion criteria.

Reliability, Validity, and Statistical Analysis

Descriptive statistics were used to calculate the demographic characteristics of the sample. A one-way X ² test was used to analyze adherence/nonadherence through the scale. A two-way X ² test was used to compare scale scores by sex, BD (type I and type II), MDD, and schizoaffective disorders and to compare the scale scores with the participants’ perceptions of their problems with taking medication. A one-way between-groups analysis of variance (ANOVA) was used to analyze the variable of education as it relates to adherence and nonadherence behaviors. We hypothesized that there will be differences between adherence and nonadherence scale scores and several patients’ variables related to sex, diagnosis, self-perception of adherence behaviors, and cognitive functioning.

The scale was tested for internal consistency and content validity. We computed the Cronbach’s alphas for the scale scores. An exploratory factor analysis was conducted to uncover the latent dimensions among the items. We calculated statistics using IBM SPSS version 21 software. Statistical significance was set at a = 0.05, two-sided.

Results

Baseline Sample Characteristics

Forty-six percent of the participants had schizoaffective disorder; 32%, BD; and 22%, major depressive disorder. The mean age of the participants was 45.5 years (SD = 11.1; range 21–60 years). Sixty-three percent were female. In regard to education, 36% had completed high school. The majority of the participants were single (66.3%). Fifty-nine percent of the sample was from an urban area. (See Table 1 for all the sociodemographic characteristics of the sample.) All the participants (100%) reported that they were currently using psychiatric medication, and 21% of them reported having problems taking their medication and not adhering to their medication during the week that they were recruited for the study. (See Table 2 for the psychiatric diagnosis, medical comorbidities, treatment details, and lifestyle characteristics of the participants by gender.)

Table 1. Sociodemographic characteristics

Table 2. Psychiatric diagnosis, medical comorbidities, treatment details, and lifestyle characteristics, by gender

† Forgot to take my medication in the week prior to taking the questionnaire.

Using X ² analysis, statistically significant relationships were found in the following variables: obesity and gender (Supplementary Fig. 1), stress and gender (Supplementary Fig. 2), past drug use and gender (Supplementary Fig. 3), and exercise and gender (Supplementary Fig. 4). We found a statistical relationship between high levels of stress and having a depression disorder (Supplementary Fig. 5). In addition, we found a statistical relationship between diagnosis and number of medications (Supplementary Fig. 6).

Factor Analysis

As part of the exploratory factor analysis, we used principal component analysis to extract the maximum variance and put it into the first factor to obtain the minimum number of factors. The latter analysis revealed that the scale had five components with eigenvalues greater than 1, accounting for 57.5% of the total variance explained (Table 3). The first component had an eigenvalue of 7413, followed by 2431 for the second, 1506 for the third, 1234 for the fourth, 1205 for the fifth, accounting for 30.89%, 10.13%, 6.28%, 5.14%, and 5.02%, respectively, of the variance. Supplementary Fig. 7 shows the scree plot. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.85 (good). The Bartlett’s test of sphericity was p < 0.001. We included the distribution of answers for each question of the scale (Table 4). No Bias was found in this sample in terms of their responses to the RAS-24.

Table 3. Total variance explained

Extraction Method: Principal Component Analysis.

Table 4. Distribution of answers to the 24 items that remain in the RAS-24

Reliability

The original scale of 40 items had high internal consistency (Cronbach’s alpha = 0.812), making it a very good index (according to Kline, 2005) [23]. However, after the factor analysis, the scale was reduced to 25 items to compact it and make it more flexible for future use. We choose the items that correlated between 0.509 and 0.718, but not values that were too high because extreme multicollinearity and singularity would cause difficulties in determining the unique contribution of the variables to a factor. The Cronbach’s alphas of the deleted items fluctuated from 0.888 to 0.900. When removing one of the items, the Cronbach’s alpha of the final version of the scale was 0.900. Then, the final scale consisted of 24 items. The principal component analysis for the 24 items that remained revealed five components related to adherence and nonadherence behaviors.

Adherence/Nonadherence Behaviors

We analyzed the data using a one-way X ² test. Different from other studies, in which frequencies of from 20% to a 60% were observed [4–6], the frequencies observed in this sample showed that 10% of our sample population had positive adherence behaviors and 90% had nonadherence behaviors, X ² (1, N = 160) = 102.4, p < 0.001. Ten percent of men and 10% of women adhered to their medication regimens. Persons with BD had 12% adherence and 88% nonadherence behaviors. For MDD, 6% had adherence, and 94% had nonadherence behaviors. Those with SD had 11% adherence and 89% nonadherence behaviors.

There was no statistically significant difference between males and females in the scale scores. There were no differences in the scale scores by diagnosis. However, we found significant differences between the scale scores and several patient variables as follows.

We used a one-way, between-groups ANOVA to analyze the variable of education in terms of adherence and nonadherence behaviors. Least significant difference post hoc comparisons examined differences between groups (p < 0.05). Participants were classified into four groups: individuals having 0 to 4 years of education, those having 5 to 8 years of education, those having 9 to 12 years of education, and those having an undergraduate or higher level of education. Data are presented as the mean ± standard deviation Participants with an undergraduate or higher degree were found to be more adherent to medication (1.82 ± 0.384 [95% CI, 1.75–1.89]; p = 0.016) in comparison to the group of participants with 5–8 years of education (2.00 ± 0.000 [95% CI, 1.84–2.16], p = 0.044) and that of the participants with 9–12 years of education (1.95 ± 0.229 [95% CI, 1.87–2.01]; p = 0.016). The differences between these groups were statistically significant, F(3, 156) = 2.80; p = 0.042; η ² = 0.051. Having had more years of education contributes to positive adherence behaviors. Supplementary Fig. 8 shows the distribution of the different levels of education with their corresponding adherence/nonadherence behaviors

Using a question that explored the difficulties of taking medication, we examined how the scores of our adherence scale matched up (or did not match up) with the perceptions of the participants regarding their own adherence or lack thereof. Before working with the adherence scale, each participant was asked whether he or she had any difficulties taking his or her medication. This question was part of the sociodemographic questionnaire. Our intention in using this question was to assess each participant’s scores regarding adherence in light of his or her own perceptions of the difficulties in adhering to a medication regimen. We analyzed our data using a 2 (problem taking medication) × 2 (measures of adherence/nonadherence behaviors ascertained by the scale) X ² test. Though only 21% of the participants reported having difficulties taking their medication, the scale scores indicated that 90% of them had nonadherence behaviors (X ² (1, N = 160) = 4.80; p < 0.05; ϕ = 0.173).

Finally, there was a statistically significant relationship between participants with cognitive impairment and nonadherence behaviors. We used a 2 (adherence/nonadherence behaviors) × 2 (normal/impairment cognitive functioning) X ² test. Fifty-one percent of nonadherent patient had cognitive impairment (X ² (1, N = 160) = 3.83; p < 0.05; ϕ = −0.155). A more detailed analysis of cognitive impairment by diagnosis with this sample was published by the main author [24].

Discussion and Conclusions

We developed and validated a culturally sensitive adherence-to-treatment scale. Compared with other studies [4–6], the number of nonadherence behaviors in this study differed from what has been found. In our study, there were more participants with nonadherence behaviors. Education was one of the variables with a significant effect on adherence and nonadherence behaviors different from other studies. Adherence rates were higher in those participants with higher levels of education. Cognitive impairment is another variable that could have an influence related to nonadherence behaviors. Using the MMSE-2, we detected that 51% of patients with nonadherence behaviors had cognitive impairment.

This study supports the notion that adherence to medication regimens can be estimated in SMI patients based on the education level of the individual as well as other variables. Different from other studies, in which participants with high levels of education were found to be more adherent (though not statistically significant so), our study found having a high level of education to be a statistically significant variable that was associated with both adherence and nonadherence behaviors [25, 26].

Prior to their being interviewed, we asked the participants what—if any—difficulties they had in managing their own medication regimens. The goal was to later assess the differences between perceived and actual (as determined by our scale) adherence; the answers to this question revealed that nonadherence behaviors were more frequently practiced than the participants thought them to be. In the literature, several studies have indicated that self-reports (compared to other assessment methods) tend to overestimate adherence behavior [27, 28]. Social desirability is one of the reasons that patients tend to overestimate their effectiveness in managing their own medication regimens. To help remedy this issue, social desirability must be addressed and a validated measure of adherence (one that focuses on the population of interest) used.

The psychometric properties of this particular scale were considered to be very good. To the best of our knowledge, this is the first scale in Puerto Rico to measure adherence and nonadherence to medication regimens in a population of patients with an SMI. Effectively identifying nonadherence behaviors is the first step in developing and, subsequently, promoting, psychosocial interventions that can enhance treatment adherence; such interventions would be an adjunct to pharmacotherapy [29]. Pharmacotherapy is the recommended first-line treatment for SMI patients, but medication adherence is frequently poor, causing relapses and worsening the psychiatric symptoms and general health of these patients [2, 30]. Our scale identified adherence and nonadherence behaviors in a sample of SMI patients (Table 5). This scale will be known as the Ralat Adherence Scale (RAS-24) and will consist of 24 items aimed at assessing adherence and nonadherence in SMI patients. The RAS-24 will make it possible for healthcare professionals to explore adherence barriers in their patients.

Table 5. The 24-item Ralat adherence scale (RAS-24: English translation)

Persons wishing to use the scale are requested to formally contact the main author. The scale is copyrighted.

Five principal components related to adherence and nonadherence behaviors were identified. The first component included four items related to adherence behaviors. We also identified the reasons for nonadherence, classifying these reasons as being patient-related, medication-related, stigma related, or related to a lack of support from family members. These five components will help healthcare professionals to identify not only the nonadherence behaviors but also the reasons for their existence, thereby enabling these professionals to offer interventions that promote adherence behaviors. When it comes to medication adherence and nonadherence, tailored psychoeducation has been proven to be more effective than generalized education. The scale described in this article offers information about which barriers a given patient has and helps the health professional to create a tailored psychosocial intervention to better manage that patient’s issues with medication; the purpose is to increase the efficaciousness of this kind of intervention by taking into account the individual’s personal characteristic, needs, beliefs, and attitudes [31].

This study has several limitations. First, the participants were not randomly recruited and are not representative of all the Puerto Rican SMI patients. Second, patients with other chronic mental illnesses (e.g., schizophrenia) were not included in the sample. Third, the reliability of the scale over time (using a test–retest strategy) was not evaluated. More studies with greater numbers of patients and in other populations with the same and other chronic illnesses are needed to test the RAS-24. In the future, measuring test–retest reliability and performing subsequent validation with confirmatory factor analysis are recommended.

Despite the limitations mentioned, the results provide relevant information about the psychometric properties of the RAS-24.

In conclusion, the RAS-24 is a self-administered instrument with very good psychometric properties; it has already yielded important information about nonadherence–adherence behaviors. The scale can help health professionals and researchers to assess patient adherence or nonadherence to a medication regimen. Identifying nonadherence behaviors and their causes in patients with an SMI will aid in the provision of psychoeducational and psychosocial interventions to both these patients and—when applicable—their caretakers, thereby promoting improvements in their (the patients) psychiatric and comorbid conditions.

The scale can be used for future research examining both preventive measures and potential treatments.