Hostname: page-component-cd9895bd7-q99xh Total loading time: 0 Render date: 2024-12-26T04:11:20.025Z Has data issue: false hasContentIssue false

The Oxford Positive Self Scale: psychometric development of an assessment of cognitions associated with psychological well-being

Published online by Cambridge University Press:  17 March 2023

Daniel Freeman*
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
Laina Rosebrock
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
Bao S. Loe
Affiliation:
The Psychometrics Centre, University of Cambridge, Cambridge, UK
Simone Saidel
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK
Jason Freeman
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK
Felicity Waite
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
*
Author for correspondence: Daniel Freeman, E-mail: daniel.freeman@psych.ox.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Background

Developing, elaborating, and consolidating positive views of the self is a plausible route to increased psychological well-being. We set out to provide an assessment of positive self-beliefs that could be used in research and clinical practice.

Methods

A non-probability online survey was conducted with 2500 UK adults, quota sampled to match the population for age, gender, ethnicity, income, and region. Exploratory factor analysis of a 94-item pool – generated with guidance from people with lived experience of mental health difficulties – was conducted to develop the Oxford Positive Self Scale (OxPos). The item pool was further reduced using regularised structural equation modelling (SEM) before confirmatory factor analysis. Optimal cut-off scores were developed using receiver operating characteristic curves. Additional validations were carried out with two further general population cohorts (n = 1399; n = 1693).

Results

A 24-item scale was developed with an excellent model fit [robust χ2 = 995.676; df = 246; CFI = 0.956; TLI = 0.951; RMSEA = 0.049 (0.047, 0.052); SRMR = 0.031]. The scale comprises four factors: mastery; strength; enjoyment; and character. SEM indicated that the scale explains 68.6% of variance in psychological well-being. The OxPos score was negatively correlated with depression (r = −0.49), anxious avoidance (r = −0.34), paranoia (r = −0.23), hallucinations (r = −0.20), and negative self-beliefs (r = −0.50), and positively correlated with psychological well-being (r = 0.79), self-esteem (r = 0.67), and positive social comparison (r = 0.72). Internal reliability and test–retest reliability were excellent. Cut-offs by age and gender were generated. A short-form was developed, explaining 96% of the full-scale variance.

Conclusions

The new open access scale provides a psychometrically robust assessment of positive cognitions that are strongly connected to psychological well-being.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Introduction

In determining psychological health, views of the self are central. Negative and positive views of the self, although inversely correlated, are distinct constructs that studies have shown are separate factors (Bryant & Baxter, Reference Bryant and Baxter1997; Faustino, Reference Faustino2022; Fowler et al., Reference Fowler, Freeman, Smith, Kuipers, Bebbington, Bashforth and Garety2006). Negative views of the self are an important element in understanding and treating mental health conditions. For example, cognitions concerning the self as a failure or unlikeable are considered a central cause of depression (Beck, Reference Beck1963). Such cognitions are then targeted in psychological interventions (Kovacs, Rush, Beck, & Hollon, Reference Kovacs, Rush, Beck and Hollon1981). Positive views of the self have less often been a focus of research. They have mainly been viewed – often within the field of positive psychology – as a factor in developing psychological well-being (Seligman, Reference Seligman2019). We similarly view positive self-beliefs as a route to psychological well-being, but also believe their development in mental health treatment can function as a potential counter-weight to negative self-perceptions. That is, they offer an alternative way to address low self-esteem. Accurate measurement of the factor of interest is essential for successful research and clinical practice. In this paper we report the development in a representative general population sample of a measure of positive self-beliefs that highlights potentially tractable cognitions to improve psychological well-being.

The dual continua model hypothesis proposes that mental health (psychological well-being) and mental ill health (disorders) are two separate but correlated dimensions (Keyes, Reference Keyes, David, Boniwell and Conley Ayers2013). Twin study data support this view of shared but also separate genetic and environmental causation for internalising disorders and psychological well-being (Kendler, Myers, Maes, & Keyes, Reference Kendler, Myers, Maes and Keyes2011). A similar perspective can be taken for negative and positive views of the self. Negative self-beliefs are likely to mean fewer positive self-beliefs. Nevertheless, the presence of negative self-beliefs does not imply the absence of positive self-beliefs. Similarly, the absence of negative self-beliefs does not guarantee the presence of positive self-beliefs. There are many scales across mental health research that assess negative thoughts, for example the Automatic Thoughts Questionnaire (Hollon & Kendall, Reference Hollon and Kendall1980) and the Cognition Checklist (Beck, Brown, Steer, Eidelson, & Riskind, Reference Beck, Brown, Steer, Eidelson and Riskind1987). Many other scales include negative thoughts within disorder-specific understandings, such as in the Posttraumatic Cognitions Inventory (Foa, Ehlers, Clark, Tolin, & Orsillo, Reference Foa, Ehlers, Clark, Tolin and Orsillo1999). Despite a focus on negative cognitions, the importance of positive cognitions has always been recognised, and counter-part scales such as the Positive Automatic Thoughts Questionnaire (ATQ-P) (Ingram, Kendall, Siegle, Guarino, & McLaughlin, Reference Ingram, Kendall, Siegle, Guarino and McLaughlin1995; Ingram & Wisnicki, Reference Ingram and Wisnicki1988) have been developed. The ATQ-P comprises 30-items forming four factors described as positive daily functioning (e.g. ‘Life is exciting’), positive self-evaluation (e.g. ‘I take good care of myself’), others evaluation of self (e.g. ‘I have a good sense of humour’), and positive future expectations (e.g. ‘My future looks bright’). The Brief Core Schema Scale (Fowler et al., Reference Fowler, Freeman, Smith, Kuipers, Bebbington, Bashforth and Garety2006) includes assessment of six positive self beliefs (e.g. ‘I am respected’ ‘I am valuable’ ‘I am talented’) that form a single factor. The Questionnaire about the Process of Recovery (Law, Neil, Dunn, & Morrison, Reference Law, Neil, Dunn and Morrison2014) comprises 15-items (e.g. ‘I feel better about myself’, ‘I feel able to take chances in life’ ‘I am able to develop positive relationships with other people’), developed from interviews about recovery from psychosis, that form a single factor.

However, in our view interventions designed to promote positive self-beliefs in order to enhance psychological well-being require –and currently lack – a scale that can select those in need, guide development of the intervention, and monitor outcomes. Such a scale would need to: be informed by people with lived experience of mental health problems; assess the cognitions highly connected to psychological well-being; assess the types of positive self-cognitions that are tractable with psychological approaches; be developed in a large representative cohort of the general population; enable interpretation by age and gender; be readily useable; and offer excellent reliability and validity. We set out to develop such a scale.

Methods

Participants

An online survey with a quota sampled UK participant group of 2500 adults (16 + years old) was conducted from 21st June 2022 to 28th July 2022 via a market research company. The quotas were based on UK Office for National Statistics population estimate data for gender, age, ethnicity, income, and region. Invited respondents did not know the topic of the survey before provisional agreement to complete it. Data from a further 1399 individuals who completed the survey via the market research company but did not form part of the representative cohort became an additional validation sample, as did a further 1693 individuals who were recruited separately via social media advertisements (the test-re-test group was drawn from this latter cohort). Ethical approval was obtained from the University of Oxford Medical Sciences Interdivisional Research Ethics Committee. Written informed consent was obtained from all participants.

Assessments

The Oxford Positive Self Scale (OxPos)

A pool of 94 items (see online Supplementary materials) assessing positive cognitions about the self was developed by the research group and a Lived Experience Advisory Panel (LEAP) of 14 young people with lived experience of psychosis, and there was also consideration of existing scales, particularly the Brief Core Schema Scale (Fowler et al., Reference Fowler, Freeman, Smith, Kuipers, Bebbington, Bashforth and Garety2006) and Social Comparison Scale (Allan & Gilbert, Reference Allan and Gilbert1995). Items were generated in the context of developing a new automated VR therapy (Phoenix) to enhance positive cognitions particularly for patients diagnosed with psychosis. Therefore there was a focus on positive cognitions that could be plausible targets in psychological intervention. Each scale item was rated on a scale comprising: Do not believe it (0), Believe it slightly (1), Believe it moderately (2), Believe it very much (3), and Believe it totally (4). Participants were asked to try to judge the beliefs on how they viewed themselves over the past week. Higher scores indicate greater endorsement of items. The final scale and the short-form are provided in online Supplementary materials.

Warwick-Edinburgh Mental Well-being Scale (WEMWBS)

(Tennant et al., Reference Tennant, Hiller, Fishwick, Platt, Joseph, Weich and Stewart-Brown2007). The WEMWBS is a fourteen-item scale assessing psychological well-being over the past fortnight. Each item is rated on a 1 (none of the time) to 5 (all of the time) scale, and therefore the total score can range from 14 to 70, with higher scores indicating a greater level of well-being. The Cronbach's alpha was 0.95.

Patient Health Questionnaire (PHQ-9)

(Kroenke, Spitzer, & Williams, Reference Kroenke, Spitzer and Williams2001). This scale assesses depressive symptoms over the past fortnight. Each of the nine items is rated on a 0 (not at all) to 3 (nearly every day) scale. Higher scores indicate higher levels of depression. The Cronbach's alpha was 0.92.

Oxford agoraphobic avoidance scale (O-AS)

(Lambe et al., Reference Lambe, Bird, Loe, Rosebrock, Kabir, Petit and Freeman2021). The O-AS lists eight simple tasks progressing in difficulty. Participants are asked whether they could do the task now or whether they could not because of anxiety (Yes = 0, No = 1), which provides the avoidance score (0–8). For each task participants are also asked on a 0 (no distress) to 10 (extreme distress) scale how anxious they would feel doing it. These distress scores are summed to provide an overall distress score. Higher scores indicate greater agoraphobic symptoms. The Cronbach's alpha was 0.91.

Revised green et al. paranoid thoughts scale (R-GPTS)

(Freeman et al., Reference Freeman, Loe, Kingdon, Startup, Molodynski, Rosebrock and Bird2021). The R-GPTS comprises an eight-item ideas of reference scale and a 10-item ideas of persecution scale. Each item is rated for the past two weeks on a 5-point (0 to 4) scale. Higher scores indicate greater levels of paranoia. The Cronbach's alpha was 0.97.

Cardiff anomalous perceptions scale-hallucinations (CAPS)

(Bell, Halligan, & Ellis, Reference Bell, Halligan and Ellis2006). This scale comprises eleven hallucination items taken from the CAPS. Each item (e.g. ‘Hear voices commenting on what you're thinking or doing’) is rated on a 0 (not at all) to 5 (daily) scale. Higher scores indicate greater levels of hallucinatory experiences. The Cronbach's alpha was 0.96.

Brief core schema scales (BCSS)

(Fowler et al., Reference Fowler, Freeman, Smith, Kuipers, Bebbington, Bashforth and Garety2006). The six-item negative (e.g. ‘I am unloved’, ‘I am worthless’) and the six-item positive self-belief (e.g. ‘I am successful’ ‘I am good’) sub-scales were used. Each item is rated on a scale from 0 (do not believe it) to 4 (believe it totally). Higher scores indicate greater endorsement of items. The Cronbach's alpha was 0.91.

Social comparison scale (SCS)

(Allan & Gilbert, Reference Allan and Gilbert1995). This measure comprises nineteen bipolar scales (e.g. inferior–superior, incompetent–competent, unlikeable–likeable) asking people to rate how they feel in comparison to others. Each was rated on a six-point Likert scale from ‘highly [e.g. inferior]’ to ‘highly [e.g. superior]’. Higher scores indicate a more positive view of the self in relation to others. The Cronbach's alpha was 0.97.

Rosenberg self-esteem scale (RSES)

(Rosenberg, Reference Rosenberg1965). The RSES is a 10-item, four-point scale (1–4) that assesses current levels of global self-esteem. The scale was scored so that a high total score was indicative of higher global self-esteem. The Cronbach's alpha was 0.89.

Analysis

Analyses were conducted in R, version 4.2.1 (R Core Team, 2022) with the psych (Revelle, Reference Revelle2020), lavaan (Rosseel, Reference Rosseel2012), semTools (Jorgensen et al., Reference Jorgensen, Pornprasertmanit, Schoemann, Rosseel, Miller, Quick and Rhemtulla2016), regSEM (Jacobucci, Reference Jacobucci2017) and cutpointr (Thiele & Hirschfeld, Reference Thiele and Hirschfeld2021) packages. We divided the data randomly and equally into two groups: an evaluation sample (n = 1250) and a validation sample (n = 1250). Exploratory factor analysis (EFA) was performed on the evaluation sample, and confirmatory factor analysis (CFA) on the validation sample. Prior to EFA, Bartlett's Test of Sphericity and the Kaiser-Mayer Olkin Measure of Sampling Adequacy (KMO) were used to evaluate the feasibility of factor recovery given the dataset (Bartlett, Reference Bartlett1950; Kaiser, Reference Kaiser1970). We used parallel analysis to determine the number of factors to retain.

EFA using Pearson correlations with oblique rotation and minimum residual factoring assessed the factorial structure of the new scale. Minimum residual was selected as the factoring method because of its robustness to non-normal data (Fabrigar, Wegener, MacCallum, & Strahan, Reference Fabrigar, Wegener, MacCallum and Strahan1999). Criteria used to retain items were based on factor loadings (>0.5), clarity of item content, and the theoretical coherence of subscales. CFA based on the robust maximum likelihood (MLR) estimator was subsequently carried out. Model goodness of fit was based on recommended guidelines where CFI and TLI >0.90 (Bentler, Reference Bentler1990), RMSEA <0.08 (Brown & Cudeck, Reference Brown, Cudeck, Bollen and Long1993), and SRMR <0.08 (Hooper, Coughlan, & Mullen, Reference Hooper, Coughlan and Mullen2008). The item pool was further reduced using regularised SEM (regSEM) with psychological well-being as the primary outcome (Jacobucci et al., Reference Jacobucci, Grimm and Mcardle2016) (see Section 1 of online supplementary materials for full details). The final set of items retained from regSEM analyses was re-evaluated using CFA. (We also performed the final CFA model on the two additional non-representative datasets to validate the model's goodness of fit.) We performed measurement invariance (MI) analyses based on age and gender to evaluate the items for potential bias (see Section 2 of online supplementary materials). We employed a higher-order CFA model to assess the appropriateness of calculating a total score. Internal consistency of the factors was assessed using omega coefficient (ω), and a one-week test–retest reliability based on 659 individuals repeating the questionnaire was evaluated by a two-way consistency intraclass correlation coefficient.

We created a short version of the new measure, in which two items were selected from each factor. The items were selected based on the highest factor loadings while ensuring that the item content was not too similar. We assessed the goodness-of-fit of the model based on the correlated factor model and the higher-order CFA model. The internal consistency of the factors was assessed using omega coefficient – ω (McDonald, Reference McDonald2013), and the overall score of the short version was correlated with the full version of the scale.

We conducted separate structural equation modelling (SEM) regressions between the new (long and short) scale factors and the well-being factor using the complete dataset. Furthermore, to identify the most important factors that predict well-being, accounting for shared variance, we also performed multiple SEM regressions using both long and short versions of the scale. Convergent and discriminant validity with the other scales was assessed using Pearson correlation coefficients.

Finally, optimal cut-off scores for the Oxford Positive Self Scale (OxPos) were developed using the receiver operating characteristic (ROC) curves (see online supplementary materials for further details). The ROC analyses were conducted using the cut-off points on the WEMWBS. We used the maximisation metric to maximise sensitivity and specificity. Specifically, the optimal cut-off score was calculated based on Youden's J statistic (Youden, Reference Youden1950), which occurs at the point which is furthest from the ROC curve and is interpreted as the maximum vertical distance between the ROC curve and the chance line (Schisterman, Perkins, Liu, & Bondell, Reference Schisterman, Perkins, Liu and Bondell2005).

Results

A summary of the socio-demographic characteristics of the participants is provided in Table 1.

Table 1. Socio-demographic information

The full item pool used to develop the scale is available in online Supplementary (Table S3.1). The Pearson correlation matrix was assessed for multi-collinearity (r < 0.95) and non-collinearity (r < 0.3), and three items (pool items 46, 52, and 77) were subsequently removed. Bartlett's test for Sphericity concluded that correlations between items were appropriate for EFA (χ2 = 118 120.3, df = 4095, p < 0.001). Sampling adequacy based on the KMO measure was at a superior level (overall KMO = 0.99, all items KMO>0.98). Parallel analysis showed that four or five-factor solutions appeared viable. Upon performing the EFA analysis, a four-factor solution emerged and was identified as the most appropriate model from a theoretical and empirical perspective. Iteratively running the EFA based on item removal criteria led to the retention of 46 items, with a further five items being removed due to ambiguous or repetitive item content (see online Supplementary Table S3.1). A CFA in the validation sample showed that a 41-item, 4 factor model was within the acceptable fit range [N = 1250; Robust χ2 = 3487.217, df = 773, p < 0.001, CFI = 0.916, TLI = 0.911, RMSEA = 0.053 (0.052–0.054), SRMSR = 0.039].

Four regSEM lasso were estimated with an iteration step of 0.03. The λ values for each regSEM lasso varied between 0 and 0.06 according to the final selected model with the lowest RMSEA. All items that had a coefficient of >0.02 were retained. The results showed that eight out of fourteen items were retained in the first factor, ten out of seventeen items were retained in the second factor, all five items were retained in the third factor, and four out of five items were retained in the fourth factor. Three additional items were removed after reviewing the item content qualitatively, resulting in a final set of 24 items with excellent model fit (N = 1250; robust χ2 = 995.676; df = 246; CFI = 0.956; TLI = 0.951; RMSEA = 0.049 (0.047–0.052); SRMR = 0.031). The factor loadings and factor correlations can be found in Tables 2 and 3.

Table 2. Confirmatory factor loadings based on 24 items (validation data)

a Items selected for the short-form version.

Table 3 . Factor correlations based on 24 items (validation data)

Model fit results were considered satisfactory based on the additional data collected from the non-representative sample [N = 1399; Robust χ2 = 820.16, df = 246, p < 0.001, CFI = 0.969, TLI = 0.965, RMSEA = 0.041 (0.038–0.043), SRMSR = 0.027] and data collected via social media advertisement [N = 1693; Robust χ2 = 2385.533, df = 246, p < 0.001, CFI = 0.912, TLI = 0.901, RMSEA = 0.072 (0.069–0.074), SRMSR = 0.048]. The factor loadings and correlation of the additional CFAs can be found in Section 4 of online Supplementary materials.

Concerning gender and age measurement invariance, the configural invariance model achieved good model fit across all indices in both models (see online Supplementary Tables S5-1-2). Adding metric and scalar constraints only led to minute changes in fit and improvement in BIC, indicating that scalar invariance held across gender and age groups. Thus, the factor scores could be directly compared between age and gender groups. A comparison between gender groups revealed that the latent factor means for the first three factors were significantly higher for male participants than female participants (F 1 = 0.153, adjusted-p = 0.032; F 2 = 0.224, adjusted-p = 0.0004; F 3 = 0.155, adjusted-p = 0.024). No significant differences were found for the fourth factor (F 4 = −0.033, adjusted-p = 0.538). Latent factor means for older participants were found to be significantly higher across all factors (F 1 = 0.194, adjusted-p < 0.001; F 2 = 0.178, adjusted-p < 0.001; F 3 = 0.252, adjusted-p < 0.001; F 4 = 0.298, adjusted-p < 0.001).

A higher-order CFA model was carried out to evaluate the appropriateness of calculating a total scale score. The result showed excellent model fit [N = 1250; Robust χ2 = 1023.153, df = 248, p < 0.001, CFI = 0.955, TLI = 0.95, RMSEA = 0.05 (0.047–0.053), SRMSR = 0.034]. All subscales showed excellent omega coefficient: F 1, ω = 0.93; F 2, ω = 0.94; F 3, ω = 0.92; F 4, ω = 0.88. The overall internal consistency of the test was excellent (ω = 0.92). The one-week test–retest reliability (N = 659) for the subscales and overall scores were excellent: F 1 = 0.88; F 2 = 0.87; F 3 = 0.82; F 4 = 0.81; overall = 0.90).

Two items from each factor were selected to form an 8-item short-form measure. The goodness-of-fit for the correlated CFA model [N = 1250; robust χ2 = 44.078; df = 14; CFI = 0.993; TLI = 0.986; RMSEA = 0.041 (0.03–0.054); SRMR = 0.015] and higher order CFA model [N = 1250; robust χ2 = 56.936; df = 16; CFI = 0.990; TLI = 0.983; RMSEA = 0.045 (0.035–0.056); SRMR = 0.021] were excellent. The factor loadings, factor correlations and correlated residuals can be found in the online Supplementary materials (Section 6 online Supplementary Tables S6-1:3). The subscales and overall measure showed excellent internal consistency (F 1, ω = 0.86; F 2, ω = 0.83; F 3, ω = 0.82; F 4, ω = 0.81; overall, ω = 0.89). The total score of the short-form version had a correlation of r = 0.98 with the total score of the full 24-item version (explaining 96% of the variance in the full version). We also evaluated the 8-item measure based on the additional data. Correlated factor model fit results were considered excellent based on the non-representative sample [N = 1399; Robust χ2 = 23.804, df = 14, p < 0.048, CFI = 0.998, TLI = 0.995, RMSEA = 0.022 (0.008–0.035), SRMSR = 0.012] and data collected via social media advertisement [N = 1703; Robust χ2 = 89.039, df = 14, p < 0.001, CFI = 0.985, TLI = 0.971, RMSEA = 0.056 (0.047–0.066), SRMSR = 0.02]. The factor loadings and correlation of these additional CFAs can be found in online supplementary materials (online Supplementary Tables S6-3–S6-6).

For the long and short versions of the scale, separate SEM regressions showed that the individual factors were each predictive of well-being (see online Supplementary Table S7-1). For the 24-item version, regressing all four positive thoughts factors onto the well-being factor in a single SEM showed that only the first three factors were significant predictors with good model fit [N = 2500; robust χ2 = 3847.593; df = 656; CFI = 0.942; TLI = 0.938; RMSEA = 0.044 (0.043–0.045); SRMR = 0.032]. The fourth factor resulted in a negative coefficient, which was previously positive in the simple SEM regressions. Such a phenomenon was likely due to suppressor effects caused by shared variance among the predictors (Maassen & Bakker, Reference Maassen and Bakker2001) and hence was removed using a backward elimination approach. The final SEM model with three significant predictors explained 68.6% of the variance in well-being (see online Supplementary Table S7–2). For the short-form, the model fit was excellent [N = 2500; robust χ2 = 1476.493; df = 201; CFI = 0.953; TLI = 0.946; RMSEA = 0.050 (0.048–0.052); SRMR = 0.048], but only the first and third factors were significant predictors of well-being (see Table 4), with 70% of the variance explained.

Table 4 . Correlations in the representative cohort (N = 2500) between the Oxford Positive Self Scale and the other assessments (using raw scores for all scales) (all correlations are significant at p < 0.001)

As summarised in Table 4, the 24-item and 8-item OxPos scales had strong positive correlations with psychological well-being (WEMBS), positive social comparison (SCS), self-esteem (RSES), and positive self-beliefs (BCSS), moderate negative correlations with depression (PHQ-9), negative self-beliefs (BCSS), and agoraphobia (O-AS), and small negative correlations with paranoia (R-GPTS) and hallucinations (SPEQ).

In the representative group the mean score for the 24-item OxPos scale was 57.94 (s.d. = 21.26) (N = 2500) (mean scores by corresponding percentile scores are presented in Section 9 of online Supplementary materials). The mean score for the WEMWBS was 45.34 (s.d. = 12.16) (N = 2500). The correlation between the OxPos and WEMWBS was high (r = 0.79, p < 0.001). To determine cut-offs for the OxPos that will allow discrimination between participants who have a very low or a low level of positive self-beliefs in relation to psychological well-being from the rest of the population, we used the lowest 15 and 25% of population scorers on the WEMWBS (score = 33, score = 38 respectively), representing approximately 1.01 standard deviations and 0.6 standard deviations below the average score in the sample. ROC analysis identified 46 as an optimal cut-off point (sensitivity = 0.809; specificity = 0.835) with an overall AUC of 0.899 for the lowest 15% of the WEMWBS and 50 as an optimal cut-off point (sensitivity = 0.811; specificity = 0.811) with an overall AUC of 0.889 for the lowest 25% of the WEMWBS. A score of 46 on the Oxford Positive Self Scale would represent 0.56 standard deviations below the mean, which is the lowest 29% of the sample, and a score of 50 on the Positive Thoughts scale would represent 0.37 standard deviations below the mean, which is the lowest 36% of the sample.

In the representative group the mean score for the 8-item OxPos scale was 19.63 (s.d. = 7.4) (N = 2500) (mean scores by corresponding percentile scores are presented in Section 9 of online Supplementary materials). The correlation between the OxPos scale and the WEMWBS was moderately high (r = 0.77, p < 0.001). Given the small number of items in the OxPos short version, we only used the lowest 25% of scorers on the WEMWBS to determine the cut-off point. ROC analysis identified 16 as an optimal cut-off point (sensitivity = 0.857; specificity = 0.731) with an overall AUC of 0.882. A score of 16 on the OxPos short-form scale would represent approximately 0.49 standard deviations below the mean, which is the lowest 33% scorers in the sample. A summary of cut-offs including by age and gender are provided in Table 5 (with further detail provided in Sections 8 and 9 of the online Supplementary materials).

Table 5 . Cut-offs for the overall sample and by age and gender for the OxPos long and short-forms

Discussion

The Oxford Positive Self Scale is very clear for people to complete, assesses cognitions closely connected to psychological well-being, and has excellent reliability and validity established in a representative general population cohort. The four factors of the questionnaire link to established psychological intervention techniques. Beliefs about achieving things, doing things well, and succeeding relate to behavioural activation and mastery and control methods (Dimidjian, Barrera, Martell, Muñoz, & Lewinsohn, Reference Dimidjian, Barrera, Martell, Muñoz and Lewinsohn2011). Beliefs about coping, not giving up, and keeping going relate to commonly used behavioural experiments in challenging situations (Bennett-Levy et al., Reference Bennett-Levy, Butler, Fennell, Hackman, Mueller and Westbrook2004). Beliefs about enjoying things and being able to relax relate to savouring and relaxation techniques (Manzoni, Pagnini, Castelnuovo, & Molinari, Reference Manzoni, Pagnini, Castelnuovo and Molinari2008; Seligman, Steen, Park, & Peterson, Reference Seligman, Steen, Park and Peterson2005). Beliefs about being a good person relate to strengths and values identification (Seligman et al., Reference Seligman, Steen, Park and Peterson2005). Hence the scale may prove useful in assessing interventions to increase psychological well-being. There is also a short-form, explaining most of the variance in the full version, that could be used for weekly monitoring during intervention or in larger scale epidemiological research studies of well-being. It is also notable that the scale includes cut-offs by gender and age that can identify people who may benefit from an evidence-based intervention. Overall this is a scale likely to fit the experiences of those asked to complete it, provide direction for intervention, and enable precise assessment of the concept of positive self-beliefs.

There are limitations to the scale development. We used a non-probability online quota sampling method, which is a much stronger approach than most scale development studies that use convenience approaches, but it will have still introduced bias as to who was approached to take part. We do know that, taken as a whole, the respondents in this survey were broadly representative of the adult general population on a number of basic demographic features but not that individual respondents were representative of the general population. It is notable that the level of recent reported contact with mental health services was high, which may have affected cut-off scores. We used a large item pool, developed with people with lived experience of mental health difficulties, but the items included would not have been exhaustive, and therefore other types of important positive self-beliefs could well have been omitted. We would not claim that these are the only types of positive self-beliefs to target to improve psychological well-being. Nor do we know whether these cognitions are causal in improving well-being, which can only be determined via randomised controlled experiments or trials that intervene on the cognitions. At this stage we do not know whether the measure will be sensitive to change, although there is nothing to suggest in its face validity that this would not be the case, nor what the minimal clinically important change in the cognitions might be. Finally, although the representative sample would have included people with mental health problems and the Oxford Positive Self Scale correlated as expected with dimensional symptom measures, it would be valuable to obtain data on the scale from clinical cohorts in mental health services.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S0033291723000624.

Acknowledgements

We would like to thank the Phoenix VR project lived experience advisors and the McPin Foundation. The study was funded by a grant from the International Foundation, with further support from the NIHR Oxford Health Biomedical Research Centre (BRC). DF is an NIHR Senior Investigator. FW is supported by a Wellcome Trust Doctoral Fellowship. The views expressed are those of the authors and not necessarily those of the National Health Service, NIHR, or the Department of Health.

References

Allan, S., & Gilbert, P. (1995). A social comparison scale: Psychometric properties and relationship to psychopathology. Personality and Individual Differences, 19, 293299.10.1016/0191-8869(95)00086-LCrossRefGoogle Scholar
Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Psychology, 3, 7785.Google Scholar
Beck, A. T. (1963). Thinking and depression: I. Idiosyncratic content and cognitive distortions. Archives of General Psychiatry, 9, 324333.10.1001/archpsyc.1963.01720160014002CrossRefGoogle ScholarPubMed
Beck, A. T., Brown, G., Steer, R. A., Eidelson, J. I., & Riskind, J. H. (1987). Differentiating anxiety and depression utilizing the cognition checklist. Journal of Abnormal Psychology, 96, 179183.10.1037/0021-843X.96.3.179CrossRefGoogle Scholar
Bell, V., Halligan, P. W., & Ellis, H. D. (2006). The Cardiff Anomalous Perceptions Scale (CAPS). Schizophrenia Bulletin, 32, 366377.10.1093/schbul/sbj014CrossRefGoogle ScholarPubMed
Bennett-Levy, J. E., Butler, G. E., Fennell, M. E., Hackman, A. E., Mueller, M. E., & Westbrook, D. E. (2004). Oxford Guide to behavioural experiments in cognitive therapy. Oxford: Oxford University Press.10.1093/med:psych/9780198529163.001.0001CrossRefGoogle Scholar
Bentler, P. M. (1990). Comparative Fit indexes in structural models. Psychological Bulletin, 107, 238246.10.1037/0033-2909.107.2.238CrossRefGoogle ScholarPubMed
Brown, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In Bollen, K. A. & Long, J. S. (Eds), Testing structural equation model (pp. 136162). Newbury Park, CA: Sage.Google Scholar
Bryant, F. B., & Baxter, W. J. (1997). The structure of positive and negative automatic cognition. Cognition and Emotion, 11, 225258.10.1080/026999397379908CrossRefGoogle Scholar
Dimidjian, S., Barrera, M. Jr., Martell, C., Muñoz, R. F., & Lewinsohn, P. M. (2011). The origins and current status of behavioral activation treatments for depression. Annual Review of Clinical Psychology, 7, 138.10.1146/annurev-clinpsy-032210-104535CrossRefGoogle ScholarPubMed
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272299.10.1037/1082-989X.4.3.272CrossRefGoogle Scholar
Faustino, B. (2022). Maladaptive and adaptive cognitions about the self and others: Confirmatory factor analysis of the brief core schemas scales. Psychological Reports, 00332941211063602.Google ScholarPubMed
Foa, E. B., Ehlers, A., Clark, D. M., Tolin, D. F., & Orsillo, S. M. (1999). The posttraumatic cognitions inventory (PTCI): Development and validation. Psychological Assessment, 11, 303314.10.1037/1040-3590.11.3.303CrossRefGoogle Scholar
Fowler, D., Freeman, D., Smith, B., Kuipers, E., Bebbington, P., Bashforth, H., … Garety, P. (2006). The brief core schema scales (BCSS): Psychometric properties and associations with paranoia and grandiosity in non-clinical and psychosis samples. Psychological Medicine, 36, 749759.10.1017/S0033291706007355CrossRefGoogle ScholarPubMed
Freeman, D., Loe, B. S., Kingdon, D., Startup, H., Molodynski, A., Rosebrock, L., … Bird, J. C. (2021). The revised Green et al., paranoid thoughts scale (R-GPTS): Psychometric properties, severity ranges, and clinical cut-offs. Psychological Medicine, 51, 244253.10.1017/S0033291719003155CrossRefGoogle Scholar
Hollon, S. D., & Kendall, P. C. (1980). Cognitive self-statements in depression: Development of an automatic thoughts questionnaire. Cognitive Therapy and Research, 4, 383395.10.1007/BF01178214CrossRefGoogle Scholar
Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6, 5360.Google Scholar
Ingram, R. E., Kendall, P. C., Siegle, G., Guarino, J., & McLaughlin, S. C. (1995). Psychometric properties of the positive automatic thoughts questionnaire. Psychological Assessment, 7, 495507.10.1037/1040-3590.7.4.495CrossRefGoogle Scholar
Ingram, R. E., & Wisnicki, K. S. (1988). Assessment of positive automatic cognition. Journal of Consulting and Clinical Psychology, 56(6), 898.10.1037/0022-006X.56.6.898CrossRefGoogle ScholarPubMed
Jacobucci, R. (2017). regsem: Regularized structural equation modeling. arXiv preprint arXiv:1703.08489.Google Scholar
Jacobucci, R., Grimm, K. J., & Mcardle, J. J. (2016). Regularized structural equation modeling. Structural Equation Modeling: a Multidisciplinary Journal, 23, 555566.10.1080/10705511.2016.1154793CrossRefGoogle ScholarPubMed
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., Rosseel, Y., Miller, P., Quick, C., … Rhemtulla, M. (2016). Package ‘semTools’. Website. Retrieved from https://cran.opencpu.org/web/packages/semTools/semTools.pdf.Google Scholar
Kaiser, H. F. (1970). A second generation little jiffy. Psychometrika, 35, 401415.10.1007/BF02291817CrossRefGoogle Scholar
Kendler, K. S., Myers, J. M., Maes, H. H., & Keyes, C. L. (2011). The relationship between the genetic and environmental influences on common internalizing psychiatric disorders and mental well-being. Behavior Genetics, 41(5), 641650.10.1007/s10519-011-9466-1CrossRefGoogle ScholarPubMed
Keyes, C. L. M. (2013). Promotion and protection of positive mental health. In David, S., Boniwell, I., & Conley Ayers, A. (Eds.), The Oxford handbook of happiness (pp. 915925). Oxford: Oxford University Press.Google Scholar
Kovacs, M., Rush, A. J., Beck, A. T., & Hollon, S. D. (1981). Depressed outpatients treated with cognitive therapy or pharmacotherapy: A one-year follow-up. Archives of General Psychiatry, 38, 3339.10.1001/archpsyc.1981.01780260035003CrossRefGoogle ScholarPubMed
Kroenke, K., Spitzer, R., & Williams, J. (2001). The PHQ-9. Journal of General Internal Medicine, 16, 606613.10.1046/j.1525-1497.2001.016009606.xCrossRefGoogle ScholarPubMed
Lambe, S., Bird, J., Loe, B., Rosebrock, L., Kabir, T., Petit, A., … Freeman, D. (2021). The Oxford Agoraphobic Avoidance Scale. Psychological Medicine, DOI: https://doi.org/10.1017/S0033291721002713.CrossRefGoogle Scholar
Law, H., Neil, S. T., Dunn, G., & Morrison, A. P. (2014). Psychometric properties of the questionnaire about the process of recovery (QPR). Schizophrenia Research, 156, 184189.10.1016/j.schres.2014.04.011CrossRefGoogle ScholarPubMed
Maassen, G. H., & Bakker, A. B. (2001). Suppressor variables in path models: Definitions and interpretations. Sociological Methods & Research, 30, 241270.10.1177/0049124101030002004CrossRefGoogle Scholar
Manzoni, G. M., Pagnini, F., Castelnuovo, G., & Molinari, E. (2008). Relaxation training for anxiety: A ten-years systematic review with meta-analysis. BMC Psychiatry, 8, 112.10.1186/1471-244X-8-41CrossRefGoogle ScholarPubMed
McDonald, R. P. (2013). Test theory: A unified treatment. New York: Psychology Press.10.4324/9781410601087CrossRefGoogle Scholar
R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.R-project.org/.Google Scholar
Revelle, W. (2020). psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston, Illinois: Northwestern University. R package version 2.0. 9.Google Scholar
Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.10.1515/9781400876136CrossRefGoogle Scholar
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 136.10.18637/jss.v048.i02CrossRefGoogle Scholar
Schisterman, E. F., Perkins, N. J., Liu, A., & Bondell, H. (2005). Optimal cut-point and its corresponding youden Index to discriminate individuals using pooled blood samples. Epidemiology, 16, 7381.10.1097/01.ede.0000147512.81966.baCrossRefGoogle ScholarPubMed
Seligman, M. E. (2019). Positive psychology: A personal history. Annual Review of Clinical Psychology, 15, 123.10.1146/annurev-clinpsy-050718-095653CrossRefGoogle ScholarPubMed
Seligman, M. E. P., Steen, T. A., Park, N., & Peterson, C. (2005). Positive psychology progress: Empirical validation of interventions. American Psychologist, 60, 410421.10.1037/0003-066X.60.5.410CrossRefGoogle ScholarPubMed
Tennant, R., Hiller, L., Fishwick, R., Platt, S., Joseph, S., Weich, S., … Stewart-Brown, S. L. (2007). The Warwick-Edinburgh mental well-being scale (WEMWBS): Development and UK validation. Health and Quality of Life Outcomes, 5, 63.10.1186/1477-7525-5-63CrossRefGoogle ScholarPubMed
Thiele, C., & Hirschfeld, G. (2021). cutpointr: Improved estimation and validation of optimal Cutpoints in R. Journal of Statistical Software, 98(11), 127.10.18637/jss.v098.i11CrossRefGoogle Scholar
Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3, 3235.10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-33.0.CO;2-3>CrossRefGoogle ScholarPubMed
Figure 0

Table 1. Socio-demographic information

Figure 1

Table 2. Confirmatory factor loadings based on 24 items (validation data)

Figure 2

Table 3 . Factor correlations based on 24 items (validation data)

Figure 3

Table 4 . Correlations in the representative cohort (N = 2500) between the Oxford Positive Self Scale and the other assessments (using raw scores for all scales) (all correlations are significant at p < 0.001)

Figure 4

Table 5 . Cut-offs for the overall sample and by age and gender for the OxPos long and short-forms

Supplementary material: File

Freeman et al. supplementary material 1
Download undefined(File)
File 37.1 KB
Supplementary material: File

Freeman et al. supplementary material 2
Download undefined(File)
File 112.1 KB