A prognosis serves at least three important functions in patient management for common mental disorders in primary care: it may be crucial to treatment decisions (Reference GoldbergGoldberg, 1992; Reference Katon, Lin and VonKorffKaton et al, 1994; Reference Spijker and NolenSpijker & Nolen, 1998); it provides the physician with a norm to monitor the course of the disorder; and it satisfies the needs of patients and relatives to get an idea of the future consequences of the disorder. Despite these important functions, no data exist on the accuracy of the general practitioner's (GP) prognosis in daily practice.
Aim
It is the aim of the present study to establish the agreement of the GP prognosis with the actual course observed, for ICD-10 (World Health Organization, 1993) cases of depression or generalised anxiety identified in primary care. Depression and generalised anxiety were chosen because these are the most common mental disorders in primary care (Reference Goldberg, Lecrubier, Üstün and SartoriusGoldberg & Lecrubier, 1995).
The agreement between prognosis and course may be limited by influences on recovery that cannot be foreseen at the time the prognosis is made. Studies on the role of life changes have identified events that may either speed up or slow down recovery from common mental disorders (Reference Davies, Rose and CrossDavies et al, 1983; Brown et al, Reference Brown, Adler and Bifulco1988, Reference Brown, Lemyre and Bifulco1992; Reference Leenstra, Ormel and GielLeenstra et al, 1995). To some degree these events occur randomly, making it impossible to incorporate their effect in the prognosis.
In order to assess the accuracy of the GP prognosis, the observed agreement with the course of the disorder will be compared with the predictive power of a statistical model. The development of this model is described elsewhere (further details available from the first author upon request). The model was built upon a broad range of prognostic factors identified in the literature. Only factors that were potentially available to the GP at the time the prognosis was made were included. The model provides optimal predictions for the predictors included. The predictive power of the model may therefore be considered an estimate of the degree to which the 1-year course of depression and generalised anxiety in primary care is in fact predictable. The observed agreement between GP prognosis and course will be compared with this estimate of the maximum agreement attainable for GPs.
METHOD
Selection of GPs
Eighteen GPs participating in a study of the effects of postgraduate training for GPs on recognition, diagnosis and management of common mental disorders in primary care (Reference Jenner, Van der Meer and Van OsJenner et al, 1995) were asked to judge, for a stratified sample of their patients, whether there were mental health problems and, if so, what the prognosis was. The GPs came from practices in the northern part of The Netherlands (Reference Tiemens, Ormel and JennerTiemens et al, 1999; Reference Van Os, Ormel and Van den BrinkVan Os et al, 1999).
Subjects and diagnosis
Samples of consecutive patients attending their GP at randomly selected days were collected (Reference VonKorff, Üstün, Üstün and SartoriusVonKorff & Üstün, 1995; Reference Tiemens, Ormel and JennerTiemens et al, 1999; Reference Van Os, Ormel and Van den BrinkVan Os et al, 1999). A two-stage sampling procedure was used. In the first stage, consecutive patients aged 18-65 years were asked to complete the 12-item version of the General Health Questionnaire (GHQ-12; Reference Goldberg and WilliamsGoldberg & Williams, 1988) while waiting to see their GP. In the second stage, a stratified random sample of the patients was invited for a baseline psychiatric interview within 2 weeks of their visit to the GP. To oversample patients with mental health problems, all patients with a high GHQ-12 score (≥5), 33% of those with a medium score (2-4) and 10% of those with a low score (0-1) were invited. These scores respectively signify high, medium and low probabilities for the presence of mental health problems.
The baseline interview consisted of the Depression and Generalised Anxiety sections of the Composite International Diagnostic Interview - Primary Health Care Version (CIDI-PHC; World Health Organization, 1990; Reference VonKorff, Üstün, Üstün and SartoriusVonKorff & Üstün, 1995). The CIDI-PHC is a standardised diagnostic interview that allows the generation of diagnoses according to ICD-10 criteria. The severity of the disorder was assessed from the number of current symptoms identified in the interview section. Patients with four or more symptoms in a section were invited for a 1-year follow-up interview with the CIDI-PHC.
The present study focuses on patients with an ICD-10 diagnosis of depressive episode or generalised anxiety disorder at baseline (World Health Organization, 1993). The ICD-10 exclusion criterion for a generalised anxiety diagnosis concerning the absence of a panic disorder, phobic anxiety disorder, obsessive-compulsive disorder or hypochondriacal disorder was omitted.
Of the 241 patients with depression at baseline, 55 (23%) were not recognised by their GP as having a mental health problem, 4 (2%) were recognised but no prognosis was given and 44 (18%) did not complete the 1-year follow-up interview. Similarly, of 119 patients with generalised anxiety at baseline, 27 (23%) were not recognised by their GP, 2 (2%) were recognised but no prognosis was given and 25 (21%) did not complete the follow-up interview. The present study therefore reports on the accuracy of the GP prognosis for 138 cases of depression and 65 cases of generalised anxiety disorder. Of these cases, 44 had both depression and generalised anxiety. They were included in the separate analyses for both diagnostic groups.
The patients studied did not differ significantly from those not recognised by their GP or those who did not complete the follow-up interview, with regard to gender (71% of the patients with depression were female and 63% of the patients with generalised anxiety were female), mean age (depression: 38.9 years; anxiety: 40.3 years), comorbidity of the mental disorders (depression: 32%; anxiety: 68%) and proportion with a previous episode (depression: 69%; anxiety: 58%). However, the non-recognised patients had milder disorders, as reflected in both the mean number of symptoms in the interview section of the CIDI (depression: 8.7 v. 11.2, P < 0.01; anxiety: 12.9 v. 14.7, P=0.08) and a duration of the present episode of 1 year or more before the index consultation (depression: 23% v. 41%, P=0.03; anxiety: 52% v. 69%, P=0.11). No differences were found on these latter two aspects between the studied patients and the patients who did not complete the follow-up interview.
General practitioner's prognosis and course
The GPs were asked to indicate their prognosis for the mental health problems they identified on the following ordinal scale: free of symptoms of the disorder within 1 month; symptom-free within half a year; improvement, but with long-lasting symptoms (1 year or more); or chronic, with hardly any improvement. The GPs were instructed to take their diagnosis and interventions into account. Because the GPs expected few patients to recover within 1 month, the first two categories of the GP prognosis were combined.
The 1-year course of depression and generalised anxiety was assessed in such a way as to fit the prognostic categories as closely as possible. Three criteria were used: absence of the baseline diagnosis at the 1-year follow-up; a reduction of 50% or more in the number of symptoms of the disorder from baseline to follow-up; and the post-baseline duration of the episode. The patient was considered ‘fully’ recovered if the first two criteria were met, partially recovered if only one was met and not recovered if neither was met. Three categories of 1-year course were distinguished; ‘full’ recovery within half a year; partial recovery or ‘full’ recovery in more than half a year; and no recovery over the follow-up period. The post-baseline duration of the disorder, defined as time to remission, was assessed in the follow-up interview. Brief periods of remission were disregarded, to keep the categories of the GP prognosis and the 1-year course comparable.
Predictive model
The predictive model for the 1-year course of depression was built upon data from 269 patients with the disorder, including the 138 cases studied here (further details available from the first author upon request). The model for generalised anxiety used data from 134 patients, including the 65 studied here. For both models the following predictors were considered: the severity of the disorder, comorbidity of depression and generalised anxiety, presence of a previous episode of the disorder, pre-baseline duration of the current episode, chronic physical illness, long-term difficulties, social support, childhood abuse, neuroticism, number of years of education, age, gender and marital status.
Analysis
Two aspects of the accuracy of the GP prognosis were studied: the agreement between GP prognosis and course, with coefficient κ (Reference NorusisNorusis, 1990); and the strength of the association (i.e. the ‘correlation’), with coefficient γ (Reference NorusisNorusis, 1990; Reference GibbonsGibbons, 1993). Clinically, the most relevant question is whether the course of the disorder can be predicted precisely. The exact agreement, however, is highly dependent on the number of categories distinguished for GP prognosis and course. In addition, a systematic bias in the GP prognosis would reduce the agreement found. The strength of the association is not materially affected by these influences.
RESULTS
Association between GP prognosis and course
The GPs proved to be somewhat pessimistic about the proportion of patients with depression who would recover within half a year (Table 1). They expected 30% of the patients to show such a favourable course, whereas 41% did (McNemar test: P=0.04). The proportion of patients expected to show a chronic course (33%) was very close to that actually observed (30%; P=0.96).
Table 1 Cross-tabulation of general practitioner's (GP) prognosis and course for depression (n=138); frequency and percentage of total
| GP prognosis | Course | |||
|---|---|---|---|---|
| Full recovery in ≤½year | Partial recovery or >½ year | No recovery | Total | |
| Symptom-free in ≤½ year | 23 (17%) | 8 (6%) | 10 (7%) | 41 (30%) |
| Improvement, but lasting symptoms | 25 (18%) | 18 (13%) | 8 (6%) | 51 (37%) |
| Chronic | 9 (7%) | 13 (9%) | 24 (17%) | 46 (33%) |
| Total | 57 (41%) | 39 (28%) | 42 (30%) | 138 (100%) |
The κ value for the agreement between prognosis and course was 0.21 (95% CI 0.09-0.33). The GP prognosis for depression therefore proved to be somewhat better than chance. The γ value for the strength of the association between prognosis and course was 0.42 (95% CI 0.21-0.62), which may be considered moderate.
The GPs were much too pessimistic about the course of generalised anxiety (Table 2). They expected only 14% of the patients to recover within half a year, whereas 31% did (P=0.04), and they expected 48% to show a chronic course, whereas 38% did (P=0.26).
Table 2 Cross-tabulation of general practitioner's (GP) prognosis and course for depression (n=65); frequency and percentage of total
| GP prognosis | Course | |||
|---|---|---|---|---|
| Full recovery in ≤½ year | Partial recovery or >½ year | No recovery | Total | |
| Symptom-free in ≤½ year | 3 (5%) | 4 (6%) | 2 (3%) | 9 (14%) |
| Improvement, but lasting symptoms | 14 (22%) | 6 (9%) | 5 (8%) | 25 (38%) |
| Chronic | 3 (5%) | 10 (15%) | 18 (28%) | 31 (48%) |
| Total | 20 (31%) | 20 (31%) | 25 (38%) | 65 (100%) |
The κ value was 0.11 (95% CI ‒0.06 to 0.27) for the agreement between GP prognosis and course of generalised anxiety. The agreement therefore was no better than chance. Nevertheless, prognosis and course were significantly related, with a γ value of 0.53 (95% CI 0.28-0.78). The marked difference between κ and γ reflects the systematic bias in the GP prognosis for generalised anxiety noted above.
Association between predictive model and course
Table 3 shows the association between the course of depression and predictions by the statistical model.
Table 3 Cross-tabulation of course of depression and predictions by a model (n=138); frequency and percentage of total
| Course predicted by model | Observed course | |||
|---|---|---|---|---|
| Full recovery in ≤½ year | Partial recovery or >½ year | No recovery | Total | |
| Full recovery in ≤½ year | 36 (26%) | 4 (3%) | 5 (4%) | 45 (33%) |
| Partial recovery or >½ year | 18 (13%) | 29 (21%) | 15 (11%) | 62 (45%) |
| No recovery | 3 (2%) | 6 (4%) | 22 (16%) | 31 (22%) |
| Total | 57 (41%) | 39 (28%) | 42 (30%) | 138 (100%) |
The κ value of agreement between predicted and observed course was 0.45 (95% CI 0.33-0.57) for the 138 patients with depression studied here. This is higher than the κ value found for the agreement between GP prognosis and course. Accordingly, the GPs may be said to perform suboptimally with respect to agreement between prognosis and course for depression.
The strength of the association between model predictions and the observed course of depression was substantial. The γ value was 0.74 (95% CI 0.60-0.88). This is higher than the γ value found for GP prognosis and course. Consequently, the GP prognosis of depression does not seem to be as closely related to the 1-year course as would be possible.
The agreement between the predicted and observed course of generalised anxiety (Table 4) was fair, with a κ value of 0.33 (95% CI 0.17-0.50). This is better than the agreement between GP prognosis and course.
Table 4 Cross-tabulation of course of generalised anxiety and predictions by a model (n=65); frequency and percentage of total
| Course predicted by model | Observed course | |||
|---|---|---|---|---|
| Full recovery in ≤½ year | Partial recovery or >½ year | No recovery | Total | |
| Full recovery in ≤½ year | 5 (8%) | 2 (3%) | 1 (2%) | 8 (12%) |
| Partial recovery or >½ year | 14 (22%) | 15 (23%) | 8 (12%) | 37 (57%) |
| No recovery | 1 (2%) | 3 (5%) | 16 (25%) | 20 (31%) |
| Total | 20 (31%) | 20 (31%) | 25 (38%) | 65 (100%) |
The γ value for the strength of association between the predicted and observed course of anxiety was as high as 0.75 (95% CI 0.54-0.96), which again is higher than the γ value found for the GP prognosis. Consequently, the association between GP prognosis and the 1-year course of generalised anxiety is not as strong as it could be.
DISCUSSION
The present study provides the first data on the performance of GPs on an important aspect of patient management for common mental disorders. Only moderate agreement between GP prognosis and course was found for depression and generalised anxiety. The prognostic performance of GPs falls significantly short of attainable performance, as assessed by the predictive power of a model based on information potentially available to the GP at the time the prognosis was made. Nevertheless, the GP prognosis was significantly associated with the course observed. Both for depression and generalised anxiety the strength of the association was fair, although stronger associations were found between the course and model predictions.
Limitations
Some methodological limitations of the study must be noted. First, the GPs were asked to give a prognosis for all mental health problems that they identified at baseline, whereas the course was assessed for specific disorders only. This may have artificially reduced the agreement between GP prognosis and course for patients with comorbid mental disorders. In the present study 44 patients had both depression and generalised anxiety, which is the most common form of comorbidity of mental disorders in primary care (Reference Sartorius, Üstün and LecrubierSartorius et al, 1996; Reference Sherbourne, Jackson and MeredithSherbourne et al, 1996). Only small changes are found when the course of both disorders is taken into account for these patients. Coefficient κ for the agreement between GP prognosis and course would change from 0.21 to 0.17 for depression and from 0.11 to 0.19 for generalised anxiety. Coefficient γ for the strength of the association would remain at 0.42 for depression and change from 0.53 to 0.55 for generalised anxiety. These changes are marginal and do not affect the conclusion that the achieved accuracy of the GP prognosis for depression and generalised anxiety is significantly less than what might be attainable.
A second methodological limitation concerns the reliability of the estimate of attainable accuracy. Ideally the predictive model should have been built on one sample of patients and tested on another. This was not possible because the available samples were too small. The reliability of the summary statistics, κ and γ, however, was examined by building a new predictive model on a random selection of two-thirds of the patients and applying it to the remaining one-third. This test was restricted to the patients with depression, because the sample of patients with generalised anxiety was too small for it. Both κ and γ showed a difference of 0.06 between the two subsamples of patients with depression. These coefficients therefore appear to be fairly reliable estimates of the maximally attainable accuracy for GP prognosis (further details available from the first author upon request).
Influence of non-recognition
The accuracy of GP prognosis can only be evaluated for those patients that are recognised by their GP as having a mental health problem. General practitioners differ markedly in their recognition of mental health problems (Reference Üstün, VonKorff, Üstün and SartoriusÜstün & VonKorff, 1995). Moreover, systematic differences exist between recognised and non-recognised cases (Reference Tiemens, Ormel and SimonTiemens et al, 1996), for example in illness severity, which may influence the accuracy of the GP prognosis. Recognition and accuracy of GP prognosis should therefore be considered as two distinct but interrelated aspects of management for common mental disorders in primary care. In the present study 77% of patients with depression or generalised anxiety were recognised by their GP, which is considerable.
Bias in GP prognosis
One explanation why the GP prognosis may be found to be suboptimal is the bias observed in it. The GPs were too pessimistic about the course to be expected, especially for generalised anxiety. Recovery within half a year was substantially underestimated by the GPs, both for depression and generalised anxiety, whereas for the latter chronicity was overestimated. This bias remains when the course is adjusted for the comorbidity of depression and generalised anxiety, as described above. The bias limits the maximum agreement possible between GP prognosis and course but it does not affect the strength of the association. Therefore, the bias may explain the marked difference in κ values for the GP prognosis and the predictive model, but it does not explain the suboptimal performance of the GPs as assessed by the γ values.
Origins of bias
One may speculate on the origins of the bias in the GP prognosis. The GPs will base their prognosis on personal experience and the literature. Most of the studies on the course of depression and generalised anxiety were conducted in psychiatric speciality settings. Studies in primary care are relatively rare and of a more recent date. These latter studies show that the mental disorders seen in primary care are usually somewhat less severe and have a more favourable course than those seen in speciality settings (Reference Sireling, Freeling and PaykelSireling et al, 1985; Reference Ormel, Oldehinkel and BirlmanOrmel et al, 1993; Reference Cooper-Patrick, Crum and FordCooper-Patrick et al, 1994; Reference Katon, Lin and VonKorffKaton et al, 1994; Reference Ronalds, Creed and StoneRonalds et al, 1997). It may therefore be speculated that the pessimistic expectations of the GPs of the course of depression and generalised anxiety are based on studies done in psychiatric speciality settings.
General practitioners' use of predictors
Apart from the systematic bias in the GP prognosis, the suboptimal performance of GPs may be explained by a poor use of predictors. General practitioners may be poorly informed about predictors of the course of common mental disorders in primary care. Alternatively, their assessment of the predictors may be lacking, or they may not weight the different predictors optimally.
Options for improvement
The ultimate goal of studying the accuracy of the GP prognosis for depression and generalised anxiety is to improve this aspect of patient management. One way to achieve this is to provide feedback to the GPs on the agreement between prognosis and course. A more effective way, however, is to provide additional feedback on the strategy that GPs use to arrive at their prognosis, and to compare this with the optimal strategy (Reference Hammond, Stewart, Brehmer, Kaplan and SchwartzHammond et al, 1975). For the GPs of the present study we examined what cues they used for their prognosis and how they weighted these cues. We compared this with the optimal strategy, as assessed by the factors and factor weights in the predictive model. The results of this investigation will be reported in a separate paper.
Clinical Implications and Limitations
CLINICAL IMPLICATIONS
-
▪ The general practitioner's (GP) prognosis for common mental disorders may be too inaccurate to fulfil its important functions in patient management.
-
▪ There is scope for improving the accuracy of the GP prognosis; the achieved accuracy falls short of the attainable accuracy.
-
▪ General practitioners are too pessimistic about the 1-year course of depression and generalised anxiety.
LIMITATIONS
-
▪ The course of the depression or generalised anxiety beyond the period of 1 year was not taken into account.
-
▪ The categories of the GP prognosis and course were rather broad, taking together many relevant variations in course.
-
▪ The GPs were not asked to what extent they expected their treatment to contribute to the prognosis.
ACKNOWLEDGEMENTS
This study was supported by grants from the Council for Medical and Health Research of The Netherlands Organisation for Scientific Research and the National Fund for Public Mental Health.
eLetters
No eLetters have been published for this article.