What constitutes success?

“Collectively (and perhaps rather belatedly) we have recognised the most important issue facing the health service is not how it should be organised or financed, but whether the care it offers actually works.” (Walshe & Ham, 1997)

In evaluating what works — the effectiveness of interventions — key questions must be asked about the criteria of success. Choices have to be made concerning whose views of effectiveness are paramount and who has the right to define what service users need.

This is not about traditional rivalries between, for example, psychiatrists and psychologists, but about differences of perspective among an increased range of stakeholders. The list of potentially interested stakeholders is large: from neighbours and shopkeepers, politicians and pressure groups, through the myriad of clinicians and providers of treatment and support, to the individual who experiences mental health problems. Each group differs not only in its definition of an ‘effective’ intervention or service but also in terms of its role, influence and ultimately its power.

SYMPTOM REDUCTION: AN ADEQUATE INDEX OF SUCCESS?

The National Service Framework for Mental Health (Department of Health, 1999) has emphasised that services should be led by the interests of consumers, and that the goals of intervention are multidimensional. Yet the vast majority of research in psychiatry and psychology defines effectiveness in terms of symptom or problem reduction. It seems often to be assumed that if this can be achieved, everything else will fall into place: symptom reduction is seen as both a necessary and a sufficient condition for an improvement in the quality of the life of someone experiencing mental health problems.

This is unlikely to be the case. Mirin & Namerow (1991: p. 1007) have, for example, argued that symptom reduction is not a sufficient condition for an improved quality of life:

“the assumption that symptom relief, reduction in the frequency of episodes of illness and improvement in functional adaptation… mean that quality of life has been enhanced may at times be unwarranted. Indeed, for some patients, these seemingly positive changes may not be accompanied by the development of meaningful interpersonal relationships, by employment that they are enthusiastic about, or by a subjective sense of satisfaction and wellbeing… Emphasis should be placed not only on level of clinical symptomatology or pathologic behaviour, but also on the functional integration of the patient into his or her occupational, social and cultural milieu.”

There is also no necessary connection between reduction in severity of symptoms and service users' sense of control or engagement in activities. For example, research has demonstrated no relationships between diagnosis or symptom severity and employment (Anthony et al, 1995) and Strauss (1994) has described how, with effective support, those who do not demonstrate marked improvement in symptoms may still report an increased sense of control or increased understanding of their problems, which can reduce hopelessness and distress.

DIFFERENT STAKEHOLDERS: DIVERGENT PERSPECTIVES

Different people involved in the mental health enterprise have different views about the desirability of different outcomes (Bond, 1994). Although clinicians typically afford pride of place to symptom reduction in their research and clinical endeavours, politicians are more concerned with rates of murder and violent crime and the alleged danger and disruption caused by people with mental health problems to their local communities. The primary interest of families is in receiving information, having their role as primary caregivers recognised and receiving the support and services that they need to sustain them (Shepherd et al, 1995). Those with mental health problems themselves are concerned with issues of choice and control (Read & Reynolds, 1996). Although they desire freedom from debilitating symptoms, they typically place at least as much emphasis on the importance of decent lives:

“safe, pleasant and affordable housing, well paying and fulfilling jobs, friends… to be treated with dignity and respect, to have control over their lives and to have genuine choices. They want to feel good about themselves and to have the opportunity to achieve the things that all of us do” (Bond, 1994: p. 490).

Numerous studies have shown the divergence between the goals of clinicians and those whom they serve. Dimsdale et al (1979) showed that staff in a Massachusetts psychiatric unit viewed ‘insight’ on the part of patients as the primary goal of intervention, whereas users themselves placed ‘insight’ at the bottom of their lists. Mitchell et al (1983) found an extremely low level of agreement between problems listed by psychiatric patients and their physicians, and Lynch & Kruzich (1986) found large differences in user and professional perceptions of barriers to using mental health services. Although users identified non-availability of services and lack of transport as the key difficulties, professionals interpreted the problems in psychological terms and instead blamed ‘client resistance’. More recently, Shepherd et al (1995), although finding some measure of agreement between service users and professionals, also identified different priorities in the two groups. Users valued assistance with housing, finance, social networks, physical health and coming to terms with their problems, whereas professionals placed greater emphasis on the availability of professional support, treatment and monitoring.

If outcome measures are designed by clinicians, then they will reflect clinicians' agendas and concerns. This is apparent in the numerous instruments for measuring symptomatology and social functioning and the relative absence of tools for measuring the choice, control and personal fulfilment valued by service users: even the various ‘patient satisfaction’ scales available have been devised by clinicians (Ruggeri, 1994). However, alternative approaches are beginning to emerge.

Many services claim to want to ‘empower’ service users; therefore Rogers et al (1997) have developed a consumer-constructed scale to measure empowerment. Starting from the dimensions of ‘empowerment’ defined by service users/survivors, they developed a scale to measure those that demonstrated good psychometric properties. The scale comprises five factors:

1. (a) self-esteem and self-efficacy;

2. (b) power and powerlessness;

3. (c) community activism and autonomy;

4. (d) optimism and control over the future;

5. (e) righteous anger.

It is interesting to speculate whether those psychiatric and psychological interventions that have been demonstrated to be effective in symptom reduction would look so effective when evaluated against such empowerment criteria.

SERVICE USER INVOLVEMENT: RHETORIC OR REALITY?

Finally, in any consideration of effectiveness, the degree to which an intervention is acceptable to service users cannot be ignored. Too often, research considers only those who accepted the intervention and yet there are many interventions where drop-out rates are extremely high. In considering the effectiveness of any intervention, attention must be paid to accurate measurement of drop-out and refusal rates: an intervention must be deemed ineffective if people find it so unacceptable that they refuse to take it.

In former times, service users were deprived of a voice by simple exclusion. They were absent from all service planning meetings and only invited to ward rounds to demonstrate their symptoms and hear the doctor's advice. The efforts of an evergrowing user/survivor movement have done much to increase the involvement of service users in both service planning and decisions about individual treatment and support.

However, it is also the case that service users remain a relatively powerless and devalued group, often deprived of any real chance to contribute to decisions about the treatment and services that centrally affect their lives. The physical presence of users is rarely accompanied by effective collaboration on these most fundamental issues, but instead there are a variety of ways in which their views are disregarded (Perkins, 1996).

Most notable among these is the assumption that the ability of people with mental health problems to define their own wants, wishes, problems and needs is diminished. This means that their views and ideas are cast into doubt, and those of other (supposedly more reliable) stake-holders tend to take precedence. Until this situation changes, the growing number of measures of users' wants, wishes and expectations will do little to ensure that those who are on the receiving end of services have a real voice.

If ‘user involvement’ is to be more than rhetoric, surely it is time both systematically to ascertain users' goals and to accord them the status currently enjoyed by the priorities of clinicians?