Validation and development of a self-report outcome measure (MAP-sc) in opiate addiction

Abstract

Aims and Method

To develop and assess the viability of a self-completion version of the Maudsley Addiction Profile for assessing and monitoring the functioning of opioid-dependent patients. A total of 206 treatment-seeking opioid-dependent patients completed the Maudsley Addiction Profile interview and a self-completion version at a single clinic appointment at a substance misuse facility. Scores from both formats were compared using correlation coefficients.

Results

Non-parametric correlation coefficients between interview and self-completion version for alcohol, drug, psychiatric, family and legal problems correlated in excess of 0.7 for the majority of the 20 items that were compared.

Clinical Implications

A short, self-administered questionnaire version of the Maudsley Addiction Profile is a feasible alternative to the interview for assessing and monitoring treatment of opioid-dependent patients. The questionnaires were usually completed by clients within 15 min. These would be particularly useful in services with very limited staffing time, such as primary care.

The policy of the current UK Government towards NHS services includes the statement that:

‘There is no place in the modern NHS for the piecemeal adoption of unproven therapies, or for hanging onto outdated, ineffective, treatments’ (Department of Health, 1998).

This heralded a new era in which individual health service providers would be expected to monitor and demonstrate their effectiveness. The demonstration of illicit drug use by urine or oral fluid analysis is regarded by many experts as inadequate to determine the effectiveness of the many aspects of care provided by addiction services (McLellan et al, 1985). Although outcome measures such as the Health of the Nation Outcome Scales (HoNOS; Wing et al, 1998) are available for many mental health sub-specialties, there are relatively few brief instruments that addictions services can routinely use to assess patient functioning and outcomes. Such an instrument would be particularly useful given the UK Government's current enthusiasm for expanding the prescribing of injectables, such as diamorphine to intravenous drug users (National Treatment Agency, 2002). There is also increasing pressure to provide rapid access prescribing particularly to patients from the criminal justice system (Best et al, 2002). We therefore have compared a self-completion version of the Maudsley Addiction Profile (MAP) with a short interview to demonstrate that the self-completion version can be used in the routine assessment of those attending substance misuse services.

The standard MAP (Marsden et al, 1998) is a well-validated, brief semi-structured interview developed to assess the substance use, risk behaviour, health and social functioning of illicit drug users. It was initially developed as a tool to measure the outcome of substance misuse treatment as a result of the recommendation of the UK Department of Health Task Force on Services for Drug Misusers. Field-testing has shown the instrument to be reliable and valid in people with substance misuse problems.

The MAP requires a health professional to complete the various sections. This takes approximately 12 min. There is no requirement for formal training to use the MAP. Nevertheless, many service providers, particularly those in primary care where doctors or nurses have appointments of less than 10 min, find this time commitment onerous, particularly as part of follow-up appointments.

Many surveys have concluded that self-reporting of drug use is reliable and valid when events are recent and patients do not face negative consequences (McLellan et al, 1985; Mieczkowski, 1990; Poole et al, 1996; Lundy et al, 1997; Darke, 1998). Studies comparing written self-completion questionnaires with interview self-reports of substance use have found similar responses to both procedures or a tendency to report higher levels of drug use in the self-completion questionnaires (Sobell & Sobell, 1981; Aquilino, 1994; Heithoff & Wiseman, 1996; Rosen et al, 2000). A self-completion version of the MAP for opiate-dependent people would be valuable in order to identify areas of need and to monitor treatment outcome. This would be particularly useful in services with very limited staffing time, such as primary care. It would also be useful for service commissioners, such as the UK Drug Action Teams, in order to measure the effectiveness of treatment. Hence, the aim of the study was to devise and validate a self-completion version of the MAP (the MAP-sc).

Method

Patients

Out-patients with a significant current opiate dependence were approached directly by the researchers when they attended one of four substance misuse facilities (Marina House, part of the Maudsley Hospital NHS Trust; and Southend, Basildon and Thurrock Community Drug and Alcohol Services). The majority of patients who were approached were attending these facilities during the various research sessions of the interviewers. Patients were excluded if they were unable to give informed consent or were unable to complete self-completion questionnaires (e.g. through illiteracy). All patients were currently undergoing treatment and satisfied DSM-IV (American Psychiatric Association, 1994) research diagnostic criteria for opioid dependence. Opioid use was also confirmed by the results of recent urine drug tests that formed part of their treatment and assessment programme. There were no cases in which the results of the urine drug screen were completely at variance with self-reported drug use. However, the self-reported drug use was based over the preceding month, rather than the preceding 48 h. Drug screening was usually performed every 2 weeks. In most cases, urinary drug screening indicates drug use up to 72 h following illicit drug use. Hence, the patients could quite possibly have used opiates or cocaine on only two occasions each week yet still test positive for opiates at every urine drug screen.

Procedure

The self-completion questionnaire was designed using the instructions available for the interview version. A pilot phase resulted in minor modifications to some instructions. The MAP is a public domain research instrument and can be used free of charge for non-profit applications (Marsden et al, 1998). The self-completion version of the MAP is available from the principal author (J.L.). It has also been submitted for access from the Senior Clinicians in Addictions (SCAN) website (http://www.scan.uk.net).

Clients were asked to complete the MAP-sc and two other brief questionnaires: the Minnesota Student Survey questionnaire for substance misuse (Fulkerson et al, 1999) and the Severity of Dependence Scale (SDS; Gossop et al, 1995). The semi-structured MAP interview was completed by the researcher. Approximately half the patients completed the self-completion version prior to the interview. Demographic information (age, gender, employment status, ethnic group), clinical information (age at first opiate use, age at dependence) was also recorded. The interview and completion of the questionnaires took approximately 30 min. Equal numbers of interviews were performed by three researchers (J.L., V.P. and D.G.). Between 30 and 80 patients were recruited at each of the four sites.

Patients gave written informed consent. Approval to undertake the study was obtained from the South London and Maudsley NHS Trust and the South Essex local research ethics committees. Patients received a nominal payment in gift vouchers for taking part.

Scoring and data collection

The scoring procedure was identical for the self-completion and interview versions of the MAP. All clinical data for the MAP were taken with reference to the previous 30 days immediately prior to the interview. For the self-completion version of the MAP, patients were asked to estimate the total number of days on which each of six substance types were used (alcohol, heroin, crack or cocaine, illicit methadone, illicit benzodiazepines and amphetamine including ecstasy). Patients were asked to estimate the quantity of alcohol or the street value of heroin, crack or cocaine they had used in a typical day on which they had used any of these drugs. They were also asked to estimate the number of days on which they injected drugs, the number of injecting episodes on these days, number of people with whom they had shared injecting equipment, number of sexual partners with whom they had unsafe sex and number of occasions of unsafe sex. The MAP contains two 10-item symptom scales for physical and psychological health (MAP section D). A 5-point Likert-type scale is used to respond to these items using the following expressions: never, rarely, sometimes, often or always (scored 0-4). The scales are scored by summing the responses to each of the 10 items to give a total score out of a maximum of 40 for each scale. Relationship problems were assessed by recording the number of days on which the patient had contact with their sexual partner, relative(s) and friends and the number of days on which there had been serious conflict (such as a major argument, verbal abuse and/or violence but not routine differences of opinion). The proportion of contact days involving serious conflict was then calculated. Three categories of criminal behaviour were recorded: sale of illegal drugs, shop-lifting and ‘ other crimes’ (such as theft from a property, theft from a person, theft from or of a vehicle, and fraud or forgery). The number of days on which each category of crime was committed and the number of occasions each offence was committed on a typical day was recorded. Consequently, the total number of crime days per month could potentially exceed 30.

Statistical analysis

Two-tailed Spearman correlation coefficients were used to compare the composite MAP scores between the interview and self-completion versions of the MAP for physical and psychological health, relationship problems, illegal activities (MAP item E1-E6 and E10; section E) and answers to questions with scaled answers (such as number of days and extent of substance misuse in the previous month).

Results

No patients met the exclusion criteria, although five agreed to take part in the project but did not complete both the interview and self-completion questionnaire.

Results are presented for 206 opioid-dependent patients who completed the study. The background socio-demographic and clinical characteristics of these patients are shown in Tables 1 and 2. The patients were primarily White, unemployed males with a mean age of 32 years. The mean age at first heroin use in this report was 21.3 (s.e.=0.7) years and they reported using opiates regularly for 14.2 (s.e.=0.9) years. The patients had a relatively high severity of opiate dependence (mean SDS score of 11.2) and tended to use heroin on most days.

Table 1. Background characteristics of 206 opioid-dependent patients who completed the study

Characteristic
Male gender, %	63
White European ethnicity, %	77
Unemployed, %	83
Age, years: mean (s.e.)	32 (0.6)
Severity of Dependence Scale score, mean (s.e.)	11.2 (0.3)
Age at first heroin use, years: mean (s.e.)	21.2 (0.7)

Table 2. Clinical characteristics of 206 opioid-dependent patients who completed the study

Behaviour1	n (%)
Any alcohol use	132 (64)
Illicit methadone	93 (45)
Cocaine or ‘crack’ cocaine	130 (63)
Illicit benzodiazepines	61 (30)
Amphetamines or ‘ecstasy’	23 (11)
Any injecting drug use	111 (54)
Any sharing of injecting equipment	16 (8)
Any episodes of unsafe sex	52 (25)

Injecting drug use and use of other illicit drugs and alcohol were common, with more than half the patients reporting some use of cocaine in the previous month. A significant proportion (46%) had also used illicit methadone in the month prior to interview. Similarly, approximately one-quarter of the patients had used benzodiazepines in the previous month. By contrast, Table 3 indicates that, although many used illicit drugs besides heroin, the reported use of illicit methadone, benzodiazepines and alcohol was infrequent (typically fewer than 8 days each month). Cocaine and injecting drug use were more common but were still restricted, on average, to 8-10 days per month.

Table 3. Correlation betwen Maudsley Addiction Profile (MAP) interview and self-completion questionnaires for 206 opiate-dependent patients¹

Item	Self-completion MAP Mean (s.e.)	Interview MAP Mean (s.e.)	Spearman correlation coefficient
Alcohol, days	8 (1)	7 (1)	0.872**
Heroin, days	23 (1)	22 (1)	0.742**
Daily spending on heroin2, £	33 (1)	24 (2)	0.623**
Cocaine or crack, days	10 (1)	10 (1)	0.876**
Daily spending on cocaine or crack2, £	24 (3)	22 (3)	0.702**
Illicit methadone, days	3 (1)	3 (1)	0.757**
Illicit benzodiazepines, days	3 (1)	3 (1)	0.653**
Amphetamines3, days	1 (0.5)	1 (0.6)	0.584**
i.v. drug use, days	10 (1)	9 (1)	0.830**
Injections per day	2 (0.3)	2 (0.3)	0.682**
Occasions on which any equipment was shared, n	0.4 (0.2)4	0.2 (0.09)	0.046
Partners (unsafe sex), n	0.3 (0.09)5	0.3 (0.05)	0.514**
Episodes unsafe sex, n	2 (0.6)	2 (0.5)	0.709**
Composite physical health score	19.2 (0.7)	18.1 (0.8)	0.978**
Composite mental health score	18.1 (1.2)	17.3 (1.3)	0.839**
Social contact6, days	32 (0.1)	34 (0.2)	0.758**
Social conflict6, days	6 (1)	6 (1)	0.736**
Proportion conflict days7, %	18 (2)	14 (2)	0.556**
Offences, n	19 (5)	22 (5)	0.755**
Crime days, n	10 (2)	15 (5)	0.394**

Table 3 shows the results and correlation coefficients between the two versions of the MAP. There was no difference in any of the correlations, irrespective of whether the interview or self-completion questionnaire was performed first. The comparison between the interview and self-completion versions of the MAP showed good reliability in most areas, with correlation coefficients in excess of 0.7 for the majority of responses. Lower scores were obtained for use of illicit amphetamines, benzodiazepines, sharing of equipment, reports of interpersonal conflict and offending days.

Discussion

Reliability and validity of responses

Results from this study indicate that a self-completion version of the Maudsley Addiction Profile (MAP-sc) elicited information on substance use-related behaviours which was similar to that obtained from the interview version. Good reliability was observed in most areas, with correlation coefficients in excess of 0.7 for the majority of responses. The correlation between the instruments for the composite physical and mental health scores and the number of days of heroin, cocaine and injecting drug use in the previous month was in excess of 0.7 for each measure.

Urine drug tests results were available to confirm the recent use of illicit drugs. However, both the interview and self-completion versions of the MAP rely primarily on self-report, including self-report of illicit drug use and crime over the previous month. Many surveys have concluded that self-reporting of drug use is reliable and valid when events are recent and patients do not face negative consequences (McLellan et al, 1985; Mieczkowski, 1990; Poole et al, 1996; Lundy et al, 1997; Darke, 1998). For example, the UK National Treatment Outcome Research Study reported that the concordance rate between self-reported use and results from a urine specimen for heroin and cocaine was 92%. Only 2-3% of clients tested positive who reported not using heroin or cocaine (Gossop et al, 1998). Hser et al (1992) assessed the reliability of retrospective self-report by 323 narcotic addicts at a 10-year interval with respect to opiate use and criminality. The diagnostic accuracy and predictive power of the self-report was found to be very high (over 80%). In regard to self-report of cocaine use, Simpson et al (1999) reported that 8.8% of people denied cocaine use but subsequently tested positive. This was based on a random sample of 352 interviewees who presented with cocaine misuse (n=1605) as part of the US DATOS (Drug Abuse Treatment Outcome Study).

Of the 206 patients in our study, 96 were not working and denied committing any acquisitive crimes in the previous month. These individuals spent an average of £595 (s.e.=£96) on illicit heroin and cocaine over this period. It is difficult to see how they were able to fund this illicit drug use without recourse to criminal activity, particularly as their total income from benefits amounted to only £230 per month each, and this includes payments to cover food and accommodation. Even with enhancements for disability or child support this amount remains less than £500 per month (Social Security Office, 2002). It is possible that these patients were receiving an undeclared income from informal work as well as fraudulently claiming unemployment benefits. However, it seems unlikely that they were willing to reveal the amount they were spending on illicit drugs and yet withhold information on undeclared employment.

The apparent inconsistency between illicit spending and income may have been reflected in the poor correlation (0.392) between the number of crime days reported in the self-completion version of the MAP compared with the interview version. Patients may have been more likely to admit to offending during an interview. By contrast, there was good correlation between actual number of offences over the previous month with both instruments (0.755). Although the validity of self-reported criminal activity is highly circumspect, it is difficult to see how such information could be collected by any other means. There are grounds for regarding criminal data based on self-report as useful and informative (Gossop et al, 2003). For example, the use of official arrest records would provide a serious underestimate of actual criminal activity. Hence, self-reported criminal activity provides the best feasible measure of criminal behaviour.

Low correlations were obtained for reports of sharing of equipment and experience of unsafe sex. However, these activities were uncommon, with only two patients reporting sharing equipment on more than two occasions and two having unsafe sex with more than two partners in the previous month. Only 16 patients reported any sharing of equipment. The infrequency of these events may partly explain their low reliability.

Socio-demographic and clinical comparisons

Table 1 shows that the heroin-dependent patients in this study were highly comparable to other clients notified in 2000–2001 to the English regional drug misuse databases in terms of gender, although the patients are approximately 7 years older (HM Government Statistical Office, 2000). This difference in age may be partly owing to the inclusion of other substance users besides heroin-dependent individuals in government statistics.

Other research on clinical and non-clinical populations of opiate users produces comparable data in terms of gender distribution, age at first heroin use and duration of addiction. For example, Rounsaville & Kleber (1985) recruited a sample of 105 untreated opiate-dependent patients in North America. The age at first heroin use was 18 years and the participants had used opiates regularly for 6.4 years. Gossop et al (1995) reported results from a community sample of 408 heroin users in London with a mean age of 28 years. Of this sample, 62% were male, 50% were in treatment and the mean age at first heroin use was 19 years.

Reports of current substance use were confirmed by urine drug testing. The severity of dependence among the heroin addicts was comparable to, and often more severe than in, other published samples. The mean SDS score for patients in this study was 11.2 (s.e.=0.3). By contrast, Gossop et al (1995) reported a mean SDS score of 8.7 (s.e.=0.2) in a community sample of 408 heroin users. Also, the sample of 105 heroin-dependent patients in our report had a mean SDS score of 10.1, indicating high levels of dependence. This is consistent with the fact that the reported sample was made of heroin addicts rather than heroin users, many of whom may not have been dependent. Gossop et al (1995) also found the SDS scores of their community sample were higher than those of four other sizeable groups of substance misusers, including a group of 222 heroin users in Sydney who were receiving methadone maintenance. These data indicate that the sample of heroin addicts reported here had significant levels of dependence.

Comparison with other instruments

The Christo Inventory for Substance-misuse Services (Christo et al, 2000) is a brief assessment tool that can be routinely used to monitor patient functioning and outcomes. This is a single page evaluation tool that produces a single score. There are ten domains, including general health, risk behaviour, drug use, employment and criminal involvement, each of which scores 0, 1 or 2. Unfortunately, this instrument is necessarily rather coarse and elicits very limited information. For example, drug and alcohol use is effectively divided into abstaining, nonproblematic use and binge/regular use. Although the Christo Inventory was validated in a study with 345 patients with stable drug and alcohol use, it was validated retrospectively in comparison with a structured interview with a single rater on only 90 of these individuals. This interview took place 3 years earlier. Moreover, the scoring relies entirely on the view of a worker within a substance misuse service rather than that of the clients themselves. The MAP-sc generates significantly more information than the Christo Inventory. Furthermore, the MAP-sc is completed by the users themselves. It has been validated against contemporaneous interviews with over 200 clients, involving four separate sites and three interviewers.

The Addiction Severity Index-self-completion questionnaire (ASC-sc; Rosen et al, 2000) was derived from the interview version of the ASI, a widely used research tool for assessing substance misuse-related problems (McLellan et al, 1992). Composite scores for alcohol, drug, psychiatric, family and legal problems produced correlation coefficients between 0.59 and 0.87 for the self-completion and interview versions.

The correlations between self-completion and interview versions of the MAP are at least as good as those for the ASI and the correlations are better in the majority of domains. The MAP-sc has better correlation with the interview version for medical composite scores than the ASC-sc (0.80 and 0.47 respectively). Unlike the ASI, individuals did not endorse more drug use and psychiatric problems on the self-completion version of the MAP than with the interview. There was no significant difference between the results for both instruments other than with regard to criminal activity. These observations may be partly a result of the delay (2 weeks) between performing the interview and self-completion versions of the ASI. By contrast the MAP interview and self-completion questionnaires were completed within 1 h of each other.

Both the MAP-sc and the ASI-sc contain a similar number of questions (36 and 52 respectively) and can be completed in approximately the same time (10-12 min). The MAP-sc is considerably easier to score than the ASI. For example, there is no recourse to use of logarithms in the MAP-sc. Furthermore, only one composite score (relationship conflict) requires a division to be calculated in the MAP-sc. By contrast, all the composite scores in the ASC-sc require long-division calculations and cannot be performed without a calculator. Scoring also requires a detailed manual. There is also greater detail provided by the MAP-sc in reporting the frequency and quantity of illicit drugs in use. By contrast, the published version of the ASC-sc does not attempt to quantify the frequency or quantity of illicit drug use over the previous month, although it could easily be modified to provide this information.

Strengths and limitations

The strength of correlation between the MAP-sc and the interview version is constrained by the test-retest reliability of the interview version. Although the MAP interview has high reliability under optimal research conditions (0.88-0.94 after 3 days; Marsden et al, 1998), its reliability in other situations is not known. The research reported here involves a population of individuals seen at a teaching hospital and by three busy provincial community drug teams. They are therefore more typical of clients of community services and as such the results are likely to generalise to populations of typical treatment-seeking patients. Other limitations that also apply to the interview version of the MAP are applicable to the MAP-sc. These include problems with reliability of self-reported illicit drug use and criminal activity and inaccurate recall of events, such as social conflict and frequency of drug use, particularly at more distant time points.

The MAP-sc was administered before the interview version in half the patients. This was done to reduce the effect that might arise as a result of improved recall if one version of the instrument had been consistently administered before the other.

Potential disadvantages of self-completion instruments include possible patient refusal or incomplete responses. This was not a significant problem during this trial with only five clients failing to complete both versions of the MAP. None of the patients who were approached refused to take part and none met any exclusion criteria, such as illiteracy. All patients were treatment-seeking opiate-dependent clients. Fewer than half were recruited at a teaching hospital. Furthermore, good correlations were obtained from three different interviewers. Hence, the results are likely to generalise well to other treatment populations in the UK. However, participants received a nominal payment in gift vouchers for taking part. Patient refusal or incomplete responses may be a problem in other situations. However, clinical interviews are not immune to such problems.

Future work could include the comparison of the self-completion MAP with other self-completion questionnaires, such as the ASI or the Christo Inventory for substance misuse services. It might also be possible to compare the outcomes of repeated scoring with hair-testing, a technique that is being adopted at some of the sites involved in this study.

Although hair-testing cannot give specific information on the absolute quantity of illicit drugs that have been consumed, it gives an accurate picture of changes in drug use over several weeks. This could be compared with self-reported changes in drug use and changes in other domains on the MAP-sc. This would further validate the accuracy of the self-report instrument.

In conclusion, the present study and previous research (Rosen et al, 2000) indicate that a self-administered questionnaire version of the Maudsley Addiction Profile (MAP-sc) is a feasible alternative to the interview. This would be particularly useful to assess and monitor treatment outcomes in services with very limited staffing time, such as primary care.

Declaration of interest

None.