Experience with an external quality assurance scheme for antimicrobial susceptibility testing of Neisseria gonorrhoeae in India, 2001–2007

Summary

Antimicrobial resistance (AMR) in Neisseria gonorrhoeae compromises patient treatment and disease control. Epidemiologically based surveillance of AMR in gonococci is needed to optimize standard treatment regimens. Validation of AMR surveillance data depends on external quality assurance schemes (EQAS). AMR surveillance data quality in India during 2001–2007 was assessed by participants testing panels of reference strains and repeated re-challenge with identical controls, accompanied by educative feedback. Overall, correct results were obtained for 944 (82%) of 1030 tests performed for five ‘core’ antibiotics. Aggregated error rates decreased from 33% (123 tests) in 2001 to 4·4% (180 tests) in 2007 with improvements in individual laboratory performance. Cephalosporin test results produced high error rates without improvement. Reference centre and network laboratory collaboration produced marked improvements in test performance through annual EQAS integrating proficiency testing and participant education. More frequent EQAS cycles would assist this process. These experiences may be applicable in similar settings elsewhere.

INTRODUCTION

Gonorrhoea remains an important public health issue because of the sheer number of new cases that are acquired annually [old, but the most recent, World Health Organization (WHO) disease estimates suggest over 60 million new cases occur globally each year] and the morbidity that accompanies gonococcal disease [1]. This morbidity includes male and female infertility, the latter case involving first trimester abortion and pelvic inflammatory disease, and severe eye infections that may lead to blindness in the newborn. Also significant is the enhanced co-transmission of the human immunodeficiency virus (HIV) that occurs in the presence of gonococcal infection. The economic burden of gonorrhoea is also high. It is estimated that these complications are responsible for the loss of more than 250 000 disability-adjusted life years (DALY) each year [2].

High rates of gonorrhoea persist in India and many less-developed countries where the management of sexually transmitted disease is based on syndromic principles with the antimicrobials for treatment being provided to individuals on the basis of their presenting symptoms. Almost uniquely among the bacterial pathogens associated with sexually transmitted diseases, Neisseria gonorrhoeae has a highly developed capacity to develop resistance to antibiotics used for the treatment and control of gonococcal disease [1]. Antimicrobial resistance (AMR) in gonococci has appeared progressively and seemingly inexorably so that emergence of resistance to the penicillins, tetracyclines and macrolides has seen the widespread removal of these cheap, oral agents from the standard treatment regimens used for syndromic management. Levels of resistance to fluoroquinolone antibiotics documented in India [3] and nearby countries [4] also compromised the efficacy of these antibiotics at both an individual and population-health level and these findings have in turn required their replacement with extended-spectrum cephalosporin antibiotics as the recommended treatment for gonorrhoea in these jurisdictions.

Effective antibiotic treatment is an essential component of gonococcal disease control so the impact of AMR in N. gonorrhoeae on the outcome of treatment and disease control is a major concern of long standing [1]. The WHO has a long-established regionally based programme of AMR surveillance for N. gonorrhoeae – the Gonococcal Antimicrobial Surveillance Programme (GASP) [5], designed to enable optimization of the components of standardized treatment regimens on the basis of epidemiologically based surveys of the distribution and extent of gonococcal AMR [1, 6]. For public health purposes, AMR at a rate of ⩾5% in gonococci sampled in a general population is the ‘threshold for action’ for removal of an antibiotic from treatment schedules and for substitution of another, effective, agent [1, 6].

GASP programmes therefore seek to determine, in epidemiological surveys, the proportion of gonococcal strains obtained from defined patient populations that are resistant to antibiotics relevant for the treatment of gonorrhoea and relate these findings to current treatment schedules [1, 6]. These strategies require that the AMR data generated be of high quality and verifiable. The special requirements for in-vitro growth and AMR testing of the fastidious N. gonorrhoeae significantly complicate this requirement. The WHO has recently reviewed the standards required for surveillance of antimicrobial resistance in N. gonorrhoeae [6]. Part of the process of ensuring the requisite data quality and validation is the distribution, by the WHO Collaborating Centre for Sexually Transmitted Diseases, of reference panels of gonococci for use in internal quality control, and external quality assurance schemes (EQAS) [7]. Participation in EQAS programmes and appropriate use of the WHO reference panel by GASP participants is a necessary requirement for the validation of gonococcal AMR data [6]. However, these EQAS programmes have to date been used in multi-jurisdictional programmes conducted directly by the collaborating centre, and not as country-based programmes run locally in developing countries.

The New Delhi-based WHO South East Asian Regional (SEAR) GASP reference laboratory has conducted a country-based GASP programme in India since 2000. We describe the first seven annual proficiency testing challenges (EQAS) in the Indian GASP where we evaluated the quality of the AMR testing data, assessed the network capability to detect newly emerging AMR and determined the effect on laboratory performance of repeated re-challenge with identical controls accompanied by liaison and feedback on areas for improvement in test procedures. Because of increasing numbers of reports of treatment failures with orally administered cephalosporins [8, 9], calls have been made for enhanced global surveillance of all forms of gonococcal AMR in order to optimize gonococcal antibiotic treatment [10]. The experience gained from this country-level field study of EQAS use and development may be applicable to GASP programmes in other less-developed settings at a time when the WHO GASP is undergoing expansion to meet the ongoing challenges of surveillance and control of gonococcal AMR.

METHODS

Enrolment of participants in the Indian GASP followed a workshop and consultation organized by the Indian coordinating laboratory in June 2000. ‘Hands-on training’ was provided in use and interpretation of the standardized disc-diffusion AMR test method [11] that was adopted for the programme. Formal minimal inhibitory concentration (MIC) determination is also performed at the Indian coordinating laboratory and some other centres; however, because of the cost and complexity of this test system, determination of phenotypic resistance by disc-diffusion methods is the only feasible system suitable for widespread use in the Indian GASP network.

A panel of six anonymized, lyophilized EQAS strains of N. gonorrhoeae was sent annually by the WHO Collaborating Centre for STD, Sydney, Australia to the Indian coordinating laboratory in New Delhi. These were then distributed to active participants in the Indian GASP, five laboratories over the four years 2001–2004 and six over the three years 2005–2007. Participants were provided with EQAS strains in 20% glycerol-nutrient broth and on chocolate agar slopes/plates, antibiotic and nitrocefin discs (for β-lactamase testing), and growth media and supplements as required. The EQAS panel used in the Indian GASP comprised current WHO reference strains and gonococci with novel resistance patterns that examined resistance patterns in N. gonorrhoeae to the penicillins (both chromosomal and plasmid-mediated resistance), quinolones (tested by a ciprofloxacin/nalidixic acid disc combination), high level, plasmid-mediated tetracycline resistance (TRNG) and spectinomycin. Repeat challenges were undertaken by the inclusion of identical strains in a number of panels: two EQAS strains were repeated four times, one strain twice and four strains once.

Data were recorded in a standard format and laboratories provided both quantitative results (measurements of zones of growth inhibition) and their qualitative interpretations of these zone sizes as appropriate (i.e. ‘S’, sensitive; ‘LS’, less sensitive; ‘R’, resistant) for each isolate and antimicrobial agent tested. Confidentiality was obtained by the allocation of a code number, known only to the individual participating laboratory and the Indian coordinating laboratory.

Results were compared to those generated at the Collaborating Centre, Australia and according to method-specific interpretive criteria [11]. A result recorded as S, LS or R was regarded as correct if it was the same as the designated result, and as an error if different. Errors were interpreted as major when S strains were reported as R, and R strains as S, and as minor when S strains were reported as LS, LS strains as S or R, and R strains as LS. An incorrect categorization (i.e. an interpretive error by the laboratory), was adjusted to the correct designation by the coordinating laboratory, but the error was separately recorded.

A trial of ceftriaxone AMR determination within the Indian GASP was also conducted using a prototype test method following reports of emergence of gonococci resistant to third-generation cephalosporins [8, 9], although parameters for defining ‘resistance’ to the extended-spectrum cephalosporins remain undefined.

Consolidated results for all the laboratories were sent in coded formats to each laboratory for self-assessment and comparative purposes. Individual feedback letters highlighted areas of susceptibility testing where errors were noted and included suggestions for enhancing performance in areas identified as needing improvement.

RESULTS

Seven laboratories participated in the Indian GASP EQAS at different times over the study period. Laboratory 1a participated in the first four challenges only and was replaced by laboratory 1b for the remainder of the period. Laboratory 6 joined the programme during 2005. Some strains could not be revived by some laboratories in the initial years of the study so that the annual numbers of test results in each centre differed.

Table 1 shows the overall network performance for tests on six strains using six antibiotics from 2001 to 2007. Data are also aggregated for all antibiotics except for ceftriaxone, which are shown separately. Overall, 944 (82%) of 1030 disc tests performed by the participants were in agreement with the expected results for the five ‘core’ antibiotics and 186 (18%) tests resulted in major or minor errors over 7 years. An additional 199 disc tests for ceftriaxone resulted in 53 errors (26·6%).

Table 1. Overall performance of laboratories participating in the Indian GASP EQAS for six strains and six antibiotics, 2001–2007

GASP EQAS, Gonococcal antimicrobial surveillance programme external quality assurance scheme.

* Incorrect results=all major and minor errors.

† Totals exclude ceftriaxone test results which are shown separately.

Aggregated error rates for individual antibiotics differed. The lowest proportion of errors was observed with tetracycline testing (12%) and the highest, other than ceftriaxone, with penicillin (23·5%), with those for ciprofloxacin at 20%, nalidixic acid 19% and spectinomycin 16%. Of the 42 incorrect disc test results for penicillin, five (2·4%) of the 208 tests were major, and the remainder, minor errors. Seven other errors occurred with β-lactamase testing. Forty-one (19·9%) of 206 tests for ciprofloxacin were incorrect. Of these, 24 (11·6%) were minor, and 17 (8·3%) major, errors. For nalidixic acid, major errors were more common – 38/202 tests (18·8%) – because there is no LS category. Spectinomycin testing resulted in 16% (33/206) incorrect results. These were also all major errors, because there is also no LS category for testing this antibiotic. Testing for TRNG saw a 12% (25/208 tests) error rate. There were an additional nine instances of incorrect categorization because of interpretive errors of technically correct results.

Error rates reduced substantially from 2005 onwards (Table 1). In 2001, 67% of all network test results were correct (number tested: 123) compared to 95·6% in 2007 (n=180). The average error rate for penicillin testing during 2001–2004 inclusive was 33·5% but 12·5% from 2005 to 2007. Similar or greater reductions in error rates occurred over the same periods with ciprofloxacin (from 29·2% to 10·4%), nalidixic acid (from 32·6% to 2·8%), spectinomycin (from 28·6% to 4%) and tetracycline (from 13·9% to 7·7%) testing.

The ceftriaxone AMR disc testing trial of strains with reduced susceptibility to ceftriaxone saw 40/52 (76·9%) tests, reported as S because of overlap in reported zone sizes between the S and LS groups. Further, in contrast to the continuing improvements noted for other antibiotics, the annual error rate for ceftriaxone testing was essentially unchanged over the duration of the study.

Table 2 shows annual trends in error rates for each laboratory for all antibiotics tested (including ceftriaxone) and for the network. Laboratory 1a had a very high error rate for the years 2001–2004 whereas laboratory 3 reported very low error rates (2·6%) over the entire period. Difficulties experienced by laboratories 4 and 5 in resuscitating panel strains because of delayed shipments may have contributed to the initially higher error rates. Next-day delivery was achieved in 2006 and 2007 so that all strains were tested by all participants.

Table 2. Annual performance of each laboratory participating in the Indian GASP EQAS from 2001 to 2007 for all antibiotics, including ceftriaxone

GASP EQAS, Gonococcal antimicrobial surveillance programme external quality assurance scheme.

Effect of repeat challenges and feedback

Multiple repeat challenges with the same panel strain with accompanying feedback on results from the coordinating centre improved performances overall and in individual laboratories. With four repeat challenges with two strains, the proportion of correct results increased from 75% to 95·7%. With two repeat challenges (using one strain) and one (with four strains), improvements from 76·7% to 88·3% and 79·5% to 87·7%, respectively were noted.

DISCUSSION

The EQAS systems used in GASP networks are essential for providing valid epidemiologically based AMR data for public health purposes [6]. The WHO has had international EQAS systems in place for proficiency testing for antibiotic resistance testing in some other bacteria of interest or public health importance since 1995 [12]. Up to 27% of test results have been ‘out of range’ in these assessments [12]. However, a WHO Working Group on AMR in N. gonorrhoeae had earlier in 1965 recommended use of quality control panels to enable international comparisons of gonococcal AMR [13], and these panels are now maintained and provided in separate EQAS by WHO collaborating centres for pathogenic Neisseria for the essential purposes of internal and external quality control and assurance [7].

Our study illustrates important improvements in individual laboratory and network capability achievable through application of these established principles over time when the ‘correct’ result was ultimately obtained for 95·6% of tests performed in the Indian GASP in 2007. This highly satisfactory outcome was achieved through continuing close collaboration between the WHO collaborating centre and the SEAR regional reference laboratory on the one hand, and the liaison of this same laboratory, in its role as the Indian GASP coordinating laboratory, with Indian GASP participants on the other. A single proficiency survey for AMR in N. gonorrhoeae in the UK with six gonococcal strains sent to 411 laboratories in 1986 saw an 11% error rate [14] for gonococcal sensitivity tests. In a survey from 14 laboratories from Western Europe [15], overall concordance using all methods (disc diffusion, agar dilution, E test) was highest for ceftriaxone (93%) and lowest for tetracycline (72%) which is in contrast to our study. Disc diffusion gave the lowest overall concordance (72%) compared to MIC determination (>88%) by either method in the study from Western Europe [15].

The Indian GASP EQAS also confirmed the value of repeat challenges with identical but anonymized strains accompanied by general anonymous feedback to participants and detailed comment to individual centres. The importance of this feedback, including that in relation to technically correct, but wrongly interpreted laboratory data was especially noted. The outcome of this study suggests that this continuing educative process, conducted anonymously by dialogue between the coordinating centre and participants on an individual basis, means that the Indian EQAS has extended its programme beyond that of proficiency testing to establish a viable network forum that has substantially strengthened the Indian GASP. Other national and regional GASP EQAS programmes obtained similar results using, initially, monthly challenges accompanied by similar feedback and network-based educative processes [4, 16]. However, the Indian GASP EQAS was restricted to annual evaluations because of limited resources. Improved laboratory performance may have been achieved earlier if more frequent challenges, accompanied by education and consultation, had been possible.

In this GASP EQAS, disc-diffusion techniques were used. Although there are limitations with this method, it was shown to be reliable in this setting where it is the only feasible option for recognizing resistant phenotypes in a network situation in India. The Indian GASP coordinating centre provides a referral service to participants for formal MIC testing if this is required. The validity of comparisons of detection of resistance phenotypic by either method in this setting has been confirmed by the Indian reference laboratory [17].

One major limitation of the disc-diffusion method revealed here was the failure to distinguish strains with ‘decreased susceptibility’ to ceftriaxone in a high proportion of tests. This finding provided valuable field data on the lack of utility in peripheral centres of the proposed system for ceftriaxone disc-diffusion testing that had worked well in a central reference laboratory. Recently additional published information provided further insights into the possible reasons for the failure of this trial of ceftriaxone disc-diffusion testing. A number of inter-related molecular changes occurred in multiple genes that are responsible for the alterations in susceptibility to extended-spectrum cephalosporins [18, 19]. However, the impact of these changes (both known and unknown) affects the extended-spectrum cephalosporins unequally. The injectable antibiotic ceftriaxone is least affected in terms of MIC increases and clinical treatment failure. The orally administered members of the group, such as cefixime and ceftibuten, are associated with greater relative MIC change and treatment failure [9, 18]. Consequently, results of susceptibility testing for ceftriaxone cannot represent the susceptibility status of all members of this group of antibiotics. Alternative testing methods relevant to all the different cephalosporins have therefore been developed and will be re-evaluated under peripheral centre conditions. Until these methods have been fully verified, the importance of using accurate MIC methods and appropriate internal controls for detecting reduced susceptibility to cephalosporins should be re-stated [11]. Finalization of these methods is relevant not only to India because of the detection of cephalosporin ‘non-susceptible’ strains of N. gonorrhoeae there [20], but also more widely because of the increasing spread and prevalence of these gonococci.

AMR surveillance of N. gonorrhoeae is critical for public health purposes [1, 6], but the data must be rigorously validated by ongoing appraisal of the quality of the results of the surveillance if it is to be used with confidence to alter treatment schedules [1, 6]. While the results of the Indian GASP EQAS challenges were encouraging, there is a clear need for continuing network-based educational programmes that emphasize adherence to proper laboratory testing methods, the importance of quality control, and the basic concepts of quality assurance, the latter including increased frequency of challenges accompanied by relevant feedback [6]. The lessons from the Indian GASP EQAS can provide helpful information in other settings at a time when there is a demonstrated need for more and better quality AMR data to assist in the control of gonorrhoea.