Enzyme-linked immunosorbent assays in seroprevalence studies of Q fever: the need for cut-off adaptation and the consequences for prevalence data

Summary

Large outbreaks of Q fever have recently increased the awareness of this disease as a public health issue. Knowledge of the general impact of Q fever relies mainly on seroprevalence studies and it is fundamental that seroprevalence is assessed accurately. Therefore we evaluated the few enzyme-linked immunosorbent assays (ELISAs) commercially available for this purpose. An outbreak in 2005 in Jena, a city of 100 000 inhabitants, gave us the opportunity for the evaluation. However, we found disappointingly low sensitivities for two (42% and 51%) of three commercial ELISAs for detecting past infection. Nevertheless, all assays had good classification potential but cut-off adaptation is needed. Based on the unequal worldwide distribution of the differently performing tests in studies, Q fever seroprevalence is likely to be underestimated in studies from Europe whereas the data from North America and Australia are likely to be more reliable.

INTRODUCTION

Seroprevalence studies are a common tool for defining the true burden of a disease. However, assays are often adapted to detect either acute or chronic infection and may therefore perform differently when used in other contexts. Coxiella burnetii, the pathogen causing Q fever, typically causes outbreaks of the disease. The largest outbreak occurred from 2007 to 2010 in The Netherlands and produced more than 4000 registered human cases [1, 2]. Knowledge of the general impact of Q fever relies mainly on seroprevalence studies [3]. It is fundamental therefore that seroprevalence is accurately assessed. We therefore evaluated the few enzyme-linked immunosorbent assays (ELISAs) commercially available for this purpose. Furthermore, as each of these assays is used in different parts of the world, our evaluation also helps in comparing seroprevalence data from different regions.

An outbreak in 2005 in Jena, a city of 100 000 inhabitants, gave us the opportunity for the evaluation. In 2005, 331 cases were reported in a densely populated area [4]. In order to detect chronic Q fever infection a follow-up study was initiated where 185 inhabitants of the affected area participated. C. burnetii infection was determined by seroconversion using the accepted reference method immunofluorescence assay (IFA) (Focus Technologies, USA). Six years after the outbreak we retested the participants for C. burnetii antibodies using three ELISAs [Virion/Serion (Germany), Panbio Diagnostics (Korea), IBL International (Germany)].

METHODS

All 185 participants in the follow-up study of 2005 were contacted, regardless of their Q fever status in 2005. Written informed consent was obtained, and the study was approved by the Ethical Committee of University Hospital Jena.

Blood samples were collected between June and October 2011 and sera stored at −80 °C. Sera of 92 blood donors were investigated in parallel.

To detect ph2 IgG, we performed three ELISAs (Virion/Serion, IBL International, Panbio Diagnostics) according to the manufacturers' instructions. The first two ELISAs were performed in 2011. The Panbio ELISA, however, was performed in 2014 because the assay only then became available in Germany. This meant that the sera had to be thawed a second time. Serodilutions and the IBL and Panbio ELISAs were conducted manually. Optical densities were read at 450 nm with a reference wavelength at 620 nm (Sunrise, Tecan). The Virion/Serion ELISA was performed using the automatic immunoassay analyser BEPIII (Siemens, Germany) and extinction was read at 405 nm.

Receiver-operating characteristic (ROC) curves combined with the Youden index were utilized to identify optimal cut-off points using SPSS software (SPSS v. 21, USA).

RESULTS

Of the 185 people contacted, 94 participated in the study. Of these 94, 84 had tested positive and 10 negative for C. burnetii infection during the 2005 outbreak. In the 84 positive persons the female:male ratio was 1:1·2 (age range 22–88 years) and of the 10 negative persons the female:male ratio was 1·5:1 (age range 39–86 years).

Of the blood donors tested all except two had ELISA results similar to the 10 participants seronegative in 2005 (Fig. 1). The two exceptions, blood donors 108 and 169, had markedly high results in all three assays. These high values indicated that these two blood donors had suffered an unnoticed C. burnetii infection.

Fig. 1. Boxplot for the different assays for participants with proven infection in 2005 (p), participants of the follow-up study without infection in 2005 (n) and blood donors (bd).

The participants that tested negative in 2005 (n = 10) and the blood donors (n = 92) were combined into a control group (n = 102).

The sensitivity of the different assays ranged from 42% to 100% and all had specificities >90% (Table 1).

Table 1. Performance of the ELISAs with and without cut-off adaptation

CI, Confidence interval.

* People without infection in 2005 and blood donors.

However, all ROC curves discriminated well between infected and uninfected individuals (Fig. 2) with area under the curve (AUC) ranging from 0·971 for the Virion/Serion ELISA to 1·00 for the Panbio ELISA.

Fig. 2. Receiver-operating characteristic curve of the different assays. AUC, Area under the curve; CI, confidence interval.

The optimal cut-off values for the tests as determined by highest Youden index was 10 U/ml (10·12 U/ml) for Virion/Serion ELISA, 7 U (7·14 U) for IBL ELISA and 20 U (19·26 U) for Panbio ELISA. By applying these cut-offs, a large increase in sensitivity was achieved for the Virion/Serion and IBL ELISAs with little decrease in specificity. An increase of the cut-off for the Panbio ELISA kept sensitivity at 100% and increased specificity to 99% (Table 1).

DISCUSSION

It is disappointing that we found low sensitivities for detection of past Q fever infection for the Virion/Serion and IBL ELISAs. However, both Virion/Serion and IBL ELISAs had excellent ROC curves indicating that these ELISAs discriminate well between seropositive and seronegative samples (Fig. 2). Good performance characteristics (sensitivity 93·4%, specificity 98·5%) were determined by the manufacturers of the Virion/Serion ELISA with sera from patients with clinical Q fever symptoms. Sensitivity and specificity >90% were declared by the manufacturers for the IBL ELISA without further explanation. A low sensitivity of 59% with a specificity of 97% for the Virion/Serion ELISA compared to IFA (cut-off 1:64) was described previously, but increasing the IFA cut-off to 1:512 increased the kappa value from 0·63 to 0·81, suggesting that only high antibody levels are detected by the ELISA [5]. As antibody levels decrease over time, the detection of low antibody levels is necessary in seroprevalence studies. For both these ELISAs, a large reduction in cut-off improved performance in our context.

The excellent sensitivity of the Panbio ELISA was not altered by a reduced cut-off but there was only little benefit for specificity. Further, thawing the sera twice, as done when testing by the Panbio method, did not seem to affect the reliability of the results.

Contrary to our findings of 100% sensitivity, a low sensitivity (71%) with high specificity (96%) was found for the Panbio ELISA in comparison to complement fixation test (CFT) and IFA in samples of recent and past infection [6]. Another study, comparing Panbio ELISA and IFA in samples from routine diagnostics, where patients are supposed to have acute infection, obtained an excellent ROC curve with an AUC of 0·99 together with a sensitivity of 98% and a specificity of 96% [7].

However, remaining seropositivity found even after 5 years and 12 years in Australian patients and patients from Birmingham, respectively, supports the applicability of assays based on antibody detection for prevalence data [8, 9]. When Q fever infection dates back several years and low antibody levels have to be considered (e.g. in seroprevalence studies) we suggest using adapted cut-off values.

Antibody responses may be influenced after antibiotic treatment as found for borreliosis [2]. Antibiotic treatment might also have influenced our results because most of our patients had received adequate treatment because of the high awareness of Q fever during the Jena outbreak in 2005. Unfortunately, there is little research on this topic for C. burnetii. Only one study has focused on the issue and found no significant differences between the ph2 IgG titre frequencies for adequately and inadequately treated patients. Similarly, there were no differences between early and late timing of treatment at 3, 6 and 12 months after infection [10].

In Europe seroprevalence studies on Q fever are usually performed using Virion/Serion ELISA, whereas Panbio ELISA was used for studies in North America and Australia [11–13]. This means, according to our data, that Q fever seroprevalence is likely to be underestimated in studies from Europe whereas the data from North America and Australia are likely to be more reliable.

In conclusion, we found that all ELISAs provided good classification potential. However, in contrast to the Panbio ELISA, cut-off adaptation for Virion/Serion and IBL ELISAs is needed when used for seroprevalence studies. Seroprevalence data from Europe are probably underestimated whereas data from North America and Australia are probably reliable.