Diagnostic accuracy of enzyme-linked immunosorbent assay (ELISA) and immunoblot (IB) for the detection of antibodies against Neospora caninum in milk from dairy cows

Abstract

The aim of the study was to estimate the sensitivity and specificity of enzyme-linked immunosorbent assay (ELISA) and immunoblot (IB) for detecting antibodies of Neospora caninum in dairy cows, in the absence of a gold standard. The study complies with STRADAS-paratuberculosis guidelines for reporting the accuracy of the test. We tried to apply Bayesian models that do not require conditional independence of the tests under evaluation, but as convergence problems appeared, we used Bayesian methodology, that does not assume conditional dependence of the tests. Informative prior probability distributions were constructed, based on scientific inputs regarding sensitivity and specificity of the IB test and the prevalence of disease in the studied populations. IB sensitivity and specificity were estimated to be 98.8% and 91.3%, respectively, while the respective estimates for ELISA were 60% and 96.7%. A sensitivity analysis, where modified prior probability distributions concerning IB diagnostic accuracy applied, showed a limited effect in posterior assessments. We concluded that ELISA can be used to screen the bulk milk and secondly, IB can be used whenever needed.

Introduction

Neospora caninum is an obligatory intracellular parasite with the indirect biological cycle. Dogs and coyotes [1] have been identified as the definitive hosts of the parasite, while other domestic and wild animals, such as cattle, small ruminants, water buffalos [2] and white-tailed deers can be intermediate hosts [3]. N. caninum infected cattle usually remain carriers of the parasite and as a result a ‘store’ reservoir of it. N. caninum can be transmitted vertically or horizontally: (a) vertical/endogenous transmission, refers to the transmission from an infected mother to the embryo, leading to abortion or birth of clinically or sub clinically infected calves and (b) horizontal/exogenous transmission, alludes to the infection of the cow by consumption of faeces of infected dogs, which are considered to be definitive hosts [4]. The majority of cases reported in the literature were attributed to vertical/endogenous transmission [5]. The presence of domestic dogs on the farm is considered as a risk factor for infection of farm cattle [6]. Risk factors related to the management of the farm, stray dogs and climatic agents could also affect the extent of the problem [7].

N. caninum can create severe reproductive problems leading to extensive economic loss [8].

Studies conducted in Europe (the UK, Switzerland) established that neosporosis is the leading cause of abortions [5]. Epidemiological studies showed that neosporosis is a major problem in cattle all over the world [9]. Moreover, Wouda et al. showed that high prevalence of neosporosis in cattle and the presence of N. caninum seropositive dogs on the farm are highly correlated [10].

Diagnostic tests detecting antibodies against N. caninum either in serum or in milk are useful tools for the diagnosis, surveillance and control of neosporosis, as they can be applied easily and promptly for the screening of the farms. Milk samples collection is easier and faster and is regarded as less invasive for animals in comparison with serum samples [11]. The most widely used serologic tests include indirect fluorescent antibody test, enzyme-linked immunosorbent assay (ELISA), immunoblot (ΙΒ) and sero-agglutination. Particularly, ELISA has been widely established for neosporosis diagnosis, as it is easy to apply and enables rapid determination of antibody levels in fluids like milk and serum [12, 13]. The IB, on the other hand, is mostly used as an adjunct to other diagnostic tests already in use, rather than as a routine diagnostic screening method [13]. IB is regarded as more time-consuming and complicated in application and result interpretation (IB is of great importance in identifying immunodominant antigens against N. caninum) [14].

In case of absence of a gold standard, as in neosporosis case, latent class models were first introduced by Hui and Walter [15]. Hui and Walter's model assessed the diagnostic accuracy of two conditionally independent tests in two populations (of different prevalence) and this was further extended to take into account potential conditional dependence of diagnostic tests. Non-identifiable issues in all those models were addressed effectively by Bayesian modeling, namely by incorporating of prior knowledge of test performances [16]. From an epidemiological point of view, Bayesian models have seldom been applied to evaluate the diagnostic accuracy of ELISA against IB to detect antibodies against N. caninum. Furthermore, a diagnostic test's accuracy may vary between species, age, type of samples and medium tested, sample quality and other factors [17].

In this study, we used a latent class approach in a Bayesian framework to investigate and compare sensitivity (Se) and specificity (Sp) of ELISA and IB tests in milk samples for N. caninum antibodies’ detection at herd level, in order to demonstrate freedom and eradicate the parasite infection, confirm the diagnosis of cases, minimise abortion rates and therefore maximise profit. Furthermore, this study attempts to evaluate the use of milk as a sample type for the serological diagnosis of neosporosis in dairy cattle. The study followed STRADAS-paratuberculosis guidelines [18].

Methods

Dairy milk samples and study design

The target and source populations of this study were previously published by Sotiraki et al. [19]. Concisely, the study was carried out in 10 dairy farms, in four of the main dairy-farming regions of Greece (Epirus, Thessaly, Macedonia and Thrace) that represented more than 80–85% of the country's dairy population. Ten dairy-cow herds were randomly selected to be included in the study from all described areas. All herds comprised Holstein cattle. The total number of cattle bred in the 10 randomly selected herds was 1538, from which 1038 were cows and 535 heifers. Only 777 samples (49.4%) were obtained exclusively from cattle that were lactating at the day of sampling. No exclusion criteria (i.e. history of abortion) were set. All milk samples of 10–15 ml were obtained before the mechanical milking procedure and were preserved with 0.1 g sodium azide medium for transportation purpose. Upon receipt at the laboratory, samples were aliquoted and placed at −20 °C. Questionnaires were also collected, recording information about the number of abortions during the last 3 years, the occurrence of epidemic outbreak, the origin of the animals (imported, born in the field), the preventive measures taken in each farm against neosporosis, the type and use of the stable holdings, the type of insemination (artificial or natural), the availability of water, the general hygienic conditions and the presence of dogs or other animals on the farms [19].

The whole data collection process was prospective, as samples were collected and the tests under evaluation (TUE) were assessed after the initiation of the study and for the purpose of ELISA and IB evaluation.

Diagnostic tests for detecting antibodies

ELISA was selected to be evaluated because it is not a common practice concerning dairy cattle [20]. Since Sotiraki's et al. study [19], IB has been previously used as (i) gold standard for evaluation of ELISA assay results [21], (ii) reference standard of other tests’ results [22] and (iii) only once in neosporosis investigation in milk samples [11]. IB though time-consuming and more complicated to apply was considered as a high sensitivity and specificity assay.

Milk samples examined using, an indirect-house ELISA (p38-milk-ELISA), were considered to be positive for N. caninum antibodies if the OD was above 0.15, using purified NSRS2 [23]. The IB test was prepared using NC-1 tachyzoites that were cultivated for 1 day under serum-free conditions, while the reactivities between tachyzoites and sera were recorded for Mr 17,29, 30, 33 and 37 kDa, as previously published in the literature [24]. For IB, the samples that gave two or more immunodominant bands were considered positive for antibodies, while where only one immunodominant band was present the samples were considered ‘inconclusive’ [19]. Those latter samples were regarded as negative for our study.

The tests (ELISA and IB) used and underwent diagnostic evaluation were both in-house produced by Federal Research Institute for Animal Health (Institute of Epidemiology, Friedrich-Loeffer-Institut, Seestrasse 55, D-16868 Wusterhausen, Germany), which is a reference laboratory for N. caninum diagnosis (personal contact with Dr Smaro Sotiraki). Laboratory personnel and readers of IB (gold standard) test were masked to ELISA's results, as commonly applied in case of reference laboratories’ practice.

All milk samples were both tested with ELISA and IB by Sotiraki et al. and are summarised in Table 1. A total of 118 milk samples were tested positive for antibodies against N.caninum using ELISA, while there were 216 IB-positive samples. As there is no gold standard, it is not possible to determine unequivocally which samples had, indeed, antibodies against N. caninum. The discrepancy in the results of the two diagnostic tests can be attributed either to low ELISA Se or/and low IB Sp. Analysing the data in detail, we concluded that all, except for two, the samples which were proved to be positive with ELISA, were also positive with IB. Furthermore, there were samples proved to be negative with ELISA, but marginally positive or negative with IB.

Table 1. Results of ELISA and ΙΒ for the detection of antibodies against N. caninum, performed in individual milk samples of 10 Greek dairy farms (according to Sotiraki et al. [19])

Statistical analysis

We applied latent-class models firstly introduced by Hui and Walter [15] in a Bayesian framework [25] to make inferences about diagnostic sensitivity and specificity of ELISA and IB, for detecting antibodies against N. caninum in the milk samples. Bayesian methodology allowed the combination of prior scientific knowledge about the parameters of interest with the data from the study, in order to produce posterior inferences [25]. Estimations of Se and Sp of the ELISA and the IB were based on their cross-classified results. In order for the Hui and Walter model to be valid, three assumptions need to be met: (i) the diagnostic tests should be conditionally independent, (ii) the source data population should have the ability to be divided in two or more subpopulations with different prevalence and (iii) diagnostic Se and Sp recorded in those subpopulation with different prevalence should be consistent [15].

As both ELISA and IB are designed to detect the same biological phenomenon, namely the presence of antibodies against N. caninum in milk, the conditional independence assumption required by Hui and Walter paradigm was not warranted. For this reason, in this study, we used Bayesian modeling, namely both methodology proposed by Johnson et al. [16] for conditionally independent tests and Branscum et al. [25], which does not require the conditional independence assumption.

To validate the assumption of difference in prevalence between the two subpopulations, test results from individual farms were pooled into two sets of samples: (a) one set of samples (named population 1) included those obtained from farms (farms 1,4,6,7 and 10) in which domestic dogs were present, as dogs are the definitive hosts of the parasite and therefore the animals of the farms that coexisted with dogs had greater possibilities to be infected by N. caninum, than others that did not reside with dogs, (b) the other set of samples (named population 2) originated from farms without dogs (farms 2,3,5,8 and 9), where animals did not come in contact neither with domestic nor with stray dogs (namely with N. caninum oocysts shed in canine faeces) and therefore had less possibilities to be infected by the parasite. The reasoning behind this categorisation was that the presence of dogs on a farm is a risk factor for the presence of neosporosis [26].

To confirm the consistency of Se and Sp across the two subpopulations, we repeated models’ running three times and every time Se and Sp were found to be similar for all subpopulations.

Based on scientific (personal contact with experts and literature) inputs for Se, Sp of ELISA and IB and the prevalence of the studied populations, specific Beta prior distributions were obtained using the Beta Buster software (http://cadms.ucdavis.edu/diagnostictests). In each case, experts expressed their best estimation concerning tests’ parameters. Beta probability distributions were used to model the assessments of the experts and the related uncertainty. When constructing beta prior distributions, the most probable value for each parameter was used as the mode of the distribution, while the lowest (or highest) possible value was considered as the 5th (or 95th) percentile of the distribution.

Statistical analysis was conducted using WINBUGS (https://www.mrc-bsu.cam.ac.uk/software/bugs/). We used appropriate WINBUGS code authored by Branscum et al. [25], which is available at http://cadms.ucdavis.edu/diagnostictests/2indt2p.html.

For calculating both tests’ repeatability and reproducibility, a sensitivity analysis was conducted.

Results

Nine out of ten herds had abortions recorded in the past 3 years, from which one herd had epidemic abortions, while the rest eight had sporadic ones. Concerning the management practices, pregnant heifers were regularly used for replacement purposes, while dogs which are regarded as risk factors had free access in five out of ten premises and the overall hygienic status was good in six and poor in four of the herds included in this study. The time interval between the collections of milk samples for both TUE was not recorded in the primary data collected by Sotiraki et al. However, no interventions were mentioned between samples’ collection [19].

Bayesian models requiring conditional independence

We applied TAGS [27] software and the Bayesian models proposed by Johnson et al. [16] and Enøe et al. [28] that require conditional independence of the evaluated tests. Analysing the data conducted and although more than 10 iterations with specific burning were applied, convergence problems occurred, obviously because of the violation of the conditional independence assumption, which is a prerequisite for this methodology application.

Bayesian models not requiring conditional independence

We tried Bayesian methodology for the estimation of test parameters and the prevalence of the populations, using models that do not assume conditional independence of the tests, as proposed by Georgiadis et al. [29] and Branscum et al. [25]. From those two, we applied the model proposed by Branscum et al., who recreated the analysis of Georgiadis et al. [25, 29].

A difficulty in the construction of the prior probability distributions for the prevalence of the populations arouse, as no reliable data for the prevalence in the farms were available. Furthermore, the two study ‘populations’ were created artificially by mixing test results from several farms. Therefore, there were no actual defined populations corresponding to population 1 and population 2, for which prevalence assessments could be made. However, this seems of no importance to our study, since in our case the prevalence parameters were not of particular interest. Therefore, realistic but very diffuse prior distributions were constructed for the two prevalence parameters, in order to express our uncertainty about these parameters prior to the study. Considering the experts’ opinions, the use of two different prior distributions for prevalence was decided, depending on whether the animals of the population coexisted with dogs or not. Prior distributions were also constructed concerning the four parameters of dependence (λ _D, $\lambda _{\rm D}^{{\rm not}} $, γ _D, $\gamma _{\rm D}^{{\rm not}} $), which are included in the model proposed by Branscum et al. [25]. The parameters (a, b) of the beta prior distributions that were used for this study and the prior assessments on which their construction was based on are shown in Table 2.

Table 2. Prior probability distributions upon respective parameters (mode and percentile) set for ELISA and IB diagnostic accuracy and populations’ (1 and 2) prevalence, based on experts’ opinions

The Beta prior distributions generated for IB Se and Sp and for population 1 and population 2 prevalence assessments are presented in Figure 1. According to Branscum et al. [25], no prior probabilities are set for sensitivity and specificity of the second diagnostic test (ELISA). In this case, we prepared diffused prior probability distributions for all four dependent parameters (λ _D, $\lambda _{\rm D}^{{\rm not}} $, γ _D, $\gamma _{\rm D}^{{\rm not}} $), whose most possible value is the most possible prior value of the respective characteristics of the second diagnostic test. Thus, the prior probability distribution for λ _D and γ _D was B (2.15, 1.38), with mode 0.75 and 5th percentile 0.20, while prior probability distribution for $\lambda _{\rm D}^{{\rm not}} $ and $\gamma _{\rm D}^{{\rm not}} $ was B (2.07, 1.27), with mode 0.80 and 5th percentile 0.20.

Fig. 1. Prior distributions for parameters required in the two-population and two-test problem for the Bayesian analysis proposed by Branscum et al. data, Y-axes show probability density from 0 to 1 and X-axes as per plot label (clockwise) ‘sensitivity of IB’, ‘specificity of IB’, ‘prevalence of population 2’ and ‘prevalence of population 1’.

Cross-classified results of ELISA and IB of populations 1 and 2 are summarised in Table 3, with 95% probability intervals. In this table, ρ _D is defined as the conditional correlation of the two diagnostic tests for an infected animal and the ρ ^D− as the conditional correlation of the two diagnostic tests for an uninfected animal. The 95% Bayesian Probability intervals for conditional correlation between the two tests among animals without antibodies for N. caninum- (ρ ^D−) did not include 0 [ρ ^D− = 0.518 (0.117–0.729)]. Therefore, we concluded that the two diagnostic tests were conditionally dependent and decided to base our final inferences on the Branscum model, which does not require the conditional independence assumption [25].

Table 3. Mean and 95% posterior Bayesian probability intervals for Sensitivity (Se) and Specificity (Sp) of ELISA and IB, prevalence of populations (1 and 2) and conditional correlation of ELISA and IB in infected (ρ _D) and uninfected (ρ ^D−) animals

No adverse events from performing both ELISA and IB are reported in the literature.

In order to determine the effect of the prior distributions on the posterior estimates and their repeatability [29], we proceeded with a sensitivity analysis, in which we modified some of the prior distributions and assessed the effect on posterior inference, modifying the mode of the prior probability distributions. Two additional analyses were carried out: a) in the first more concentrated prior probability distributions for sensitivity were used, whereas b) in the second analyses specificity of IB was used. The estimates of IB and ELISA Se and IB and ELISA Sp, with B (36.70, 2.88) (most possible mode 0.95 and 95th percentile 0.85) and B (92.85, 10.08), respectively are presented in Tables 4 and 5. All changes had limited effect on the posteriors assessments, as the tests’ parameters were similar to the ones estimated previously (Table 3).

Table 4. Mean and 95% Bayesian probability intervals for Sensitivity (Se) and Specificity (Sp) of ELISA and IB, using models that do not require conditional independence of the tests, in case of B (36.70, 2.88) (sensitivity analysis)

Table 5. Mean and 95% Bayesian probability intervals for Sensitivity (Se) and Specificity (Sp) of ELISA and IB, using models that do not require conditional independence of the tests, in case of B (92.85, 10.08) (sensitivity analysis)

Discussion

Neosporosis can affect the performance and the profitability of a dairy farm, that is why prevention and control efforts should be ongoing. The validity of the diagnostic tests (under the specific conditions of the farm) used in these efforts, should be known, although this is sometimes difficult.

The aim of our study was to evaluate sensitivity and specificity of an ELISA and an IB test for the detection of antibodies against N. caninum in the milk of Greek dairy farms. To our knowledge, this is the first study to evaluate sensitivity and specificity of the diagnostic tests ELISA and IB for the detection of antibodies against N. caninum in milk from cows in dairy farms.

In a study conducted in Greece, Papadopoulos et al. concluded that the prevalence of neosporosis in Greece was 18.2% in bovine, after testing blood samples with ELISA [30]. Sotiraki et al. carried out a study in Greek dairy farms, intending to investigate neosporosis and the data collected were used in our study [19].

In this study, sensitivity and specificity of ELISA were estimated to be 60% and 96.7%, respectively, whilst the corresponding estimates for IB were 98.8% and 91.3%. Some of these estimates differ from those derived by Staubli et al. [31] and Schares et al. [32]. Past studies have demonstrated mixed superior and inferior estimates for all the rest parameters.

Staubli et al. found that sensitivity and specificity of IB were 98% and 100%, respectively, while the corresponding parameters for ELISA were both 87% [31]. That study used blood samples from experimentally infected animals, while the infection was confirmed by polymerase chain reaction. The researchers concluded that IB could be the most suitable accessory diagnostic tool of ELISA, as far as the serological diagnosis of neosporosis is concerned. Schares et al. estimated sensitivity and specificity of IB at 94.2%, after testing blood samples of experimentally infected bovine [21]. Frössling et al. calculated Se and Sp of ELISA 33.3% and 97.7% (in case of OD = 0.100) and 65.1% and 99.7% (in case of OD = 0.200), using logistic regression models [33]. Schares et al. concluded, after linear regression application, that Se and Sp of ELISA was about 93% [34]. Paré et al. estimated that Se and Sp of ELISA were 88.6% and 96.5%, respectively, using cut-off point 0.45 [35]. Björkman et al. concluded that ELISA Se and Sp was 100% and 96%, respectively, compared with Western blot analysis, while using both serum and milk samples, the agreement between those two types of samples was 95% [12].

One of the most probable reasons for the difference in estimates of the characteristics of the diagnostic tests in our study is that the data were derived from a study carried out under field conditions, with samples from naturally infected animals, whereas previous studies [31, 32] used experimentally infected animals. Test parameter assessments, which use data from experimental animals, are typically overestimated [36]. For instance, in an experimental study, the time of infection is known and the validation of the diagnostic test is performed when the concentration of antibodies is high. In contrast, in a natural field study, it is not known whether the animals were infected or not. Differences noticed with our results may, also, be attributed to the threshold (OD of ELISA) set. In some of the studies mentioned above, the diagnostic test set as comparison test was other than IB or/and ELISA. It is very probable that a serological test might have lower sensitivity in detecting antibodies at lower compared with higher titres. Additionally, the low sensitivity of ELISA in our study might be attributed to the fact that some animals could have been infected in the past and had a low concentration of antibodies that could be detected by IB, but not by ELISA. Differences may, also, be attributed to the use of different biologic materials (milk versus blood serum) and even more to individual test performance, which varies from herd test performance [37]. The age of animals, the lactation stage, the gestation stage and the time after birth are some of the factors that can affect the concentration of antibodies [38]. For example, infected calves 7–12 months old often provide false negative results, as the concentration of antibodies decreases after birth [39].

This hypothesis agrees with the existent, albeit limited, literature, presenting IB as a useful diagnostic tool for the serological diagnosis of neosporosis that can track even low concentrations of antibodies, where other serological methods cannot [31]. In the present study, it is concluded that IB has greater sensitivity, compared with ELISA, although, it is not a gold standard. In spite of the fact that some researchers refer to IB, considering it as a gold standard [21], our estimates for the test-parameters’ were lower than 1 which means that the test might give false results. Despite apparent advantages of IB or ELISA values, there are also other factors when considering the ideal test for a given herd situation and diagnostic laboratory. For example, although IB has greater sensitivity, it would be more suited in a large unknown status herd to initiate screening by use of ELISA, with follow-up confirmation of negative results by use of IB, rather than to initiate testing with a relatively expensive alternative.

The Bayesian methodology used in this work requires the use of prior information for the parameters of interest, in the form of prior probability distributions. In this case, prior information on sensitivity and specificity of the diagnostic tests was limited. For this reason, a combination of information from peer-reviewed papers and from experts’ opinions was used. The prior distributions that were applied for the analysis expressed the increased uncertainty associated with these prior assessments. Τhe two populations used for our study derived from a pooling of samples of animals from different farms and do not represent any existing natural population of animals. Technically, it was important that the two populations would have different prevalence. This is the reason why the pooling was based on a strong risk factor for the infection, the co-existence of the tested animals with dogs. This difference in prevalence was, probably, achieved as can be seen from the fact that there were no technical (i.e. convergence) problems observed during the application of the statistical methods and from the discrepancy in the posterior estimates of the two prevalence assessments. On the other hand, this study's prevalence estimates do not apply to any actual population. This does not create any problem for our study, as long as an interpretation of the prevalence is not sought. In any case, the aim of this study was to validate the diagnostic tests and not to estimate the prevalence of presence of N. caninum antibodies in the tested populations.

Finally, an assumption of the statistical methodology used is that each test would have the same sensitivity and specificity in animals from the two tested populations. If this assumption is not valid, bias affects the estimates regarding the population of highest prevalence [40].

Conclusions

A precise appraisal of diagnostic test accuracy is of major important for a better assessment of neosporosis surveillance and control programs. Our study shows that milk samples may also be used for neosporosis diagnosis via serological methods and this is of great importance in field conditions. We demonstrated that conditional independence assumption for ELISA and IB was not supported, as expected because both diagnostic tests were of the same biological background. While accounting conditional dependence of the two tests, more precise estimates arose. IB demonstrated greater sensitivity compared with ELISA and thus, IB can detect even lower concentrations of antibodies. However, ELISA demonstrated slightly higher specificity than IB and thus, fewer probabilities to give false negative results. These findings advocate for further research and studies needed to assess the consequences of these estimated diagnostic test’ accuracy.