Validation of the register-based lifetime antimicrobial usage measurement for finisher batches based on comparison with recorded antimicrobial usage at farm level

Abstract

Assessing the relationship between antimicrobial usage (AMU) and antimicrobial resistance (AMR) requires the accurate and precise utilisation of register data. Therefore, validation of register-based data is essential for evaluating the quality and, subsequently, the internal validity of studies based on the data.

In this study, different smoothing methods for Veterinary Medicine Statistic Program database (VetStat)-records were validated by comparing these with farm-records. Comparison between measurements included accuracy as; completeness and correctness, and precision as; a relative difference of the error, correlation with Fisher's z transformation and reliability coefficient. The most valid methods of those examined were then used in re-analyses of the abundance of AMR genes in 10 finisher batches from a previous study.

Improved accuracy was found when detailed smoothing methods were applied. Although the precision also increased, the effect was not as pronounced, as the usage estimate of all smoothing methods deviated moderately compared with the farm-registrations. Applying the most valid methods to the 10 finisher batches increased estimates of statistical model fit for aminoglycosides, lincosamides, tetracyclines and decreased estimates of statistical model fit for macrolides. The estimates of statistical model fit for sulfonamides and broad-spectrum penicillins remained the same.

Through refined data transformation, VetStat-records can be used to calculate a daily amount of AMU per pig reflecting the true usage accurately and moderately precisely, which is the foundation for calculating lifetime AMU.

Introduction

As the emergence and spread of antimicrobial resistance (AMR) bacteria is increasing worldwide, an understanding of the complex associations between antimicrobial usage (AMU) and AMR is urgently needed [1]. The AMU is generally acknowledged as the main cause. However, less is known of the quantitative relationship between AMU and AMR, as well as the interrelational effects between usage in humans, agriculture and veterinary sectors [2, 3]. Due to the potential risk of conveying resistance from animal microflora to human pathogenic bacteria, AMU for animals has gained increased attention [4].

Since antimicrobials (AMs) are vital for the treatment of bacterial diseases in veterinary medicine, responsible AM interventions aimed at reducing usage must be sufficiently effective to reduce AMR without compromising treatment options and animal welfare. Consequently, knowledge of the quantitative ‘AMU-AMR’ relationship is fundamental in order to obtain predictable results from interventions targeting AMU in animal production [5].

Several surveillance databases on AMU for animals have been established [6]. Among the first was the Danish Veterinary Medicine Statistic Program database (VetStat), which records purchases of medicines prescribed for animals [7, 8] and is commonly used for epidemiological studies of AMU–AMR relationships in Danish production animals [9–12]. As data from VetStat lack information on actual usage in farms, studies using these data share a mutual challenge in accuracy and precision compared with primary data and should, therefore, be validated [13, 14].

Farmers are obliged to register AMU for production animals on a daily basis. These records are often summed either by the farmer or by the veterinarian for the period between two consecutive visits by the veterinarian, which usually occurs at intervals of 30–65 days depending on production type and Health Advisory Contract. In this study, the farm records were the summed daily AMU between consecutive veterinarian visits. The farm records are not mandatory, but they provide the farmer and veterinarian with a quick overview of AMU and remnants from recent prescriptions. Validation measurements of VetStat-records compared with farm-records should include (1) accuracy, as the completeness and correctness and (2) precision, as the correlation, as the relative difference and as the coefficient of reliability of VetStat data. These measurements will demonstrate the quality of the data, which will be supportive when evaluating the trustworthiness of studies of AMU–AMR relationships using such data [13–15].

Currently, the most influential exposure characteristics of AMs, e.g. route of administration, level of dose, or duration of treatment, have not been fully determined in relation to the selection of AMR [5]. In previous studies utilising VetStat as the data source, data on AMU for pig herds have been extracted at the unit (piglets-sows/weaners/finishers) or farm level for periods of 6–12 months prior to sampling [10, 12]. This constitutes minimal differentiated estimates that do not take into account the variations within the extracted period in question. A study used a method that summed up a daily AMU as doses for finisher batches from birth to slaughter, calculating the lifetime AMU through the movements between units, thus, the method was independent of rearing site and captured variations over time [9]. In the same study, the daily usages were calculated by smoothing the amount (a recorded entry) based on days between records. Subsequently reflecting the number of days between one record and the next, within each age-group unit per farm. In contrast, this way of smoothing data does not take into account that different AMs and dispensing-types may be used differently by the farmer.

The objective of this study was to validate five different methods to smoothing VetStat data to estimate the number of ADDkg per pig day, reflecting the ‘true’ usage at the farms by comparing the results to farm-records in terms of accuracy and precision. The results from a previous study focusing on the effect of AM lifetime exposure on the abundance of AMR genes were then re-analysed with the most valid methods of those examined, for calculating AMU at finisher batch level. Two different farm size adjustments were then used to evaluate the same methods.

Materials and methods

Data sources

Two data sources on AMU were applied in this study: farm-records and VetStat-records.

The farm-records were manually registered by the owners or employees and contained information on the amount of an AM product used, including the dispensing-type, within the age-groups; piglets-sows, weaners and finishers, during specified periods. The farm-records were conveniently collected during farm visits related to an ongoing AMU-AMR study consisting of 83 randomly identified farms. A total of 25 farmers were asked to participate and 12 accepted. A total of 745 records on AMU were obtained, comprising 12 farm owners, 16 farms and 23 units within the period from January 2014 to May 2016.

Data from VetStat contains records on purchased medicines prescribed by veterinarians for animals. Each record has information on the product name, active-substance, dispensing-type, amount, target species, age-group, diagnosis group and farm code (ID) [7]. Data from VetStat were extracted from 1 year before the first farm recorded date to 3 months after the last of each farm to establish sufficient buffer time before and after the study periods to account for negative entries [16]. The data were then cleaned according to guidelines by correcting mismatches of animal species and/or age-group through cross-validating the data with Central Husbandry Register (CHR) data [16].

In order to produce comparable data across records, active compounds were converted into a unit measuring how many kilograms of pig could be treated per day, known as – Animal Defined Daily Doses per kilogram (ADDkg) [17].

Two sources of biomass estimates were applied as the adjustment factor for farm size; (i) number of pigs on any given day at the farms, obtained from the CHR, where all farms with production animals are recorded and (ii) the yearly production adjusted to the number of pigs on any given day, obtained from the Pig Movement Database (PMD) [7]. The CHR stores information on a farm code (ID), which refers to a specific geographical location and includes information such as ownership, animal species and the number of animals per age-group (sows/weaners/finishers), on any given day. Although sows and piglets are in the sow unit, the number of sows is included in this age-group, since piglets are not registered in the CHR. In the PMD, the number of pigs, date, ID of origin farm and ID of destination farm for each movement is recorded [7].

Estimation of AMU

Validation

The usage of an AM product (l), during a period (k), in an age-group (j) (piglets-sows/weaners/finishers) in a farm (i) was estimated as Doses _{i ,j,k,l} with the unit; ADDkg/pig day, using formula (1):

$$ \# Doses_{i,j,k,l} [ADDkg / pig\; day] = \displaystyle{{\# mg_{i,j,k,l}} \over { \# days_k \; {\ast} \; ADDkg_l \; {\ast} \; \# pigs_{i,j,k}}}$$

where: #mg = the amount of an AM product registered as usage or recorded as a purchase in a specific farm/age-group/period, #days = the number of days of the period when the recorded amount was used, #pigs = the number of sows/weaners/finishers on any given day registered in CHR, or the yearly production adjusted to the number of pigs on any given day registered in PMD.

The #days was calculated using five different methods. The first method (1) assumed that the AMU in a farm recorded period was equivalent to the purchases of AMs in that recorded period. The other four methods (2–5) were all calculated assuming that the amounts of recorded AM products were used in a period between one recorded date and the next. The subsequent date was defined based on different assumptions related to usage pattern over time at the farms. Consequently, the four smoothing methods differed in the number of days (#days) between one record entry date and the next, when the age-group, dispensing-type and antimicrobial class (AMC) alternately and together were taken into account (Fig. 1). In the less detailed method 2, the #days between two record entries was set at the age-group level, assuming that a new record of any AM product was due to the previous recorded AM products were consumed by that age-group. Method 3 assumes that when a new record of an AM product of either parenteral or peroral dispensing occurs within an age-group, all the former AM products of the same dispensing-type were consumed. Method 4 assumes that when new recorded AM product of an AMC occurs within an age-group, all the former recorded AM products of the same AMC, irrespectively of dispensing-type, were consumed. Method 5 was a combination of methods 3 and 4 (Fig. 1).

Fig. 1. Illustration of the differences in Number of days (#days) for the five methods of calculating antimicrobial usage at the farm level. For method 1, the #days was based on the farm record periods. From methods 2 to 5, the #days increased as the intervals between one record entry date and the next increased when similar records based on age-group, dispensing-type and antimicrobial-class (AMC) were matched.

The calculation of #days was based on three assumptions. First, if the #days was less than 8 days, the following subsequent record date was used instead. Second, if no subsequent date was found, the mean of the former was applied. If no subsequent date was found and no mean of prior #days was available, 365 days was utilised. Third, all #days exceeding 365 days were substituted by 365 days.

The calculated Doses _{i ,j,k,l} obtained for methods 2–5 were date-specific estimates. In order to compare these with the farm-records, the date-specific estimates were summed equivalent to the periods during which the farm-records were calculated and a daily average Doses _{i ,j,k,l} usage was calculated.

Re-analyses

For the ten finisher batches from the study [9], the date-specific lifetime AMU (Doses) in the different age-groups was re-calculated by means of formula 1 for the AMCs; aminoglycosides, broad-spectrum penicillins, lincosamides, macrolides, sulfonamides and tetracyclines, using the most valid methods. Furthermore, two different biomass adjustments were applied as the number of pigs on any given day; (A) the CHR and (B) the PMD.

The number of Doses was summarised at AMC level for each rearing period per unit, based on the finisher batches’ rearing periods in days; days 1–85 in the finisher unit, days 86–135 in the weaning unit and days 136–160 in the sow unit [18], where day 1 corresponds to the day of sampling. The number was then adjusted to suit the proportion of animals being moved from a farm. Subsequently, for each AMC, the lifetime AMU were calculated for each finisher batch by summarising Doses through the rearing pathways. Even though AMU for sows was included in the usage for piglets, previous studies have shown that this affects the abundance of AMR genes in the piglets’ microbiota, thus, it was assumed equivalent to usage for piglets [19].

Data analyses

Validation

Throughout the validation, the VetStat estimates were compared against the farm-record estimates, which were assumed to be the ‘true’ state of AMU at the farms.

For the accuracy and precision assessments of the relationship between farm-records and VetStat-records, the calculations performed for the observations were mutually independent and dependent, respectively. Consequently, to adjust for potential within-level clustering, all of the validation results were average-adjusted by farm, age-group, dispensing-type or AMC levels to assess the impact of clustering compared with the crude estimates.

Accuracy – completeness and correctness

The completeness constitutes the observed number of VetStat-records compared with the number of farm-records (a/(a + c)) and the correctness constitutes the number of correctly identified VetStat-records compared with the number of VetStat-records that were found (a/(a + b)), set in a 2 × 2 table [13, 14].

Precision – relative difference

The relative difference of the error was calculated as the absolute difference between farm and method, divided by the arithmetic mean of the usage given by farm and method (rd _error = (Doses _farm − Doses _method)/((Doses _farm + Doses _method)/2)).

Precision – correlation coefficient

The correlation coefficient ( r _z ) was calculated by applying Fisher's z transformation ( r _z  = (e ^2z –1)/(e ^2z  + 1), where z = 0.5ln((1 +  r )/(1 −  r )) [20]. The adjusted r _z should be interpreted as the general correlations between farm and method at the level of adjustment. The averaged correlations are less affected by sampling distribution skew, suggesting a less biased statistic [20].

Precision – reproducibility (reliability coefficient)

The reliability coefficient (ρ _xx  = 1/(1 + (σ _Error /σ _Dosesfarm)²), where Error = Doses _farm − Doses _method), between the Doses _farm and Error obtained, was calculated for each of the five methods [21]. The reliability coefficient describes the average magnitude of the error, the reproducibility. For linear regression, this equals the bias factor; β _observed = ρ _xx  * β _true and thus, can potentially be used for adjustment of β _observed [21]. Subsequently, the effect estimates obtained in the re-analysed linear regression models presented below were adjusted for the attenuation effect of data error.

Re-analyses

To investigate the influence of the validation results of this study, the findings from the previous study [9] of the effect of six AMCs on the abundance of the same classes of AMR genes were re-analysed by applying the most valid methods in calculating the lifetime AMU. In that study, AMR genes for the classes: aminoglycoside, lincosamide, macrolide, beta-lactam, sulfonamide and tetracycline were obtained using whole community sequencing (WCS) and were measured as reads per kilobase reference per million [22].

The lifetime AMU measure, CHR adjusted (A), for the ten finisher batches was used in linear regression re-analyses to assess each effect on the abundance of AMR genes by evaluating the changes in adjusted R-squared (Adj.R ²), Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). In addition, the reliability coefficient of the most valid method of the presented was applied to adjust the β-coefficients from the linear regression re-analyses.

Finally, the difference of effect of the two lifetime AMU measures, (A) CHR adjusted and (B) PMD adjusted, was evaluated.

Tools

WPS Workbench, Version: 3.1.1.0.0, Microsoft Excel 2016 and R, version 3.3.3 were used for data processing and data analyses.

Results

While cleaning the VetStat data, 19 records were encountered that could not be corrected. Some AM products were prescribed and purchased (recorded in VetStat) one time only, but the usage of these could not be found in the farm-records. In addition, AM products were recorded for one age-group but registered as usage at the farm for another age-group, or for two age-groups.

Validation

Completeness and correctness

Table 1 shows the completeness and correctness results obtained by comparing Doses _farm to the five Doses _method, respectively. The smoothing methods from 1 to 5 had a positive effect on the completeness, which increased from 0.60 to 0.86 and a minor negative effect on the correctness, which decreased from 0.91 to 0.84 (Table 1). The results obtained when performing the average adjustments at farm, age-group, dispensing-type and AMC levels led to a decrease in the overall completeness and correctness results, though the beneficial trend when smoothing remained the same (Table 1).

Table 1. The correctness and completeness of Doses _method 1 to 5, compared with Doses _farm at population level and average-adjusted at farm, age-group, dispensing-type and antimicrobial-class levels

Relative difference

The distributions of the rd _error for the smoothing methods are shown in Figure 2, illustrating that the number of farm-records not found by the smoothing method (rd _error = 2) decreased from Doses _method 1 to 5. However, concurrently, the number of spurious records (rd _error = −2) was shown to increase, while the distribution of rd _error narrows around zero going from method 1 to 5.

Fig. 2. The count distribution of the relative difference of the error (rd _error) for the Doses _method 1 to 5 compared with Doses _farm.

The boxplots of the rd _error of the five Doses _method, compared with Doses _farm show that the 0.75 quantile decreases substantially and the rd _error observations together with the median move toward zero from method 1 to 5 (Fig. 3). Furthermore, it shows that going from method 1 to 5, the mean and the range of the standard deviation of the rd _error decreases towards zero (Fig. 3).

Fig. 3. Boxplots of the relative difference of the error (rd _error) for the Doses _method 1 to 5 compared with Doses _farm. The black dots show the individual observations. The orange dots and error bars represent the mean and standard deviation of the rd _error.

Since the smoothing methods (#days) depended on similar VetStat-records regarding the age-group, dispensing-type and AMC levels, the rd _error of the five Doses _method was average-adjusted accordingly. Boxplots at an age-group level were in concordance with general findings (Fig. 4). In contrast, boxplots at dispensing-type level revealed that method upscaling from 1 to 5 was beneficial for parenteral dispensing, but not for peroral (Fig. 5). For peroral dispensing, method 3 provided a better result for the rd _error.

Fig. 4. Boxplots at the age-group level of the relative difference of the error (rd _error) for the Doses _method 1 to 5 compared with Doses _farm. The black dots show the individual observations. The orange dots and error bars represent the mean and standard deviation of the rd _error.

Fig. 5. Boxplots at the dispensing-type level of the relative difference of the error (rd _error) for the Doses _method 1 to 5 compared with Doses _farm. The black dots show the individual observations. The orange dots and error bars represent the mean and standard deviation of the rd _error.

The boxplots of the rd _error of the five Doses _method at farm level show considerable variation between farms, which is most likely to be related to the difference seen between dispensing-types (result not shown). Similar observations were made at AMC level and at AMC combined with dispensing-type level, (result not shown).

Correlation coefficient

In Table 2, the correlation coefficient ( r ) between the Doses _farm and the five Doses _method, show that by incorporating age-groups, dispensing-type and AMC in the smoothing methods, the correlation also increases. This mainly follows the beneficial trends of smoothing from the completeness and relative difference of the error results.

Table 2. The correlation coefficient ( r ), the Fisher z transformed correlation coefficient (r _z ) and the reliability coefficient (ρ _xx ) between the Doses _farm and the five Doses _method are shown, respectively along with the average-adjustment by farm, age-group, dispensing-type and antimicrobial-class levels, of each coefficient

In relation to the z average-adjusted correlation coefficient (r _z ) of the five Doses _method, the farm-level adjustment changed the results most, followed by age-group, dispensing-type and AMC level. However, the upscaling smoothing method trend remained the same, independent of the average adjustment level (Table 2). Furthermore, regardless of the level at which the average adjustment is performed, the r _z remains within a narrow range.

Reliability coefficient

For the five smoothing methods, the coefficient of reliability (ρ _xx ) ranged from 0.60 to 0.68 and the average adjustment at farm, age-group, dispensing-type and AMC levels had a similar decreasing effect on the values compared with previous findings. However, the beneficial upscaling method trend remained the same, independent of the average-adjustment level (Table 2). The reliability coefficients of smoothing methods 1 to 5 were all values below 1, meaning that the methods underestimate the AMU compared with the ‘true’ state, obtained from the farm-records (Table 2).

Re-analyses

For the ten finisher batches in the previous study, smoothing methods 5 and 3 for parenteral and peroral AMs, respectively and farm size adjusted using CHR, were applied to calculate the lifetime AMU for the AMC; aminoglycosides, lincosamides, broad-spectrum penicillins, macrolides, sulfonamides and tetracyclines. The lifetime AMU estimates sum up usage for the entire rearing period per AMC. The lifetime AMU estimates were used as explanatory variables in linear regression re-analyses on the abundance of AMR genes attributed to those AM classes. These results were subsequently compared with the regression results obtained in the previous study (Table 3).

Table 3. The results of the linear regression of the previous model and the smoothing methods 3 combined with 5, adjusted by CHR model for usage and abundance of AMR genes to aminoglycosides, lincosamides, broad-spectrum penicillins/betalactam, macrolides, sulfonamides and tetracyclines

The application of smoothing methods 5 and 3 for parenteral and peroral AMs, respectively, increased the estimated fit of the models (Adj.R ², AIC and BIC) and therefore potentially explained a larger part of the abundance of AMR genes against aminoglycosides, lincosamides and tetracyclines. For sulfonamides and broad-spectrum penicillins/betalactam, the estimated fit of the models decreased slightly. In contrast, the estimated fit of the model for macrolides decreased substantially (Table 3).

When the β-coefficient estimate of the model comprising methods 3 and 5 was adjusted in relation to the population ρ _xx , the β-coefficient increased by 49%.

The model comprising methods 3 and 5 combined was further evaluated based on alterations of the biomass, model A and B (Table S1 in the supplementary material). The adjustment change of the number of pigs from CHR to PMD had an overall improved effect on tetracyclines, broad-spectrum penicillins, macrolides and lincosamides and the opposite result was found for aminoglycoside and sulfonamides (Table S1 in the supplementary material). The impact of adjusting with PMD rather than CHR related mainly to usage in the age-group; piglets. For the CHR, the number of sows is used as the adjustment factor, resulting in a high number of doses for the piglet age-group, compared with the PMD adjusted estimates (result not shown).

The most notable results were the B models, which had AMU split by dispensing-type into two variables. For these, the estimates of statistical model fit were improved for aminoglycosides, lincosamides and tetracyclines, (Table S1 in the supplementary material).

Discussion

Validation

The completeness of VetStat-records increased from the less detailed method 1 to the more detailed method 5. This was due mainly to the pattern for parenteral usage of AMs, small amounts were used each month and rarely recorded in VetStat. Therefore, the detailed method 5 reflected the true usage of parenteral AMs more closely. Simultaneously, the pattern for peroral usage of AMs caused a reduction in the correctness. According to the farm-records, large amounts of AMs were used for group-treatment within a limited time. As a result, more detailed smoothing caused spurious AMU. Major variations in correctness and completeness could be observed between farms, which could mainly be attributed to dispensing-type and incorrect VetStat-records.

The same pattern for parenteral and peroral AMU affected the relative difference and the correlation coefficient. These became more precise for parenteral usage only when more detailed smoothing methods were applied. Consequently, our results indicate that, due to the differences in usage patterns seen between dispensing-types, the overall most valid method, method 5, for smoothing out the VetStat-records is not applicable for both parenteral and peroral dispensing. For the latter, method 3 is the most valid of the examined methods.

In order for the secondary data, to reflect the true state in a population, high completeness and correctness are required [13, 14]. For the overall most valid method, method 5, the completeness can be categorised as fair [23] and applying different smoothing methods to dispensing-type increased the completeness. In addition, obtaining values of the precision and the impact of the estimate on the statistical association are important for result assessments [15, 21]. A good correlation between farm-records and smoothing method was demonstrated, though it has been pointed out that correlation estimates may not be the optimum method for assessing agreement between methods [15]. In contrast, the standard deviation of the relative difference of the error and the reliability coefficient demonstrated a less precise estimate. Regardless, the reliability coefficient can be used to adjust the β-coefficient in a linear regression, thus the estimate influences the statistical association between AMU and AMR [15, 21].

VetStat gives unique opportunities to study AMU at farm level and its effect on AMR. AM stewardship at farm level and correct recording in VetStat are essential to improve data transformation further. VetStat can provide accurate and precise measurements of AMU through data transformation, which was observed for a number of farms in the validation part. Moreover, VetStat is easily accessible for large parts of a population at farm level [24]. Access to accurate and precise data can then form the basis for establishing knowledgeable guidance and/or adjustments of AMU practices at herd level, with considerably lowering effect on AMU as a result [25]. In addition, the knowledge may also be supportive for detailed risk assessments and trend analyses.

Re-analyses

The results of the re-analyses study indicate that using the alternative smoothing methods produces a better fit regarding the models estimating the effect of AMU on AMR gene abundance. Moreover, when the estimated effects were adjusted by applying the population reliability coefficient, an even higher effect of the lifetime AMU on the abundance of AMR genes was observed, which indicates that the effects estimated in the regression analyses are all underestimated. These results highlight the general importance of valid data in epidemiological studies in order to obtain unbiased quantitative estimates of effects [13–15, 21, 26]. As indicated by the results from the re-analyses, by optimising the utilisation of register data as a proxy for the AMU in pigs and adjusting the regression results obtained based on the results of this validation study, the usage, measured as lifetime AMU, can explain up to 70–80% of the variation in abundance of AMR genes observed between finisher batches.

The deviating result of the effect of macrolide may arise from the time of usage, as the estimated lifetime AMU takes no time-component into account, e.g. usage at different ages has a different impact on the abundance of AMR genes [27–29].

The results of the biomass adjustments according to the CHR and PMD number of pigs revealed that the latter could be a potential substitute for the former. The PMD adjustment was the number of pigs on any given day, estimated from the production of pigs 1 year prior to sampling. This estimate is neutral, as it solely reflects the number of animals being moved, in contrast to the CHR number of pigs, which is a farmer's evaluation of management performance and averages on any given day, thus, more subjective to bias.

Conclusions

Based on the validation results, it can be concluded that the VetStat database can be used for refined data transformation to improve accuracy and precision to reflect ‘true’ AMU at the farm level. Furthermore, the reliability coefficients show that the calculations of the daily amount of AMs used per pig underestimate the usage independent of method.

The knowledge obtained was used to re-calculate lifetime AMU, which in linear regression models provided an overall more beneficial effect on the estimates of statistical model fit than the previous calculation of lifetime AMU. The linear models can be compared only in terms of estimates of statistical model fit, whereas the coefficient estimates should be interpreted with caution due to the limited number of finisher batches in the study.

The PMD could represent an alternative to the CHR for biomass adjustment or should be used to cross-validate the CHR.