All systems of scoring animal units (groups, farms, slaughter plants, etc) according to the level of the animals’ welfare are based inevitably on normative decisions. Similarly, all methods of labelling, in terms of acceptability, are based on choices reflecting ethical values. The evaluative dimension of scoring and labelling does not mean that we should reject them, but it does mean that we need to make the normative and ethical background explicit. The Welfare Quality® scoring system is used as a case study in order to highlight the role of underlying value-based decisions. In this scoring system, which was designed in accordance with assessments and judgments from experts in animal and social sciences and stakeholders, we identify value-based decisions at the following five levels. First, there are several definitions of animal welfare (eg hedonist, perfectionist, and preferentialist), and any welfare scoring system will reflect a focus upon one or other definition. In Welfare Quality®, 12 welfare criteria were defined, and the entire list of criteria was intended to cover relevant definitions of animal welfare. Second, two dimensions can structure an overall evaluation of animal welfare: the individual animals and the welfare criteria (here 12). Hence, a choice needs to be made between the aggregation of information at the individual level (which results in a proportion of animals from the unit in a good vs bad state) and the aggregation at criterion level (which results in a proportion of criteria to which the unit complies vs does not comply). Welfare Quality® opted for the second alternative to facilitate the provision of advice to farmers on solving the welfare problems associated with their farms. Third, one has to decide whether the overall welfare assessment should reflect the average state of the animals or give priority to worse-off animals. In the Welfare Quality® scoring system the worse-off animals are treated as much more important than the others, but all welfare problems, major or minor, count. Fourth, one has to decide whether good scores on certain criteria can compensate for bad scores on others. In the opinion of most people, welfare scores do not compensate each other. This was taken into account in the Welfare Quality® scoring system by using a specific operator instead of mere weighted sums. Finally, a scoring system may either reflect societal demands for high levels of welfare or be based on what can be achieved in practice — in other words, an absolute assessment or a relative one may be proposed. Welfare Quality® adopted an intermediate strategy: absolute limits between welfare categories (Not classified, Acceptable, Enhanced, or Excellent level of welfare) were set, but the rules governing the assignment of an animal unit to a category take into account what had been observed on European farms. The scientists behind Welfare Quality® are keen to make the value-based choices underlying assessments of animal welfare transparent. This is essential to allow stakeholder groups to understand the extent to which their views are acknowledged and acted upon.

This paper focuses on the reliability of the multi-criteria evaluation model included in the Welfare Quality® protocol for growing pigs to aggregate the animal-based indicators, first to criteria, then to principle level and finally to an overall welfare score. This assessment was carried out in a practical application study on a sample of 24 farms in Germany. Altogether, 102 protocol assessments were carried out in repeated visits to these farms in order to evaluate the inter-observer and test-retest repeatability of the overall scores calculated by the multi-criteria evaluation system. Reliability is then assessed by the calculation of different reliability and agreement parameters: Spearman Rank Correlation Coefficients (RS), Intraclass Correlation Coefficients (ICC), Smallest Detectable Changes (SDC) and Limits of Agreement (LoA). Inter-observer repeatability was insufficient for the criteria comfort around resting, absence of injuries, expression of social behaviours, expression of other behaviours, good human-animal relationship and positive emotional state as well as for the principles good housing and appropriate behaviour. This is probably due in the main to insufficient repeatability of the underlying indicators that have been revealed in previous studies. Test-retest repeatability is predominantly insufficient. Overall, the present results highlight the importance of absolutely reliable indicators at the baseline level. Furthermore, it could be shown that the calculation procedure is partly incorrect and consequently needs correction. Therefore, this study is an important contribution to the future progression of the Welfare Quality® protocols and animal welfare assessment tools in general.

Animal welfare has become an important subject of public and political debate, leading to the necessity of an objective evaluation system for on-farm use. As welfare is a multi-dimensional concept, it makes sense to use a multi-criteria aggregation system to obtain an overall welfare score. Such an aggregation system is provided by the Welfare Quality® Network. The present paper focusses on the assessment of the multi-criteria evaluation model included in the Welfare Quality® protocol for growing pigs in order to aggregate the animal-based indicators first to criteria, then to principles and finally to an overall welfare score. Specifically, the importance of the indicators on the overall assessment of growing pig farms is analysed in a given population which consisted of a total of 198 protocol assessments carried out on a sample of 24 farms in Germany. By means of partial least squares modelling, the influence of measures in the calculation procedure is estimated by calculation and interpretation of Variable Importance for Projection (VIP) scores. Variable Importance for Projection scores revealed some meaningful, unexpected influences as the multi-criteria evaluation model of Welfare Quality® aimed at avoiding interferences and double-counting. Some of these influences led to the assumption that some measures might have potential as iceberg indicators, whereas others showed lesser importance. Thus, feasibility can be gained by the deletion and special weighting of indicators according to their importance. Altogether, the study is an essential contribution to the further development of the Welfare Quality® protocols as well as the application of multi-criteria decision systems in the field of animal welfare science in general.

The aim of this paper was to validate an alternative multi-criteria evaluation system to assess animal welfare on farms based on the Welfare Quality® (WQ) project, using an example of welfare assessment of growing pigs. This alternative methodology aimed to be more transparent for stakeholders and more flexible than the methodology proposed by WQ. The WQ assessment protocol for growing pigs was implemented to collect data in different farms in Schleswig-Holstein, Germany. In total, 44 observations were carried out. The aggregation system proposed in the WQ protocol follows a three-step aggregation process. Measures are aggregated into criteria, criteria into principles and principles into an overall assessment. This study focussed on the first two steps of the aggregation. Multi-attribute utility theory (MAUT) was used to produce a value of welfare for each criterion and principle. The utility functions and the aggregation function were constructed in two separated steps. The MACBETH (Measuring Attractiveness by a Categorical-Based Evaluation Technique) method was used for utility function determination and the Choquet integral (CI) was used as an aggregation operator. The WQ decision-makers’ preferences were fitted in order to construct the utility functions and to determine the CI parameters. The validation of the MAUT model was divided into two steps, first, the results of the model were compared with the results of the WQ project at criteria and principle level, and second, a sensitivity analysis of our model was carried out to demonstrate the relative importance of welfare measures in the different steps of the multi-criteria aggregation process. Using the MAUT, similar results were obtained to those obtained when applying the WQ protocol aggregation methods, both at criteria and principle level. Thus, this model could be implemented to produce an overall assessment of animal welfare in the context of the WQ protocol for growing pigs. Furthermore, this methodology could also be used as a framework in order to produce an overall assessment of welfare for other livestock species. Two main findings are obtained from the sensitivity analysis, first, a limited number of measures had a strong influence on improving or worsening the level of welfare at criteria level and second, the MAUT model was not very sensitive to an improvement in or a worsening of single welfare measures at principle level. The use of weighted sums and the conversion of disease measures into ordinal scores should be reconsidered.

The aim of this paper was to present an alternative multi-criteria evaluation model to assess animal welfare on farms based on the Welfare Quality® (WQ) project, using an example of welfare assessment of growing pigs. The WQ assessment protocol follows a three-step aggregation process. Measures are aggregated into criteria, criteria into principles and principles into an overall assessment. This study focussed on the first step of the aggregation. Multi-attribute utility theory (MAUT) was used to produce a value of welfare for each criterion. The utility functions and the aggregation function were constructed in two separated steps. The Measuring Attractiveness by a Categorical Based Evaluation Technique (MACBETH) method was used for utility function determination and the Choquet Integral (CI) was used as an aggregation operator. The WQ decision-makers’ preferences were fitted in order to construct the utility functions and to determine the CI parameters. The methods were tested with generated data sets for farms of growing pigs. Using the MAUT, similar results were obtained to the ones obtained applying the WQ protocol aggregation methods. It can be concluded that due to the use of an interactive approach such as MACBETH, this alternative methodology is more transparent and more flexible than the methodology proposed by WQ, which allows the possibility to modify the model according, for instance, to new scientific knowledge.