A structured expert judgement elicitation approach: how can it inform sound intervention decision-making to support household food security?

Abstract

Objective:

To examine structured expert judgement (SEJ) elicitation as a method to provide robust, defensible data for three determinants of household food security (food cost, household disposable income and physical access) for quantifying a proof-of-concept integrating decision support system for food security.

Design:

SEJ elicitation is a validated method for obtaining unavailable data, but its use in household food security in high-income countries is novel. Investigate Discuss Estimate Aggregate (IDEA) elicitation protocol was implemented, including quantitative and qualitative elements. Using specific questions related to three determinants, food security experts were encouraged to Investigate – estimate individual first-round responses to these questions, Discuss – with each other evidence on the reasoning and logic of their estimates, Estimate – second-round responses, following which these judgements were combined using mathematical Aggregation.

Setting:

Victoria, Australia.

Participants:

Five experts with a range of expertise in the area of household food insecurity participated in the SEJ elicitation process.

Results:

The experts’ ability to provide reliable estimates was tested and informed the aggregation of the collection of individual estimates into a single quantity of interest for use in decision support. The results of the quantitative elicitation show the impact of combinations of varying household income, food cost and physical access on household food security status and severity and is supported by the experts reasoning during elicitation.

Conclusion

This research provides insight to the application of SEJ where elicited data can inform and support intervention decision-making specific to household food security, especially where evidence is absent or of poor quality.

Despite an abundance of food and relative wealth, food insecurity is a complex and serious public health challenge increasingly faced by high-income countries, affecting the health and wellbeing of those experiencing it^(1–6). Food insecurity is the limited or uncertain availability of households’ and individuals’ physical, economic, and social access to sufficient, safe, nutritious and culturally relevant food by socially acceptable means⁽⁷⁾. Households may transition between episodic and chronic experiences of food insecurity, suffer anxiety and concern about adequate budget and/or supply of food, changes in diet quality, reduction in food intake and hunger⁽⁸⁾.

The prevalence of food insecurity in high-income countries measured by different validated multi-item tools was found to be 12 % in Canada⁽⁹⁾, 8 % in England, Wales and Northern Ireland⁽¹⁰⁾, and 11·8 % in the USA⁽¹¹⁾. In Australia nationally, using a two-part measure, 4 % of people lived in a household that reported to be food-insecure⁽¹²⁾. This measure assesses one aspect of the food security experience and potentially underestimates the prevalence of food insecurity in Australia by approximately 5 %⁽¹³⁾. In Victoria, the second highest populated state of Australia, 4 % of Victorians were food-insecure⁽¹⁴⁾.

Responses to address food insecurity in high-income countries vary from food relief initiatives, community food programmes to social protection policies and programmes. The dominant response is the provision of food relief through the charitable food sector; however, this does not focus on the underlying determinants of food insecurity⁽¹⁵⁾. A particular challenge in food insecurity responses, including policy, is that the determinants are complex and interacting. Solutions neglecting to address these determinants are unlikely to have significant or widespread impact. The complexity derives from the interaction between the multidimensional determinants that include structural and socio-ecological factors, for example, low or unstable income, un- and/or under-employment, health status, stressful life events, household composition, low education, home ownership and geographic (urban v. rural or remote)^(16–18). In addition to these aforementioned factors, the four interacting dimensions underpin food security status that include

• 1 Availability of sufficient nutritionally adequate food supply;

• 2 Adequate financial and physical access to food;

• 3 Resources, infrastructure and ability to utilise food;

• 4 The stability of 1–3 over time⁽¹⁹⁾. However, a primary predictor of food insecurity is inadequate income and other financial resources available for acquiring food.

Health promoters, public health practitioners and policymakers wishing to make evidence-based decisions to ameliorate household food insecurity require suitable information and evidence, to evaluate candidate policies. Using high-quality evidence in a relevant localised context is ideal, but key aspects of the system of determinants may not be amenable to investigation or their acquisition may be prohibitive based on cost, time, effort or ethical considerations. The incomplete nature of the supporting evidence provides a serious challenge and can leave policymakers using heuristics to fill the gaps.

Structured expert judgement (SEJ) elicitation is a robust and defensible method for producing evidence for policymakers. The use of expert advice and opinion to support policy decision-making is commonplace, but generally, the manner in which contributions are synthesised to inform the eventual decision is not transparent. Where informal heuristics and elicitation are employed, experts are subject to a number of well-documented biases: social biases, deferring to the member with the most compelling personality or who is seen as the most senior, bias towards the most readily available information and misunderstandings due to semantic differences^(20,21) . The results are often not reproducible and can be unreliable and heavily biased. These difficulties can be significantly reduced using structured approaches designed to mitigate the most pervasive and debilitating psychological and contextual frailties of expert judgement^(22–25).

Structured elicitation of expert opinion in pursuit of decision support is an increasingly important technique across areas affecting health. Examples include, food and feed safety risk assessment⁽²⁶⁾, assessment of health risks⁽²⁷⁾, and the quantification of uncertainty in the risks of herbicide-tolerant crops⁽²⁸⁾. Validated protocols for SEJ fall into three broad categories based on how they aggregate the individual contributions of experts into a single estimate: behavioural aggregation (seeking consensus); mathematical aggregation (combining individual estimates using a formula) and mixed aggregation. There are several well-established methodologies for structured expert elicitation protocols, each with its own strengths and limitations⁽²⁶⁾. The recently developed Investigate Discuss Estimate Aggregate (IDEA) elicitation protocol⁽²⁹⁾ encourages experts to Investigate and estimate individual first-round responses, Discuss, Estimate second-round responses, following which judgements are combined using mathematical Aggregation. IDEA combines the strengths and ameliorates some limitations of older methods.

In this study, we used IDEA SEJ to provide evidence that was missing on food security determinants for Victoria, Australia, required to develop a food security integrated decision support system (IDSS) similar to the one developed for the United Kingdom⁽³⁰⁾. As part of the development of this, IDSS food security determinants for Victoria were mapped. However, there were no data to quantify households’ expected food security category for three of the key determinants of food insecurity: (1) food cost, (2) household income and (3) physical access.

This paper describes the application of a SEJ elicitation method using the IDEA protocol to derive the unknown quantities of cost of food, household disposable income and physical access. These quantities were required to support the development of a household food security proof-of-concept IDSS for Victoria, Australia, to predict the probability of household food security status to inform policy and intervention.

Methods

In the development of a household food security integrating decision support system, a determinants map was initially devised by the authors, informed by the pre-existing literature⁽³¹⁾ and localised knowledge. The IDEA protocol for SEJ was used to elicit information from local food security experts required to build a food security integrating decision support system⁽³²⁾. The IDEA protocol was previously validated in a forecasting tournament over a 12-month period. In summary, each month a new set of questions was released and experts had approximately 3 weeks to investigate the question, provide first-round individual estimates, view the estimates of other individuals in their group, discuss and provide second-round individual estimates⁽²⁹⁾. The IDEA protocol method is described according to its three stages of elicitation: before, during and after.

Before elicitation

Drafting elicitation and calibration questions

In the pre-elicitation stage, the risk of semantic or other misunderstandings of the areas to be elicited are minimised by careful and precise wording of the questions, which are used to elicit the missing data values, called questions of interest. This stage helps to identify suitable experts to participate in the elicitation itself. During the IDEA elicitation, in addition to the quantities of interest, experts are asked calibration questions. Calibration questions have answers that are unknown to the experts providing the missing quantities, but known to the analysis team. Calibration questions are used to assess individual experts’ ability to estimate probabilities, which is known to be a difficult task⁽²⁶⁾. The accuracy and informativeness of experts’ responses to the calibration questions are used to calculate performance measures and used as weights for the mathematical aggregation step of the protocol (Table 1). Formulae and details of the hypothesis test for calibration are given in Ref. 33. Calibration questions are close to the subject-matter of the questions of interest as possible because it is expertise in a specific subject area which is important for SEJ.

Table 1 Structured expert judgement elicitation and calibration questions

* Calibration questions.

Drawn from the determinants map the questions of interest pertained to the relation of the determinants of food cost, household disposable income and physical access prediction of household food insecurity (Fig. 1).

Fig. 1 Part of the integrating decision support model for household food security in Victoria, Australia

Expert identification

Sixteen academics and health promotion/public health professionals from Victoria identified through snowball sampling as experts in the areas of food insecurity determinants were invited to participate in the elicitation process via personal email sent by an independent researcher. These experts were specifically involved in knowledge and/or response generation related to specific determinants of food insecurity and/or population groups experiencing food insecurity across the health promotion continuum and sectors. Identified experts included epidemiologists, social service sector policymakers or health promotion workers (youth, housing disability and employment, energy sector, work and economic security), social planners, practitioners with specific portfolios in indigenous and asylum-seeker services, peak bodies of food and material aid relief. Since SEJ involves the combination of expert judgement, diversity of experts is more important than large numbers. Greater than fifteen experts does not significantly improve the findings, but fewer than five may reduce the chance of providing adequate diversity in opinion⁽³⁴⁾. Nine of these sixteen experts initially agreed to participate in the elicitation process including pre- and post-activity and 1-d face-to-face workshop. A list of supporting and briefing background materials including relevant food security literature identified through a literature search, an outline of the IDEA protocol and the quantities of interest refined to specific questions was provided in advance of the face-to-face session.

Due to unanticipated work commitments, four of the nine experts withdrew at short notice limiting the capacity for further expert recruitment. The final five participating experts were employed in university, local government, and not-for-profit sector with experience in areas of food security determinants, poverty alleviation and food security response planning, implementation and evaluation, and/or front-line service provision.

During elicitation

The 1-d face-to-face expert elicitation process was audio-recorded and transcribed to provide further insight to the processes, framing, discussions and negotiations that experts undertook through the elicitation. These discussions serve to improve the accuracy of experts’ quantitative estimates, including uncertainty bounds, improving the quality of the second-round estimates, which will be used as data in the IDSS model. Testing of the IDEA protocol showed that when this discussion exercise is undertaken on calibration questions, experts tend to move their estimates closer to the true values in this exercise.

Agreement on question meaning

For a successful elicitation, agreement upon clear definitions of the variables to be quantified was required. Initial proposed definitions provided by the researchers and informed by literature were discussed with the expert panel and collaboratively refined as required to reach consensus.

The validated and widely implemented United States Department of Agriculture Household Food Security Survey Module 18 item (USDA HFSSM) was used to determine food security status and severity level⁽⁸⁾.The food security quantities of interest were the proportions of people expected to fall into the four food security severity categories defined in the USDA HFSSM, under each combination of the levels of the three determinants (income, food cost and physical access), making forty-eight questions of interest in all. Severity of household food security was defined according to the USDA HFSSM criteria of number of indications (affirmative responses) to survey items⁽⁸⁾:

• High food security: no reported indications of food access problems or limitations.

• Marginal food security: one or two reported indications – typically of anxiety over food sufficiency or shortage of food in the house. Little or no indication of changes in diets or food intake.

• Low food security: reports of reduced quality, variety or desirability of diet. Little or no indication of reduced food intake.

• Very low food security: reports of multiple indications of disrupted eating patterns and reduced food intake.

To avoid over-burdening the experts participating in the structured elicitation, the number of categories in the income, food cost and physical access variables were restricted: equivalised disposable income was categorised as high, moderate and low; food access (cost of food) was categorised as good (low cost) or poor (high cost), and physical access was categorised as good (nearby) or poor (large distances).

Equivalised disposable income per household was defined as total household income divided by the sum of weightings for household members. The Australian Bureau of Statistics (ABS) uses the Organisation for Economic Co-operation Development (OECD)-modified equivalence scale which assigns a weighting value of 1 to the household head, 0·5 to each additional person 15 years or older, and 0·3 to each child under 15 years⁽³⁵⁾. The ABS mean weekly equivalised disposable household income quintile data for Victoria were used to inform income categories⁽³⁵⁾. Weekly equivalised disposable income was categorised as high greater than AUD$1152 (above 4th Quintile); moderate between AUD$859 and AUD$1152(3rd and 4th Quintile); and low below AUD$859 (below 3rd Quintile).

Food cost, influencing the availability of food, was determined using the validated Victorian Healthy Food Basket data collected across the state⁽³⁶⁾. For a ‘typical family of 4’ (44-year-old male and female, 18-year-old female and 8-year-old male), the highest cost basket is AUD$561·08, moderate (median) cost basket is AUD$422·50 and lowest cost basket is AUD$359·48 per fortnight. Therefore, food availability was good if a healthy basket for a ‘typical family of 4’ costs AUD$422·20 (median) or less and poor if a healthy basket for a ‘typical family of 4’ costs more than AUD$422·20.

Physical access categories were informed by the 20-min neighbourhood project⁽³⁷⁾. Physical activity was categorised as good if the nearest supermarket or strip shopping was either: within 500 m walking distance, within 20 min on available private transport, within 500 m walking distance of access to public transport (bus stop, station) and shopping within 30 min journey time by shortest available route. Physical activity was categorised as poor if the nearest supermarket or strip shopping is further than these distances.

The calibration exercise

The calibration questions were derived from household food security survey data used to validate an Australian food security tool⁽³⁸⁾ and compare its performance with the USDA HFSSM questions⁽⁸⁾. This data had not yet been made public. The calibration questions were treated in the same way as the questions of interest, and in this exercise, were mixed among the questions of interest in a single questionnaire. Using standard measures of performance according to Hanea (2017), the second-round answers to the calibration questions were analysed⁽²⁹⁾.

Expert estimates

Following agreement on question understanding for the questions of interest, the experts each provided private, individual first-round estimates for the lowest plausible, highest plausible and best estimate for the natural frequencies of how many people out of 100 would be in each food security category given various levels and combinations of income, food access and availability (Table 1). The first estimation was undertaken privately to encourage independent thinking and avoid anchoring on other experts’ estimates. The experts’ estimates to the scenario questions were plotted on graphs in anonymised form for the subsequent discussion phase.

Experts then discussed how they had arrived at their estimates and reasons for the width of the interval between their lowest and highest plausible estimates, using the anonymised graphs. In particular, it was important for the facilitators to understand whether a wide interval was indicative of the expert’s perceived uncertainty in the system or related to the question or a reflection of their own uncertainty. Following discussion, the experts each gave private, individual second-round responses in line with the IDEA protocol.

After elicitation

After the workshop, the experts’ first- and second-round estimates were compared. The first-round estimates form the basis of discussion, sharing information and challenging perspectives which improve estimates in the second round.

The second-round responses to the questions of interest were aggregated in line with the IDEA protocol: using a performance-weighted average, the aggregated lowest plausible, highest plausible and best estimates for the probability of categories of household food security status were calculated from the second-round estimates.

Those experts who gave the most accurate and informative answers on the calibration questions are given most weight in the aggregation of questions of interest.

Results

Estimates of proportions of households falling into each of four categories of household food security under a combination of key variables were obtained from five experts participating in the SEJ elicitation and aggregated based on performance on calibration questions. In each case, the lowest plausible, highest plausible and best estimate of the proportion were obtained and treated as 5th and 95th percentiles and median of a probability distribution, respectively.

The comparison of experts’ first- and second-round estimates across the questions indicated that whilst some responses were unchanged, others changed considerably, showing that the discussion brought new perspectives to the table. As example, Figure 2 describes the expert estimates with equal weight and performance weight combinations for comparison with two questions of interest. Question 7(Q7) ‘Out of 100 people with moderate equivalised disposable income and good physical access when food availability is good, how many will be food-secure?’ While Question 16(Q16) asks ‘Out of 100 people with low equivalised disposable income and good physical access when food availability is good, how many will be very low food-secure?’ The solid lines in Fig. 2 indicate the experts estimates of ranges of lowest plausible, highest plausible and best estimate (median). The broken lines depict each expert’s private second-round estimate after discussion. The top two lines are the equal-weighted and (top) performance-weighted combinations of all judgements. Expert estimates to all calibration questions and questions of interest are available in Supplemental Files 1 and 2.

Fig. 2 Expert estimates with equal weight and performance weight combinations for comparison

Of particular note, many of the experts reduced the interval between highest plausible and lowest plausible values in their second-round estimates, suggesting that they were more certain about the interval within which a good estimate should lie following the discussion.

Figure 2 illustrates that in Q7, there was a varied change in response between experts’ initial and second estimates. Expert 1 changed their estimate considerably following discussion on Q7 and increased their uncertainty. In contrast, Expert 4 reduced their uncertainty, and Experts 2, 3 and 5 had not changed their estimates following discussion. In Q16, the difference between round 1 and round 2 estimates was small, except in one case but despite this the plausible intervals overlapped. In Q16, Expert 5 had a very different estimate from the other four, and this supports a strongly different pooled estimate between the equal weight and performance weight combinations. The graphs for all the questions of interest and the calibration questions are included in Supplemental Files 1 and 2.

The discussions undertaken by the experts during the elicitation process provided opportunity for debate, clarification, agreement and reasoning of the questions of interest or calibration questions being considered and contributed to the outcome in the scenarios impacting on food security status. Careful inspection of the discussion showed the diversity of sector experience in these experts provided a range of viewpoints. As example of the expert discussion in the scenario Q7 good physical access, food availability (cost) and moderate equivalised income:

‘Expert 5: So we’re almost covering the whole scale

Expert 3: I used the same reasoning, where I put my majority on as moderate. So moderate income will be moderately FS, so I put that as my highest one out of 100

Expert 2: I would be very surprised if there were more than 50 % in this situation.

Expert 3: Perhaps I should increase my upper limit

Expert 1: This is interesting. Using the healthy food access basket, 25 % of your income goes on food with a moderate income of $859 …… that’s a fair whack of income going towards that healthy food basket. ….So maybe they’re spending less on healthy food and they were at that higher end of that income bracket. So I thought a lot of them would be FS, more than 50 % of them would be FS. But maybe even more.

Expert 2: But then isn’t income a really important determinant of FS? If that’s not high, it’s moderate?

Expert 3: For me, income is the biggest predictor. So if you’ve got moderate compared with high, then you have more of a chance at being moderately FS than being food-insecure with a moderate income…….

Facilitator: Because you think a significant influence on FS is income? The two other variables?

Expert 3: I always put income first. That’s what the evidence shows, isn’t it?

Expert 4: I do like your rationale (Expert 3). …. looking at the definition of moderate FS and how there’s actually no change to the diet or food intake but it can take into account that anxiety around stress, so someone could be eating a good diet and not change, but just have constant underlying worry…I won’t change mine’

This sample highlights the robust discussion that occurred across the experts, with diverse fields of expertise, when discussing the variations in the conditions in the questions of interest. It also illustrates the variety of perspectives within the expert panel.

The results of the calibration exercise found in Supplemental File 3: Supplemental Table 1 showed that performance-weighted aggregation was preferred to equal-weighted aggregation, although both are shown for comparison.

Table 2 highlights the sensitivity of household food security to all three factors. The probability that a household is food-secure, given good physical access to food, good availability of food (prices average) and a high equivalised income is predictably high, at median 88 % and between 80 and 96 %. Some food insecurity can occur even in these optimal circumstances because asset:debt ratio can be very high, meaning that the high income is spent on servicing debt. Similarly, given the worst possible scenario of poor physical access, poor availability (prices high) and low income, the most probable outcome is low food security (median 43 % and between 17 and 56 %). In this scenario, the probability that a household is food-secure is low (median 15% and between 7 and 25 %).

Table 2 Probability of food security status expressed as a percentage according to varied scenarios of physical access, food availability and equivalised income

* Medians were asked for, these did not necessarily sum to 100, so were normalised and the lowest and highest plausible values were normalised using the same factor.

In the scenario of low equivalised income, good physical access and food availability, Questions 13–16 (Supplemental table) the probability of high food security was 22·1 %, moderate 28·9 %, low 40·9 % and very low 8 % (Table 2). The probability of high food security varies dramatically between high and low income when the other two factors are the same, regardless of the levels of them. In contrast, food availability and physical access each have smaller effects on food security status for the same income levels.

Discussion

This paper reports the use of SEJ, specifically the IDEA protocol as a method to quantify unknown aspects of data when developing a proof-of-concept IDSS to describe food insecurity in Victoria, Australia. These data can be added to the model of complex, interacting determinants of food security to provide decision support to intervention (inclusive of policy) decision-makers. Using this approach, the probability of household food security status under various combinations of the determinants disposable household income, physical access and food availability (cost) could be predicted, overcoming the prohibitive difficulties of obtaining these by designed experiment or other data sources. SEJ provides a way to estimate these quantities in a transparent and defensible manner. In the elicitation of quantities from experts, the differences in expertise between acknowledged specialists can be properly and robustly addressed and reduced by the careful use of facilitated discussion, avoiding the severe problems associated with unstructured elicitation.

Consistent with being a primary determinant of food security status, the quantities provided by the experts supports that income, in this case disposable income, has a predominate effect on food security status. Hence, it is important for intervention decision-makers to understand the demands on household financial resources including cost-of-living pressures, housing, utilities, cost of food^(39–41) and additional costs such as health conditions and medical expenses⁽⁴²⁾. Subsequently, interventions targeting increasing disposable household income and determinants that impact on this are likely to be supportive of higher levels of food security⁽¹⁵⁾. In addition, the information gathered through the elicitation allows for intervention decision-makers to determine likely effect of policies on food security under the effects of uncontrollable determinants, such as cost of food. For example, estimating the effect of economic or environmental policies on food availability (cost) can estimate the subsequent impact on household food security^(32,43) .

Eliciting best-estimate values for the probability of each food security outcome and also the lowest and highest plausible values (Table 2) provides further evidence of the varying impact of determinants. Whilst the effects of physical access and food availability on food security status are significant, household disposable income is by far the strongest determinant. This helps intervention decision-makers to anticipate the effect of different candidate responses at their disposal. For example, policies and or interventions to improve physical access, such as transportation and walkability alone, are unlikely to have a strong effect on household food security; however, they may have additional health-promoting benefits, for example, reducing social isolation. Similarly, improving access to food access through a range of interventions such as food bank/pantry, food literacy or community food programmes such as growing schemes will have limited improvement in ongoing household food security^(15,44) .

In order to quantify the likely effects of changes in the levels of determinants, perhaps via interventions, on household food security in a range of contexts health promotion, public health practitioners and policymakers may use the quantities derived through elicitation in combination with known data of other key determinants. This provides opportunity to test policy interventions alone and/or in combination for the likely impact on household food security status to inform policy choice or potential interventions.

A strength of this research is the use of an established and validated method to derive quantities of interest related to food security that may support health promotion and public health workforce in decision-making on how best to address this issue within a range of geographic and/or demographic contexts. The provision of the likely effect of combinations of determinants on household food security inclusive of those explored in this proof-of-concept decision support system is of importance to support policy design. In particular, these findings can be used to evaluate potential policies to support decision-makers in making evidence-based choices. The availability of calibration questions very close to the questions of interest is a strength of the study, providing high confidence that the weighting of the experts is appropriate. The experts who contributed to the elicitation process and provided estimates are experienced in a range of aspects food security, providing confidence that these estimates are likely to be reliable given the current state of knowledge.

The limitations of the study include the discretisation of the continuous variables. In order to undertake the elicitation in the available time, each of the three areas of interest for elicitation were reduced to two-three discrete categories. Ideally, a more nuanced categorisation would be preferred. However, more levels per variable lead to a rapid rise in the number of conditional probabilities to be elicited, hence in an increased elicitation burden. Finally, a number of experts who had committed to taking part in the elicitation process were unavailable at very short notice, reducing the number and potentially the breadth of expertise contributing to the elicitation. However, diversity of experts is more important than large numbers for SEJ. Greater than fifteen experts does not significantly improve the findings, but fewer than five may reduce the chance of providing adequate diversity in opinion⁽³⁴⁾. Nevertheless, a strength of the IDEA protocol is the discussion between experts allows verification that a broad range of views and considerations were represented when undertaking this elicitation. The example of this discussion highlights the complexity of food security and the importance of decision-makers to have an awareness and understanding of the range and interaction between determinants and their impact. The variety of opinions between experts (such as in Q16) is dealt with through the aggregation process, combining experts using their performance on the calibration questions. It is also possible to undertake sensitivity analysis, by including and excluding subsets of experts. Whilst a decision-maker may wish to re-run the elicitation with a larger group before using the full model for decision support, we are satisfied that the diversity in the experts’ backgrounds, experience and perspective is rich enough for a proof of concept.

Based on new methodology for coherent inference in networked systems⁽³²⁾, future work will include the incorporation of these values with other evidence on major determinants of household food security. This will provide a proof-of-concept IDSS, which would demonstrate how to support policymakers to evaluate the effect on household food security of plausible scenarios. Doubtless, were policymakers to adopt this methodology, they would wish to scrutinise the model itself and re-run the structured expert elicited judgement with the number and variety of experts that would be satisfactory to them.

Conclusion

This research provides insight to the application of a novel approach for food security studies in high-income countries, SEJ. This provides useful, transparent and defensible evidence, when evidence-based decision-making is required particularly in context of complex issues such as household food security. In order to address the issue of household food insecurity beyond current responses, interventions must consider the range of varied and interacting determinants. In the instance where evidence is absent or of poor quality, public intervention decision-makers may consider using structured expert elicitation as a method of evidence generation to inform intervention decisions.