8 - Evaluating impact

Summary

Aim

This chapter looks at how to design evaluation that is effective at measuring impact and outcomes, and can inform the commissioning cycle process. The types of evaluation and how to feed evaluation findings into the commissioning cycle are discussed. As not all evaluations show good outcomes, what to do with poor performance or outcomes is considered.

What are we measuring?

To ensure they have achieved what they set out to do, commissioners must evaluate the outcomes and outputs of a service. Outcomes are the benefits realised by an action or series of actions. Measuring outcomes ensures real results are being achieved for people and that what is being done is working. Outputs are the activities and steps taken. Measuring outputs allows commissioners and providers to demonstrate that the activities put into place can be attributed to the successful outcomes. Outcomes are the ‘what’ you want to achieve, and the outputs are the ‘how’.

We measure and evaluate the success of services in order to:

• • ensure delivery of plans;

• • confirm if the service is a success or failure:

  • o if a success, evaluation can inform spread and scale;

  • o if a failure, there is a need to correct or replace;

  • o for either success or failure, there is value in sharing the learning;

• • demonstrate results and win stakeholder support and confidence;

• • be accountable to regulators;

• • demonstrate value with taxpayer money.

Good evaluation will examine not just operationality and ability for churning out activity, but a range of factors that indicate true success. Effective evaluation measures consider whether the activity is:

• • delivering any benefits;

• • value for money;

• • safe;

• • accessible to all who need it;

• • delivering a good experience for people;

• • supporting and enabling a confident workforce;

• • sustainable;

• • creating unintended consequences;

• • meeting national standards;

• • offering benefits with societal value.

Outcome measures are now rightly considered the epitome of evaluation for health and wellbeing initiatives. When we refer to outcomes, we are including a range of factors. As seen in the earlier list, success is wider than an outcome for an individual, important as that may be.