If you are looking for some help with evaluation, you may find it useful to look at our  publication, ‘Prove & Improve – A Self-Evaluation Resource for Voluntary and Community Organisations’.  Funded by the Big Lottery Fund, this guide was developed as part of our Evaluation Training and Development Project, and is designed to help organisations through the process of self-evaluation, breaking it down into a set of logical steps and stages.  It particularly focuses on identifying and measuring outcomes – the benefits or changes which a project brings about for its users.

Contact us to request a copy or download it here.  Alternatively, see the Frequently Asked Questions below, for some guidance.

• What is evaluation and how does it differ from monitoring?

  Monitoring involves on-going recording and gathering evidence on activities in a systematic way.  It may include recording the type of activities delivered, the numbers attending, or feedback from users.

  Evaluation goes beyond collecting evidence to a more structured process of review. It involves making a judgement, comparing what was achieved with what was originally planned, exploring the reasons why things happened as they did.  It usually includes making recommendations for future development or change.  Monitoring information can feed into the evaluation process and inform these assessments.

• What are the benefits of evaluation?

  Evaluation helps to:

  • Document and record what is being done and how, who is benefiting and in what ways
  • Improve practice as you reflect on what works or doesn’t work so well
  • Plan more effectively as you use the learning to inform future development of activities
  • Enhance accountability to funders and other stakeholders
  • Support sustainability as you can demonstrate and evidence achievements
  • Share learning about experiences, both internally and externally

  
  
  
• What questions do I need to think about before starting?

  You need to think about:

  • WHY you are going to evaluate – the main purpose of the evaluation, and how it is going to be used;
  • WHO the evaluation is for – the audience - including funders and other stakeholders;
  • WHAT your evaluation will focus on – the key issues or questions you want it to address.

  
  
• When should evaluation be done?

  Evaluation is often left until the end of the life of a project; however, it is important to think about it from the beginning, so that relevant information can be collected throughout the lifetime of a project.  You might want to carry out evaluation at key points in time – for example, at an interim or midway stage – to allow you to track progress and make any changes needed to bring the project back on track. 

• What is self-evaluation?

  Self-evaluation is when the organisation or project seeks to understand and assess the value of its work, using its own staff and skills. It involves planning for evaluation from the outset, trying to anticipate what information will be needed, putting in place the mechanisms for collecting data and then analysing and using this to help with future development.

• What is external evaluation?

  This involves commissioning support from an external source to conduct an evaluation.  This can provide an objective assessment of your project by a consultant who has the knowledge and skills to carry out the necessary tasks.   

• How much does an external evaluation cost?

  The cost varies, depending on the size and complexity of the project or organisation to be evaluated, the scope of the evaluation (the Terms of Reference) and the fees of the person or group selected to carry out the evaluation.

• Which should I use – self-evaluation or external evaluation?

  There are pros and cons to both approaches, and organisations often combine elements of self-evaluation with external evaluation support to lend some objectivity to the process.  You will need to decide on the best approach for your project, and consider the implications – including costs and time – of this decision.

• What’s involved in implementing self-evaluation?

  Self-evaluation involves a series of logical steps and stages:

  • Planning
  • Collecting evidence
  • Analysing information
  • Reporting and using

  
  
• What does planning for self-evaluation mean?

  Planning well, before you begin to gather information is an essential part of self-evaluation.  This helps to establish clear boundaries for the self-evaluation process, and agree on what needs to be measured and how.  You will find it useful to develop a self-evaluation plan which covers four key areas.

  • Identify anticipated achievements – what is your project trying to achieve?
  • Establish indicators – how will you measure success?
  • Identify evidence required – what evidence do you need to collect?
  • Agree data collection methods – how will you collect the evidence?

  
  
• What are outcomes?

  Outcomes are the benefits or changes which you envisage as a result of your work.  They may be expressed in positive terms - ‘to increase knowledge’ – or negative terms - ‘to reduce incidents of crime’.  They may also be about maintenance -  ‘to maintain positive community relationships’ or prevention - ‘to prevent accidents in the home’.

• What are ‘anticipated’ and ‘unanticipated’ outcomes?

  Anticipated outcomes are planned from the beginning; you define the outcomes you expect from your project at the start, based on evidence of needs amongst your target group. You may find that your work has unanticipated outcomes – either positive or negative – which emerge as different factors impact upon the project as it progresses from its initial planning stage.

• What are ‘soft’ and ‘hard’ outcomes?

  Hard outcomes are quantifiable, and therefore considered to be more objective; for example, homeless people are housed, unemployed people gain employment. 
  
  Soft outcomes are often less tangible, and may include things like increased confidence, improved self-esteem or changed attitudes.  While they rely on more subjective evidence – for example, reported feedback from users – and can be seen as difficult to measure, they give a much fuller picture of what is going on behind the numbers, and the successes of the project.

• When do outcomes happen?

  Outcomes happen over a period of time.  Sometimes when you deliver activities there are immediate outcomes; for example, new knowledge and skills gained through training.  You may then wish to track changes further down the line, to check to what extent learning has been implemented and how it has impacted on participants.  Progression from the point of initially engaging with an activity, through to achievement of the desired outcome at the end, is sometimes referred to as ‘intermediate outcomes’ or ‘distance travelled’.  This acknowledges that not all participants start at the same place and that there are steps or milestones along the way which are important to record, even if the final anticipated outcome is not achieved.

• What are the benefits of evaluating outcomes?

  There are a number of benefits to be gained from focusing on outcomes:

  • Increased clarity about the  changes you are trying to achieve

  • Greater understanding about how your activities help to achieve change
  • More effective planning to achieve the desired outcomes
  • Being more client-focused rather than activity-focused
  • Increased motivation for staff as they see evidence of change
  • Potential to demonstrate how your work contributes to meeting funders’ priorities

  
  
• What are indicators?

  When conducting evaluation we need to have some criteria or indicators on which to base judgements about success.  Indicators may be:

  • Quantitative, relating to numbers or amounts
  • Qualitative, relating to reported change or perceptions of change

  
  
• What kinds of evidence are used to measure success?

  Like indicators, evidence can be both quantitative and qualitative.

  Examples of quantitative evidence:

  • Number and type of participants - age, gender, ethnic background
  • Number of activities delivered

  Examples of qualitative evidence:

  • User satisfaction with services provided
  • Participant feedback on benefits they have gained from the services

  
  
• When should I collect evidence?

  Evidence may be gathered at various times but, ideally when measuring outcomes you will need to capture information at the start, during and at the end of the project to show how participants have progressed.  Information gathered at the start provides the baseline position. 

• What methods should I use to collect evidence?

  Evidence may be gathered from a combination of sources:

  • Documents and records eg internal reports, staff diaries, minutes of meetings
  • Questioning, through interviews, focus groups or questionnaires/surveys
  • Observation of events or activities
  • Case studies
  • Creative methods such as video diaries, photography or creative writing

  
  
• How do I analyse information?

  Analysis involves making sense of the information you have collected, and using it to help inform the future development of your work.  You might find it useful to think of analysis as a series of steps, through which you filter and distil the body of evidence you have collected:

  • Reflecting
  • Collating
  • Describing
  • Interpreting
  • Concluding
  • Recommending.

  
  
• How do I present evaluation findings?

  One of the main ways is through a written report; in some cases, this may be published and circulated more widely.   There are other ways in which you can communicate learning with different target audiences -  perhaps by organising an event or writing up key findings in a newsletter or on your website.  You may also want to use more creative approaches – a CD or DVD for example – to disseminate information; these are not always as expensive as you may think.
  

• What should a good evaluation report include?

  A good evaluation report should include the following:

  • Title page
  • List of contents
  • Executive Summary
  • Background information on the project
  • Methodology for the evaluation
  • Findings and Analysis
  • Conclusions and Recommendations

  
  
• What happens when an evaluation is finished?

  Reporting is not the end of the evaluation process.  You need to take time to consider the recommendations which have been made, and set out a plan of action to carry these forward. 
  Remember too, that evaluation is a cycle - you need to keep up the momentum of planning what you want to achieve, measuring success and agreeing improvements or changes,  to ensure that your project continues to be relevant and beneficial to all of your stakeholders.