Frequently
Asked Questions
Frequently Asked Questions
What is evaluation
about and why bother?
Evaluation is a structured process of review which compares what
actually happened against what was intended to happen. Evaluation
assesses the value or worth of a service or activity and helps when
deciding what needs to be changed or further developed for maximum
impact. Evaluation looks at the overall plan of the organisation,
particular projects within the organisation, specific issues within
the project and the structure of the organisation.
Evaluation provides the following benefits:
• Improves practice;
• Aids decision making and policy formulation;
• Helps with planning;
• Argues the case for continued/additional funding;
• Increases accountability and transparency;
• Shares experiences and lessons allowing for continuous improvement;
and
• Builds skills level.
When should I
plan for evaluation?
Evaluation should be planned for before an activity or service begins
and should be actively implemented throughout the life of the activity.
It is recommended that evaluation takes place on a formative basis
(on-going throughout the activity), at key stages or milestones
within the activity and on a summative basis (towards the end of
the activity)
What's the difference
between evaluation and monitoring?
Monitoring feeds into evaluation through its systematic recording
of activities and through the documentation and gathering of information.
It is however a value-free process, an essential difference between
it and evaluation.
What is self-evaluation?
Self evaluation is a form of evaluation whereby the organisation
itself seeks to assess and understand the value of its work. The
use of self-evaluation techniques allows those involved (managers,
staff and users) to reflect on practice and improve effectiveness.
(CES 1988).
Self evaluation has 5 distinctive features:
• It is initiated from the beginning of the activity;
• It is an ongoing process of review – responding to
change;
• It provides a greater sense of ownership of the evaluation
process;
• It is an empowering process – developing skills and
reflective learning; and
• It involves stakeholders – providing enhanced participation
and feedback.
Where
does self-evaluation fit with external evaluation?
Whilst self evaluation can be described as an ongoing cycle of internal
self reflection, external evaluation usually only takes place at
key stages (the end) and measures the overall impact of the activity
whilst assessing strengths and weaknesses. Effective external evaluation
should be carried out by individuals with a good understanding of
the methodology and subject.
What's
involved in implementing self-evaluation?
Self evaluation requires planning and commitment. A self evaluation
plan should contain the following 4 elements:
• Clarify anticipated outcomes/changes;
• Agree activities
• Establish indicators of success; and
• Clarify and plan information needs & methods.
How
do I involve different stakeholders in our evaluation?
A useful way of involving all the different stakeholders is to create
an evaluation sub-group which is responsible for co-ordinating and
overseeing the evaluation process. The sub-group would meet at key
stages to review progress, receive and comment on reports and be
representative of key stakeholder interests.
What
are outcomes?
Outcomes are changes that result from a given activity or service.
These can be changes in individuals, organizations, communities,
services, policies etc and take place over the immediate, medium
or longer term. Measurement of outcomes is both quantitative and
qualitative.
Why
are indicators important?
Indicators are important because they allow for baselining and benchmarking
and encourage people to focus on results. Indicators are a yardstick
or standard against which achievement can be measured. Indicators
should be specific, measurable and realistic – both qualitative
and quantitative – and agreed following negotiations with
stakeholders.
What
kind of evidence do I need to be collecting?
Evidence should be both qualitative and quantitative. Examples could
be:
Quantitative
• No./type of users
• No./type of activities
• Attendance figures
• No. of press releases
• No. received certificates
• Progression
Qualitative
• Benefits gained
• Feelings re health and wellbeing
• Ability to cope with symptoms
• Increased knowledge
• Confidence/self esteem
What
methods should I use to gather information about my project?
Some suggestions:
Reading – written reports, minutes, staff diaries, project
journals
Recording – internal records, personal records, case studies,
comments book.
Observing/Listening – events, activities, meetings, photographs.
Questioning – individual/group, interviews, discussions, questionnaires
How
do I analyse or make sense of all this information?
Information analysis should be carried out in 2 stages – firstly,
the information should be collated and organised, then it should
be used to evaluate through interpretation, judgement, conclusions
and recommendations.
Any
tips on report writing?
The report should be well presented and structured. Bear in mind
the audience for the report and have someone proofread the final
document. A good report should follow the structure below:
• Title page
• Contents page
• Summary
• Background information on the project
• Information on the evaluation process
• Findings
• Analysis and conclusions
• Recommendations
• Appendices
How
much does an external evaluation cost?
The cost of a given external evaluation varies depending on the
size and complexity of the project to be evaluated, the detail held
within the Terms of Reference and the person or group selected to
carry out the evaluation.
How
long does an external evaluation take?
As with the previous question, this can vary for the reasons outlined
What
does an external evaluation involve?
External evaluation involves:
• Reviewing documentation
• Consultation with stakeholders
• Collating and analyzing findings
• Reporting
• Incorporating results into future planning
Any
tips on putting together terms of reference?
Terms of reference should cover the overall aim and purpose of the
evaluation and the specific issues you wish to be addressed.
What's
involved in commissioning an evaluator?
The following steps are a useful guide to commissioning an evaluator:
• Agree method of recruiting;
• Agree method of application;
• Develop selection criteria;
• Issue commissioning document;
• Review applications against criteria;
• Identify appropriate evaluator; and
• Appoint evaluator.
What
should a good evaluation report include?
A good evaluation report should be well laid out and meet the agreed
terms of reference.
Are
there other sources of support that can help?
CENI offers an information
service and access to its information library for interested individuals
and groups. CENI also produces a variety of publications and interested
parties can sign on to the mailing list and indeed become members
of ceni if they so wish.
|
