◘ Post implementation simple observation evaluation ◘ Pre-/post-evaluation comparison ◘ Evaluation based on non-corresponding control group comparison ◘ Evaluation based on equivalent control group comparison (genuine experimenting)
Post implementation simple observation evaluation
This method is the easiest to apply and appropriate for evaluating means and performance.
It seeks:
However, this method runs the great risk of subjectivity as it is difficult to isolate policy impact form other possible growth factors. It may be difficult to use in evaluating impact or providing explanations to the findings.. A cause and effect link may be established, but findings made at this level are shaky and replication in other contexts is restricted.
Techniques used
Consultation of files, direct observation, expert opinion, case studies,
statistical surveys, data analysis, ratio calculations, standard comparison, etc.
Pre-/post-evaluation comparison
This method is very common. It is a standard evaluation method that attempts to ensure consistency between the policy implementation period and adjustment of some indicators. It gives a more detailed and quantified description of the impact.
However, this method has the same shortcomings as the above method although it is possible to use more specific indicators and to draw many commonalities to test the accuracy of findings.
Techniques used
Prerequisites: sound description of the initial situation for all project outcome indicators.
Consultation of files, direct observation, expert opinion, case studies, statistical surveys, data analysis, time series analysis, ratio calculations, standard comparison.
Evaluation based on non-corresponding control group comparison
This method compares a specific policy target group with a control group with similar characteristics to the target group. It helps in properly gauging the impact or external impact of a policy (without establishing the cause and effect relation) and establishing mechanisms and behaviours resulting from incentive policies.
This method does not appropriately and clearly establish the causal relation. An increasing number of control groups ascertains the findings. When a group is set up for the specific purpose of evaluating rather than comparing the use of complex statistical techniques to reduce bias accruing from lack of equivalence (in the case of virtual experiment).
Techniques used
Case studies, statistical surveys, data analysis, time series analysis, multiple analysis, modelling.
Evaluation based on equivalent control group comparison (genuine experimenting)
It is the only completely stringent evaluation method. It unequivocally identifies causal relations, hence, the real impact of a policy or project.
The principle of this method involves choosing randomly beneficiaries of a policy or project from possible beneficiaries. This random selection automatically generates a target group or control group of equal statistical value inasmuch as the sample sizes are appropriate. The programme impact is thereafter measured simply by the difference of average target group and control group sizes.
While this approach is often considered as optimal, it raises a number of practical problems. It may be difficult from an ethical or policy point of view to support one group rather than another for the specific purpose of impact assessment study.
Techniques used
Statistical surveys, data analysis, time series analysis, multiple analysis, modelling.
◘ Post implementation simple observation evaluation ◘ Pre-/post-evaluation comparison ◘ Evaluation based on non-corresponding control group comparison ◘ Evaluation based on equivalent control group comparison (genuine experimenting)