Policy evaluation

Introduction

Without evaluation, governments cannot know whether their money is well spent. Evaluation closes the policy cycle by feeding lessons back into new decisions.

Types of evaluation

• Formative evaluation: during implementation, to improve the programme.
• Summative evaluation: after completion, to judge overall success.
• Process evaluation: was the programme implemented as planned?
• Outcome evaluation: did it achieve its intended results?
• Impact evaluation: what difference did it make compared with what would have happened without it (the counterfactual)?
• Cost-benefit and cost-effectiveness evaluation.
• Ex-ante (before) and ex-post (after) evaluation.

Methods

• Randomized controlled trials, popularized in development by Abhijit Banerjee, Esther Duflo and Michael Kremer (Nobel 2019).
• Before-and-after comparisons with baseline data.
• Surveys, interviews and focus groups.
• Logical framework (logframe) with indicators for inputs, outputs, outcomes and impact.
• Audits and citizen feedback.

Evaluation in Pakistan

• The Planning Commission's Projects Wing monitors and evaluates projects using PC-III, PC-IV and PC-V forms.
• The Auditor-General conducts performance audits.
• BISP and other programmes have commissioned independent impact evaluations.
• Weaknesses: limited capacity, focus on spending rather than results, and reports rarely made public.

Examples of success

• Lady Health Workers Programme: improved immunization and family health in rural areas.
• NADRA: a national identity database used for elections, welfare and services.
• Benazir Income Support Programme: targeted cash transfers to poor women; the Ehsaas emergency cash programme reached millions during COVID-19.
• Rescue 1122: a modern emergency service that began in Punjab and spread to other provinces.

Examples of failure or difficulty

• Persistent losses of state-owned enterprises such as Pakistan Steel Mills and power distribution companies.
• Power sector circular debt.
• Repeated education policies that did not achieve universal enrolment.
• Many development projects delayed with large cost overruns.

Why governments succeed or fail

• Success: clear goals, political commitment, adequate funding, capable leadership, technology and continuity.
• Failure: poor design, weak implementation, political interference, lack of evaluation, and frequent policy changes.

Conclusion

Evaluation turns experience into knowledge. Building a culture of honest, independent and published evaluation would help Pakistan expand what works and stop what does not.