In this article, adapted from our 12 Steps to Effective Impact Measurement guide, we consider two examples of how different evaluation approaches can influence your monitoring, evaluation and learning activities. The approach you choose will depend on your objectives, learning questions, and the context of the activities you are evaluating.

In our video on impact evaluations we looked at a ‘counterfactual approach’, which estimates what would have happened without the programme or policy. As we described, n experimental design, also known as a randomised controlled trial (RCT), randomly assigns participants to either a group receiving the intervention (treatment group) or a control group that does not, comparing outcomes between them. When applied to mature activities with a clear theory of change and measurable impacts, this approach can provide robust evidence. However, it requires careful planning for data collection and analysis, including a large sample size, experienced staff, strict ethical procedures, and expertise in quantitative methods.

On the other hand, developmental evaluation is used in situations where services, products or activities are in their early stages, are innovative, or are operating in complex, uncertain environments that require ongoing adaptation. Unlike traditional evaluation methods that assess outcomes against predefined goals, developmental evaluation supports the development of a product or service by providing real-time feedback and learning to help refine and evolve. This approach impacts data collection, analysis, and reporting by emphasising continuous, flexible, and iterative processes. Data is collected regularly to capture emerging patterns, challenges, and opportunities, and analysis focuses on understanding the dynamic and complex context in which the programme operates. Reporting is typically more frequent and formative, providing stakeholders with timely insights to make adjustments and inform decision making.

For example, you could apply a developmental evaluation approach to digital products and services. Often developed in an agile way, these projects can require a flexible evaluation approach. You can find out more in this guide from Catalyst.