System Evaluation Theory (A Blueprint for Practitioners Evaluating Complex Interventions Operating and Functioning as Systems)

Too often evaluation findings end up on the shelf! Why? Many interventions have several moving parts working together to achieve something each part cannot do independently. Unfortunately, many of the available evaluation approaches oversimplify this reality. A major reason for this is that evaluators do not have a blueprint to plan, and the associated methods to execute, an evaluation fit for this purpose. As such, they revert to using methods with which they are familiar, but are not suitable, for evaluating complex interventions. Consequently, the evaluation findings do not make sense to decisionmakers, so they are ignored. This book provides evaluators of all skill levels with a simple 3-step framework, System Evaluation Theory, that will lead to more actionable recommendations because the methods better capture the reality in which interventions operate; that is as a system.

The book first reviews the limitations of program evaluation approaches that pointed to a need for a different way of thinking, one grounded in systems. After defining a system, the book explains how SET is intentionally aligned to evaluate a system’s two essential properties: interdependence and emergence. Individual chapters are dedicated to explaining how to execute each SET step. Throughout, the author draws on real-world examples and those from his own evaluations to help bridge the theory-practice divide. The reader is then shown how to use SET to develop actionable recommendations.