Conduct Impact Evaluations

Impact evaluations seek to determine whether a behavior or policy change has affected the overall extent or severity of the problem being addressed. For example, your environmental approach to underage drinking may have been designed to pass a new community law or regulation that strengthens enforcement of underage drinking. And, if you were able to get the law passed, you can state that you reached your goal and you had a positive outcome: You got the law passed. An impact evaluation then goes a step further. It seeks to determine whether passing the law and the subsequent greater enforcement of underage drinking laws results in a decrease in underage drinking. Also, impact evaluations can be used to determine whether the prevention effort was ultimately in line with the program’s mission statement.

Practitioners typically avoid conducting impact evaluations for two reasons. First, since change may be slow or difficult to sustain, impact evaluations often require measurement over an extended period. Second, it is often difficult to isolate the effects of a prevention project from those of other strategies used to influence a target audience or community. Thus, it is difficult to attribute change in the general population to a specific intervention.

If an impact evaluation is important to a practitioner’s ongoing substance abuse prevention efforts, collaborative efforts are especially important. Since impact evaluations can be expensive and complex, requiring a long-range commitment to the intervention, the best chance for a useful impact evaluation is through a combination of resources. Ideally, the collaborators can agree on one or two areas of impact to measure and sustain a quality evaluation effort over a sufficient period of time to document results. The contributing parties may include community-based organizations, local government, and interested researchers.

The Resource Library provides several practical tools to describe different types of impact evaluations appropriate for environmental approaches and tips for conducting them.