Research has shown that evaluations of some of the most widely advocated school-based alcohol and drug prevention programs employ questionable practices in their data analysis and presentation such as selective reporting among numerous outcome variables, changes in measurement scales before analysis, multiple subgroup analysis, and selective use of alpha levels above 0.05 and one-tailed significance tests (Brown, 2001; Gerstein & Green, 1993; Gorman, 1998; 2002a; 2002b; 2003; in press-a; in-press-b; Manski, Pepper & Petrie, 2001; Moskowitz, 1989; 1993). This has resulted in an overestimation the effects that these programs have in terms of preventing alcohol and drug use among children and youth. However, it is unclear just how widespread the use of such practices is within the overall field of alcohol and drug prevention since the focus of the existing critiques has been on a fairly narrow range of school-based programs. Very little attention has been focused on community and family-based interventions, and only the most prominent school-based curricula have been examined in any detail. The project described herein is therefore intended to assess the quality of the research base of alcohol and drug prevention programs in a more systematic and comprehensive manner than has been conducted to date. Accordingly, it will address the following specific questions: i. What is the prevalence in the published drug and alcohol prevention research literature of irregular and questionable data management and analysis practices? ii. Does the use of such irregular and questionable data management and analysis practices result in significant overestimation of program effects? iii. What proportion of alcohol and drug prevention programs is independently evaluated (that is, evaluated by researchers other than those who developed the program)? iv. Do evaluations that are conducted independently employ better data management and analysis practices? These questions will be addressed through the following two procedures: First, a systematic review of the data management and analysis practices used in the evaluations of the 54 programs t hat appear on the Center for Substance Abuse Prevention's list of model substance abuse programs; Second, reanalysis of data from an independent Drug Abuse Resistance Education (DARE) evaluation that produced predominantly null results.
| Gorman, Dennis M; Conde, Eugenia (2010) The Making of Evidence-based Practice: The Case of Project ALERT. Child Youth Serv Rev 32:214-222 |
| Gorman, Dennis M; Conde, Eugenia (2007) Conflict of interest in the evaluation and dissemination of ""model"" school-based drug and violence prevention programs. Eval Program Plann 30:422-9 |
