The goal of this SBIR is to develop an online tool, Synthetic Controls, to help local evaluators and others charged with demonstrating evidence of effectiveness of drug prevention programs obtain evidence for assessing the effectiveness of their interventions. Synthetic Controls will provide useful analyses and simultaneously relieve local evaluators of the burden of having to identify, convene, maintain, and administer surveys to a comparison group. The system will meta-analytically combine archived data from previously administered surveys - including evaluation data from intervention groups of relevant programs and from control groups where available - to establish counterfactuals for local evaluators. Thus, rather than relying on gold-standard methods (e.g. randomized control trials) for assessing impact, the goal of this approach will be to provide a statistically reliable method for determining the impact of local programs by comparing current performance against benchmarks set by an accumulated database of previous evaluations. During Phase I, we were able to demonstrate the feasibility of using an extant database for completing a pilot test of proposed methods. We refined statistical methods for completing analyses using referent data. We completed extensive analyses to demonstrate the feasibility and utility of the proposed methods. During Phase II, we propose to complete the following tasks: 1. We will design a database that will efficiently reconstitute various configurations of data to allow users to select appropriate comparison groups. 2. We will develop data analysis algorithms that will construct appropriate comparisons for behavioral and mediating variable analyses. 3. We will develop protocols to automatically link the application to data collected by local evaluations conducted using Evaluation Lizard. 4. We will develop protocols to provide a method to import data collected by local evaluations conducted using other (not Evaluation Lizard) services. 5. We will design a Synthetic Controls user interface that will allow clients to register, manage projects, complete analyses, and download reports. 6. We will conduct an alpha test to debug the system. 7. We will conduct a beta test to demonstrate that the system will work with a variety of potential clients.

Public Health Relevance

Currently, local evaluations of drug prevention programs lack the ability to assess effectiveness because few, if any, include control or comparison groups against which to establish the degree to which they have reduced the rate of drug use onset. This project will provide an innovative strategy whereby local drug prevention evaluations can compare their findings with the findings of previous local evaluations. This will allow programs to assess their relative effectiveness.

Agency
National Institute of Health (NIH)
Institute
National Institute on Drug Abuse (NIDA)
Type
Small Business Innovation Research Grants (SBIR) - Phase II (R44)
Project #
5R44DA026219-03
Application #
8248162
Study Section
Special Emphasis Panel (ZRG1-HDM-R (11))
Program Officer
Diana, Augusto
Project Start
2011-06-01
Project End
2014-05-31
Budget Start
2012-06-01
Budget End
2014-05-31
Support Year
3
Fiscal Year
2012
Total Cost
$484,165
Indirect Cost
Name
Tanglewood Research, Inc.
Department
Type
DUNS #
964211247
City
Greensboro
State
NC
Country
United States
Zip Code
27409
Hansen, William B; Derzon, James H; Reese, Eric L (2014) A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming. Eval Health Prof 37:258-82