Quality of Research
Review Date: December 2006
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudinal test. Science, 247, 1299-1305. (data set 1)  Study 2Ellickson, P. L., Bell, R. M., & Harrison, E. R. (1993). Changing adolescent propensities to use drugs: Results from Project ALERT. Health Education Quarterly, 20(2), 227-242. (data set 1) Study 3Ellickson, P. L., Bell, R. M., & McGuigan, K. (1993). Preventing adolescent drug use: Long-term results of a junior high program. American Journal of Public Health, 83(6), 856-861. (data set 1) Study 4Ellickson, P. L., Bell, R. M., Thomas, M. A., Robyn, A. E., & Zellman, G. L. (1988). Designing and implementing Project ALERT: A smoking and drug prevention experiment. The RAND Corporation, R-3754-CHF, December. (data set 1) Study 5Ellickson, P. L., McCaffrey, D. F., Ghosh-Dastidar, B., & Longshore, D. L. (2003). New inroads in preventing adolescent drug use: Results from a large-scale trial of Project ALERT in middle schools. American Journal of Public Health, 93(11), 1830-1836. (data set 2) Study 6Ghosh-Dastidar, B., Longshore, D. L., Ellickson, P. L., & McCaffrey, D. F. (2004). Modifying pro-drug risk factors in adolescents: Results from Project ALERT. Health Education & Behavior, 31(3), 318-334. (data set 2)
Supplementary Materials Data collection forms
Educators Guide
Grade 7 Monitoring Form, Session 1
Grade 7 Student Survey
Grade 8 Student Survey
Teacher Checklist
Outcomes
| Outcome 1: Substance use (alcohol, tobacco, and marijuana) |
|
Description of Measures
|
For alcohol, cigarettes, and marijuana, student questionnaires asked about lifetime use, frequency of use within the past month and year, and the amount used. Students were also asked if they had suffered negative consequences due to the use of these drugs (e.g., missed school, did something they later felt sorry for, got into trouble at home or at school).
|
|
Key Findings
|
In multiple randomized control group studies, Project ALERT produced lasting outcomes for participants from a variety of ethnic and economic backgrounds who were at low, moderate, or high risk for alcohol, tobacco, or marijuana use. Overall, Project ALERT was equally effective when taught solely by classroom leaders and when teen leaders were included in classroom delivery.
Analyses at the end of grade 8 (15 months after baseline) assessed students by risk level for future drug use. Among students who had tried neither cigarettes nor marijuana at the beginning of 7th grade, Project ALERT participants were nearly 50% less likely than other students to become current marijuana users by 8th grade. After incorporation of the 8th-grade booster sessions, this figure increased to more than 60%. Project ALERT participants were 30% less likely than other students to begin using marijuana. All of these findings were statistically significant (p < .05).
For alcohol use, Project ALERT initially produced only modest, short-lived reductions. However, the revised Project ALERT, which incorporated additional material on alcohol, reduced the likelihood of alcohol use by 24%. Participants in the revised Project ALERT schools were also less likely to suffer alcohol-related consequences such as fighting and getting in trouble at home or school because of drinking (p < .05). These effects continued into the 8th grade.
The revised Project ALERT curriculum also had preventive effects on alcohol, cigarette, and marijuana use among students who had tried these substances by 7th grade (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3, Study 4, Study 5, Study 6
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
4.0
(0.0-4.0 scale)
|
| Outcome 2: Attitudes and resistance skills related to alcohol, tobacco, and other drugs |
|
Description of Measures
|
This outcome was measured by student surveys. The surveys inquired about risk and protective factors such as student awareness of consequences of substance use, beliefs about the prevalence of substance use and its acceptability to others, resistance self-efficacy, and expectations of substance use in the next 6 months.
|
|
Key Findings
|
In multiple randomized control group studies, analyses of program effects on attitudinal risk factors showed that Project ALERT dampened prodrug beliefs about cigarette and marijuana use (low resistance self-efficacy, low perceived consequences of use, tolerance of drugs, expectations of future use, and low estimates of peer drug use). Effect sizes for beliefs about cigarette and marijuana use, calculated as the standardized adjusted mean difference between groups, were small for all students combined (0.07-0.17) and for students at different risk levels (0.07-0.27). The curriculum had a more limited impact on beliefs about alcohol.
Although Project ALERT's effects on drug use dissipate in high school, its effects on knowledge and beliefs persist during the high school years. In 10th grade, former ALERT participants were more likely than other students to believe that drug use has negative social consequences and produces dependency, that resistance has benefits, and that fewer peers use and approve of use. Effects on beliefs about the risk of dependency, social consequences of use, and lower peer use continued into 12th grade.
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3, Study 4, Study 5, Study 6
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
4.0
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
51% Female
49% Male
|
71% White
10% Asian
9% Hispanic or Latino
8% Black or African American
2% American Indian or Alaska Native
|
|
Study 2
|
13-17 (Adolescent)
|
51% Female
49% Male
|
71% White
10% Asian
9% Hispanic or Latino
8% Black or African American
2% American Indian or Alaska Native
|
|
Study 3
|
13-17 (Adolescent)
|
51% Female
49% Male
|
71% White
10% Asian
9% Hispanic or Latino
8% Black or African American
2% American Indian or Alaska Native
|
|
Study 4
|
13-17 (Adolescent)
|
51% Female
49% Male
|
71% White
10% Asian
9% Hispanic or Latino
8% Black or African American
2% American Indian or Alaska Native
|
|
Study 5
|
13-17 (Adolescent)
|
50% Female
50% Male
|
87.5% White
8.6% American Indian or Alaska Native
3.9% Race/ethnicity unspecified
|
|
Study 6
|
13-17 (Adolescent)
|
50% Female
50% Male
|
87.5% White
8.6% American Indian or Alaska Native
3.9% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use (alcohol, tobacco, and marijuana)
|
4.0
|
4.0
|
4.0
|
4.0
|
4.0
|
4.0
|
4.0
|
|
2: Attitudes and resistance skills related to alcohol, tobacco, and other drugs
|
4.0
|
4.0
|
4.0
|
4.0
|
4.0
|
4.0
|
4.0
|
Study Strengths The measures and items are well described based on extensive literature review. Most items have been commonly and successfully used in previous research. The items were adjusted for the age of respondents, and five survey versions were pretested. The use of saliva cotinine for truth in tobacco reporting was excellent.
The materials reviewed describe extensive measures to ensure intervention fidelity, including a teacher guide and checklist, and monitoring forms to document further support. Steps were taken to minimize attrition and account for missing data and the developers conducted analyses to assess the impact of attrition. Sample sizes were large. Attrition rates approached 10% but did not surpass this level and therefore were not a threat to internal validity.
Schools were blocked by district, and restricted assignment was used in randomization to enhance comparability between intervention and control schools and minimize potential confounding variables. Intervention effects can be attributed to the Project ALERT intervention outcomes reported in the publications reviewed. Data were analyzed both at the student and school levels and by students' risk levels. Adjustments were made for baseline differences and intraschool correlation.
Study Weaknesses No weaknesses were noted by reviewers.
|
|
Readiness for Dissemination
Review Date: December 2006
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
BEST Foundation. (2005). Project ALERT: A supplemental resource manual.
Ellickson, P., Miller, L., Robyn, A., Wildflower, L., & Zellman, G. (2004). Project ALERT: A drug prevention program for middle grades.
Project ALERT Peer Teen Leader Manual
Project ALERT program posters and sample newsletters
Project ALERT program Web site, http://www.projectalert.com
Project ALERT Readiness for Dissemination (overview document submitted to NREPP)
Project ALERT Training Workshop Manual
Project ALERT videos:
- Project ALERT: Eight Classroom Lesson Videos & Guided Tour (2006)
- Project ALERT: Substance Abuse Prevention That Works! (2004)
- Project ALERT Training Video: Lessons 1, 4, and 8 (2004)
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
4.0
|
4.0
|
3.5
|
3.8
|
Dissemination Strengths Implementation materials are comprehensive, well articulated, and cogently structured. The videos and classroom poster realistically depict situations and should be engaging for young adolescents. Implementation materials are also available in Spanish. Videos are closed-captioned for the hearing impaired. The Project ALERT Web site is easy to navigate. On-site training is available both before and during implementation, and training can also be completed online. Fidelity measures, pre- and postintervention measures, and scoring information are provided to support quality assurance.
Dissemination Weaknesses Additional training, support, and/or technical assistance may be required for administrators to support implementation fidelity.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Curriculum in e-reader format with online videos and files for posters that can be projected
|
Free
|
Yes
|
|
Online training
|
Free
|
No
|
|
Toll-free phone support, online resources, and ALERT Educator newsletter
|
Free
|
No
|
|
Fidelity instrument and alignment and assessment tools
|
Free
|
No
|
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
* Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudinal test. Science, 247, 1299-1305.
* Ellickson, P. L., Bell, R. M., & Harrison, E. R. (1993). Changing adolescent propensities to use drugs: Results from Project ALERT. Health Education Quarterly, 20(2), 227-242.
* Ellickson, P. L., Bell, R. M., & McGuigan, K. (1993). Preventing adolescent drug use: Long-term results of a junior high program. American Journal of Public Health, 83(6), 856-861.
* Ellickson, P. L., Bell, R. M., Thomas, M. A., Robyn, A. E., & Zellman, G. L. (1988). Designing and implementing Project ALERT: A smoking and drug prevention experiment. The RAND Corporation, R-3754-CHF, December 1988.
* Ellickson, P. L., McCaffrey, D. F., Ghosh-Dastidar, B., & Longshore, D. L. (2003). New inroads in preventing adolescent drug use: Results from a large-scale trial of Project ALERT in middle schools. American Journal of Public Health, 93(11), 1830-1836.
* Ghosh-Dastidar, B., Longshore, D. L., Ellickson, P. L., & McCaffrey, D. F. (2004). Modifying pro-drug risk factors in adolescents: Results from Project ALERT. Health Education & Behavior, 31(3), 318-334.
|
|
|
