Quality of Research
Review Date: November 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Morehouse, E. R., & Tobler, N. S. (2000). Project SUCCESS final report: Grant number 4 HD1 SP07240. Report submitted January 26, 2000, to the Center for Substance Abuse Prevention, U.S. Department of Health and Human Services. Study 2Vaughan, R., & Johnson, P. (2007). The effectiveness of Project SUCCESS (Schools Using Coordinated Community Efforts to Strengthen Students) in a regular secondary school setting. Unpublished manuscript.
Outcomes
| Outcome 1: Alcohol, tobacco, and other drug (ATOD) use |
|
Description of Measures
|
ATOD use was measured using a revised version of the American Drug and Alcohol Survey (ADAS). The survey was revised so that it could be administered in one class session. A drug use index was created by summing the scores of self-reported use of 13 drugs: tobacco, alcohol, marijuana, crack, cocaine, heroin, inhalants, LSD, PCP, amphetamines, meta-amphetamines, ecstasy, and "andrenochomes," a false drug included to identify students who overreported drug use.
In one study, for the purposes of analysis, students were classified as ATOD users and nonusers based on their pretest use status.
|
|
Key Findings
|
At posttest in the first year of a study involving alternative secondary school students:
- Self-reports showed a 37% decrease in ATOD use among Project SUCCESS participants relative to students in the comparison group who did not participate in Project SUCCESS (p < .001).
- Of the students using ATOD at pretest, 23% of those in the Project SUCCESS program reportedly stopped ATOD use, whereas only 5% in the comparison condition reported stopping (p < .001).
- For those Project SUCCESS students who did not discontinue ATOD use, there was a significant reduction in reported ATOD use across the drugs assessed, ranging from 17% (p < .05) to 26.6% (p < .01).
At follow-up in the second year of the same study, among Project SUCCESS students who reported using ATOD at pretest, 33.3% reportedly stopped using alcohol, 45.0% reportedly stopped using marijuana, and 22.9% reportedly stopped using tobacco (all p values < .05).
In another study, 21 months following the intervention, regular secondary school students who were involved in Project SUCCESS were less likely than students in the control group to report having ever used marijuana, having smoked in the past month, and having ever used any other substance alone (all p values < .05).
Among pretest users, 21 months following the intervention:
- Among students who used alcohol and cigarettes at pretest, students in the control group were 2.32 times more likely than similar intervention students to report continued use of alcohol and cigarettes; 4.3 times more likely to report use of alcohol, cigarettes, and marijuana; and 5 times more likely to report use of illicit substances (all p values < .05).
- Among students who used alcohol, cigarettes, and marijuana at pretest, students in the control group were 4.16 times more likely than similar intervention students to report continued use of alcohol and cigarettes; 4.54 times more likely to report continued use of alcohol, cigarettes, and marijuana; and 7.33 times more likely to report use of illicit substances (all p values < .05).
Among students who used illicit substances at pretest, students in the control group were 4.76 times more likely than intervention students to report continued use of alcohol and cigarettes; 5 times more likely to report continued use of alcohol, cigarettes, and marijuana; and 2.7 times more likely to report continued use of illicit substances (all p values < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 2: Risk and protective factors for ATOD use |
|
Description of Measures
|
Risk and protective factors were measured using a revised version of the Prevention Planning Survey (PPS). The survey was revised so that it could be administered in one class session. In one study, 10 scales measuring risk and protective factors from the school, peer, and family domains were retained from the PPS. For the purposes of analysis, students were classified as ATOD users and nonusers based on their pretest use status. In another study, six risk factor indices were created based on the PPS: depression, direct peer pressure, number of friends using ATOD, perceived harm of ATOD use, lack of self-control, and violence. In addition, four protective factor indices were created: self-esteem, family protection, positive peer relationships, and peer and school nondrug activities. This study also used the Community Oriented Programs Environment Scale (COPES).
|
|
Key Findings
|
In a study with alternative secondary school students, Project SUCCESS participants had fewer friends who used ATOD (p < .01) and fewer drug-related problem behaviors (p < .01) relative to students in comparison schools.
Surveys of students attending Project SUCCESS schools (including students who did not participate directly in the intervention) also showed greater participation in peer and school nondrug activities (p < .05) and greater decreases in ATOD-related problem behaviors (p < .05) relative to students in comparison schools, indicating a schoolwide benefit to students regardless of direct participation in the intervention.
In a study with regular secondary school students, 9 months and 21 months following the intervention, comparisons of pretest ATOD users in the intervention group with pretest users in the control group indicated:
- An increase in the degree to which Project SUCCESS students reported they cared about their families (p < .05).
- An increase in the amount of help Project SUCCESS students said they expected to receive from the police, when needed (p < .05).
- A decrease in Project SUCCESS students' reported number of friends who smoked cigarettes (p < .05).
- Greater confidence among Project SUCCESS students that their parents would try to stop them if they were to start smoking (p < .05).
Results for pretest nonusers were somewhat inconsistent 9 months following the intervention. For example, relative to nonusers in the control group, nonusers exposed to Project SUCCESS reported increased perception of harm from using marijuana once or twice (p < .01) but also decreased liking by their teachers (p < .05), a decrease in the extent to which they thought their family would try to stop them from getting drunk (p < .05), and a decrease in their friends' liking of their teachers (p < .01).
No significant differences were found for nonusers in the control and intervention groups for risk and protective factors 21 months following the intervention.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
56% Male
44% Female
|
65.6% Black or African American
19.8% Hispanic or Latino
13.6% White
0.9% Race/ethnicity unspecified
|
|
Study 2
|
13-17 (Adolescent)
|
54.4% Male
45.6% Female
|
41.5% Black or African American
23.1% Hispanic or Latino
20.9% White
12.6% Race/ethnicity unspecified
1.6% Asian
0.3% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Alcohol, tobacco, and other drug (ATOD) use
|
3.8
|
4.0
|
2.5
|
2.5
|
3.0
|
3.5
|
3.2
|
|
2: Risk and protective factors for ATOD use
|
3.8
|
4.0
|
2.5
|
2.5
|
3.0
|
3.5
|
3.2
|
Study Strengths The developers used well-established outcome measures in a naturalistic setting. Intervention effects were found, despite the presence of confounds that reduced the likelihood that the intervention would produce effects.
Study Weaknesses Although some aspects of treatment fidelity (e.g., attendance in counseling sessions) were assessed, others were not. For example, the content of the PES and counseling sessions was not objectively observed or evaluated. Contamination across treatment conditions occurred within program schools, in that all students were exposed to universal prevention activities.
|
|
Readiness for Dissemination
Review Date: November 2007
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Morehouse, E., Fallick, A., & Pierce, B. (2006). Project SUCCESS implementation manual (Rev. ed.). Tarrytown, NY: Student Assistance Services Corporation.
Program Web site, http://www.sascorp.org
Project SUCCESS Free Dissemination Materials Packet
Student Assistance Services Corporation. (2006). Process evaluation data collection log. Tarrytown, NY: Author.
Student Assistance Services Corporation. (2006). Student Assistance Services resource manual: Out-of-town. Tarrytown, NY: Author.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.8
|
4.0
|
4.0
|
3.9
|
Dissemination Strengths The implementation manual is detailed and easy to understand, and it has all the pertinent programmatic and theoretical information nicely displayed. All materials and training are comprehensive and support the implementation and overall management of the program. On- and off-site technical assistance is also available through the developer. Implementation and supervision checklists, outcome measures, and a suggested performance review format are provided to support quality assurance.
Dissemination Weaknesses A few photocopied forms and handouts are poorly reproduced and may be difficult to use after continued duplication.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Implementation manual (includes implementation checklists)
|
$150 each
|
Yes
|
|
Brochure for teachers
|
$0.50 each
|
No
|
|
3-day training in Tarrytown, NY (includes implementation manual, resource manual, and brochure for teachers)
|
$350 per person
|
No
|
|
3-day, on-site training (includes implementation manual and resource manual)
|
$4,200 for up to 30 participants, plus travel expenses
|
No
|
|
Scheduled telephone conference calls
|
$150 per hour
|
No
|
|
On-site consultation
|
$200 per hour plus travel expenses
|
No
|
|
Process evaluation data collection log
|
$50 each
|
No
|
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Barnes, B. (2006). Grants To Reduce Alcohol Abuse Program (Project TACT: Teenage Alcohol Curtailment Team). Final performance report: Grant number S184A020074. Report submitted October 10, 2006, to the U.S. Department of Education.
* Morehouse, E. R., & Tobler, N. S. (2000). Project SUCCESS final report: Grant number 4 HD1 SP07240. Report submitted January 26, 2000, to the Center for Substance Abuse Prevention, U.S. Department of Health and Human Services.
* Vaughan, R., & Johnson, P. (2007). The effectiveness of Project SUCCESS (Schools Using Coordinated Community Efforts to Strengthen Students) in a regular secondary school setting. Unpublished manuscript.
|
|
|
