•  

Intervention Summary

Back to Results Start New Search

Refuse, Remove, Reasons High School Education Program

Refuse, Remove, Reasons High School Education Program (RRR) is a substance abuse prevention program that is designed to reduce high school students' favorable attitudes toward the use of alcohol, tobacco, and other drugs (ATOD); decrease their misperception of normative peer ATOD use; and increase their refusal skills for ATOD use. RRR is based on the premise that students' behavior can be positively modified by increasing their knowledge of the risks of ATOD use and by modeling appropriate behavior within a social context. The program embraces a mutual aid approach to discussion and activities, through which peers learn from each other under the guidance of an adult facilitator and recognize that they are not alone in their thoughts and feelings.

The curriculum is delivered by a trained facilitator over 6 weeks through four classroom sessions that address (1) alcohol and tobacco, (2) marijuana, (3) steroids and over-the-counter and prescription drugs, and (4) consequences of ATOD use. The program curriculum provides students with information about substances, the potential harm from the use of substances, and refusal skills. Each session includes a 5- to 10-minute video segment that provides age-appropriate information communicated through the real-life stories of teenagers. The video segment is followed by focused questions designed to help students integrate the information while practicing refusal skills for ATOD use. Each session concludes with an activity to provide students with additional opportunities to focus on healthy decisionmaking. RRR includes three homework assignments, which consist of Web-based videos and written self-reflection activities that reinforce the objectives of the classroom sessions. RRR also includes resources for facilitators to share with parents, such as videos and print materials to increase their awareness of the program and talking points that can be used in discussions with their children about topics that are often difficult (e.g., substance abuse).

The facilitator must complete training before implementing the program. In addition, the facilitator should have a master's degree in social work or school counseling or a certification in substance abuse prevention, although this is not a requirement for program implementation.

Descriptive Information

Areas of Interest Substance abuse prevention
Outcomes Review Date: June 2013
1: Perception of normative peer ATOD use
2: Refusal skills for ATOD use
3: Attitudes toward ATOD use
Outcome Categories Alcohol
Drugs
Tobacco
Ages 13-17 (Adolescent)
Genders Male
Female
Races/Ethnicities American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White
Race/ethnicity unspecified
Settings School
Geographic Locations Urban
Suburban
Rural and/or frontier
Implementation History RRR was first launched in September 2009 and included sessions on alcohol and tobacco, marijuana, and over-the-counter and prescription drugs. Results obtained from first-year pre-and posttests resulted in the addition of a fourth session on the consequences of ATOD use. To date, the program has been administered to approximately 2,000 students and their families through 18 parochial schools in the New York City dioceses.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: No
Evaluated in comparative effectiveness research studies: No
Adaptations No population- or culture-specific adaptations of the intervention were identified by the developer.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories Universal

Quality of Research
Review Date: June 2013

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Blizzard, G. A. (2013, April). Comprehensive report on Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program (Rev. ed.). New York, NY: Archdiocese Drug Abuse Prevention Program and Connect With Kids Network.

Supplementary Materials

ADAPP Pre- and Post-Survey Tool Item Analysis and Reliability

American Drug and Alcohol Survey: Reliability and Validity (With Permission From RMBSI)

Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: Facilitator Assessment

Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: High School Post-Test

Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: High School Pre-Test

Rationale for Use of the American Drug and Alcohol Survey (ADAS)

Outcomes

Outcome 1: Perception of normative peer ATOD use
Description of Measures Perception of normative peer ATOD use was assessed using 4 items from a modified version of the American Drug and Alcohol Survey, which measures respondents' drug- and alcohol-related behaviors and attitudes and involvement with various substances (marijuana, cocaine, steroids, cigarettes, alcohol, over-the-counter drugs, and narcotic painkillers). Using 4-point Likert-type scales that vary with the question, participants responded to each of the 4 items (e.g., "How much would your friends try to stop you from [using each assessed substance]?" and "How often have your friends asked you to use [each assessed substance]?"). Responses to the items were added to create a composite score, with higher scores indicating greater misperception of normative peer ATOD use.
Key Findings A study was conducted with high school students, who were assigned to the intervention or control group on a school-level basis. Students in the intervention group received RRR, and those in the control group received the school's regular health class curriculum. Participants were assessed at pretest (before the 6-week intervention) and at posttest (immediately after the intervention). From pre- to posttest, participants in the intervention group had an improvement in their perception of normative peer ATOD use (p = .01). Although participants in the control group also had an improvement in their perception of normative peer ATOD use from pre- to posttest, this improvement was not statistically significant.
Studies Measuring Outcome Study 1
Study Designs Quasi-experimental
Quality of Research Rating 2.1 (0.0-4.0 scale)
Outcome 2: Refusal skills for ATOD use
Description of Measures Refusal skills for ATOD use were assessed using 5 items from a modified version of the American Drug and Alcohol Survey, which measures respondents' drug- and alcohol-related behaviors and attitudes and involvement with various substances (marijuana, cocaine, steroids, cigarettes, alcohol, over-the-counter drugs, and narcotic painkillers). Using a 4-point Likert-type scale ranging from 1 (false) to 4 (true), participants responded to each of the 5 items (e.g., "I have clear strategies for refusing substances if they are offered to me," "I have clear strategies for removing myself from a situation if substances are offered to me," "I have clear reasons for refusing substances if they are offered to me"). Responses to the items were added to create a composite score, with higher scores indicating better refusal skills for ATOD use.
Key Findings A study was conducted with high school students, who were assigned to the intervention or control group on a school-level basis. Students in the intervention group received RRR, and those in the control group received the school's regular health class curriculum. Participants were assessed at pretest (before the 6-week intervention) and at posttest (immediately after the intervention). From pre- to posttest, participants in the intervention group had an improvement in their refusal skills for ATOD use (p < .001). Although participants in the control group also had an improvement in their refusal skills from pre- to posttest, this improvement was not statistically significant.
Studies Measuring Outcome Study 1
Study Designs Quasi-experimental
Quality of Research Rating 2.1 (0.0-4.0 scale)
Outcome 3: Attitudes toward ATOD use
Description of Measures Attitudes toward ATOD use were assessed using 6 items from a modified version of the American Drug and Alcohol Survey, which measures respondents' drug- and alcohol-related behaviors and attitudes and involvement with various substances (marijuana, cocaine, steroids, cigarettes, alcohol, over-the-counter drugs, and narcotic painkillers). Using Likert-type scales that vary with the question, participants responded to each of the 6 items (e.g., "How much do you think people harm themselves (physically or otherwise) if they use alcohol 1 or 2 times?" and "Prescription, Over the Counter Drugs and Steroids can be just as dangerous as illegal drugs"). Responses to the items were added to create a composite score, with higher scores indicating a less favorable attitude toward ATOD use.
Key Findings A study was conducted with high school students, who were assigned to the intervention or control group on a school-level basis. Students in the intervention group received RRR, and those in the control group received the school's regular health class curriculum. Participants were assessed at pretest (before the 6-week intervention) and at posttest (immediately after the intervention). From pre- to posttest, participants in the intervention group had a reduction in favorable attitudes toward ATOD use (p < .001). Although participants in the control group also had a reduction in favorable attitudes toward ATOD use from pre- to posttest, this reduction was not statistically significant.
Studies Measuring Outcome Study 1
Study Designs Quasi-experimental
Quality of Research Rating 2.1 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 13-17 (Adolescent) 53.5% Male
46.5% Female
40.5% White
28.4% Hispanic or Latino
15.5% Black or African American
12.6% Race/ethnicity unspecified
2.5% Asian
0.3% American Indian or Alaska Native
0.3% Native Hawaiian or other Pacific Islander

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures
Validity
of Measures
Fidelity Missing
Data/Attrition
Confounding
Variables
Data
Analysis
Overall
Rating
1: Perception of normative peer ATOD use 2.5 2.5 1.0 2.0 2.5 2.0 2.1
2: Refusal skills for ATOD use 2.5 2.5 1.0 2.0 2.5 2.0 2.1
3: Attitudes toward ATOD use 2.5 2.5 1.0 2.0 2.5 2.0 2.1

Study Strengths

The survey items from the modified version of the American Drug and Alcohol Survey used in the study have uniformly acceptable levels of reliability and evidence of construct and face validity. A quasi-experimental design was used, and analysis established that intervention and control groups were equivalent at pretest on drug use during the past 30 days.

Study Weaknesses

The fidelity measure developed for and used in the study measured student activity rather than facilitator adherence to the curriculum and mode and administration. Attrition was greater for the intervention group than the control group, and no further attrition analysis was conducted. There was no indication of use of imputation or sophisticated modeling of missing data. Inability to match some participants' anonymous pre- and posttests was a concern, and the process of nonrandom assignment of schools to intervention and control conditions, including the potential of self-selection, introduced another potential confound.

Readiness for Dissemination
Review Date: June 2013

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Archdiocese of New York Drug Abuse Prevention Program (ADAPP) & Connect With Kids. (2012). Refuse, Remove, Reasons: Facilitator's guide. Bronx, NY: ADAPP.

Archdiocese of New York Drug Abuse Prevention Program (ADAPP) & Connect With Kids. (2012). Refuse, Remove, Reasons: Parent guide. Bronx, NY: ADAPP.

Archdiocese of New York Drug Abuse Prevention Program (ADAPP) & Connect With Kids. (2012). Refuse, Remove, Reasons: Student workbook. Bronx, NY: ADAPP.

Archdiocese of New York Drug Abuse Prevention Program (ADAPP) & Connect With Kids. (n.d.). Refuse, Remove, Reasons [Informational brochure]. Bronx, NY: ADAPP.

Archdiocese of New York Drug Abuse Prevention Program & Connect With Kids (n.d.). Videos for parents [DVD]. Atlanta, GA: Connect With Kids.

Archdiocese of New York Drug Abuse Prevention Program & Connect With Kids (n.d.). Videos for the classroom [DVD]. Atlanta, GA: Connect With Kids.

Connect With Kids Network. (2010). Refuse, Remove, Reasons: Implementation guide. Atlanta, GA: Author

Program Web site, http://rrr.connectwithkids.com/

Other documents:

  • Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: Facilitator Assessment
  • Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: High School Post-Test
  • Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: High School Pre-Test
  • Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program: Tracking Sheet
  • Comprehensive Report on Archdiocese Drug Abuse Prevention Program (ADAPP) Refuse, Remove, Reasons High School Education Program

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials
Training and Support
Resources
Quality Assurance
Procedures
Overall
Rating
3.7 2.9 2.6 3.1

Dissemination Strengths

The short program videos are well produced and engaging, featuring teenagers talking about personal experiences and decisions they have made regarding substance use. The facilitator's guide, student workbook, and parent guide are well organized, concise, clear, and appropriate for their respective audiences. Extensive on-site training for facilitators is available. Quality assurance tools include pre- and posttests for students, a facilitator assessment, and a tracking sheet. Instructions are provided for administering the pre- and posttests to students. The tracking sheet can be used by the facilitator to monitor each student's completion of each session, homework assignments, and pre- and posttests.

Dissemination Weaknesses

There are minor inconsistencies between the information in the videos for parents and the parent guide. There are no opportunities for multischool training or shared learning among schools that are implementing the program, and existing training opportunities are not fully described on the program Web site. No comprehensive fidelity protocol is available to support quality assurance. Although quality assurance materials are available, there is limited guidance on how to use them for measuring behavioral changes or tracking program outcomes.

Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
Annual license for use of the program Web site (includes access to program videos and downloadable homework pages, facilitator's guide, and information sheets for parents) $750 per school per year Yes
Materials set (includes DVDs for use with parents and in the classroom, 2 student workbooks, and 2 facilitator's guides) $599.95 per set No
Student workbook $8.95 each No
Facilitator's guide $8.95 each No
Parent guide $5.95 each No
1-day, on-site training for facilitators $2,500 Yes, one training option is required
Online training for facilitators (includes two 2-hour Webinars during the school year) $800 Yes, one training option is required
Support and consultation via phone, email, or Webinar Free with annual license Yes
Homework tracking form, pre- and posttests, and facilitator assessment Free No
Replications

No replications were identified by the developer.

Contact Information

To learn more about implementation or research, contact:
Stacey DeWitt, J.D.
(404) 459-8081
sdewitt@cwknetwork.com

Fran Maturo, LCSW
(718) 904-1333
frances.maturo@archny.org

Christine Cavallucci, LCSW
(718) 904-1333
ccavallucci@adapp.org

Consider these Questions to Ask (PDF, 54KB) as you explore the possible use of this intervention.

Web Site(s):