•  

Intervention Summary

Back to Results Start New Search

Challenging Horizons Program (CHP)

The Challenging Horizons Program (CHP) is a school-based set of interventions for middle/junior high school students with attention-deficit/hyperactivity disorder (ADHD). Building on behavioral and cognitive theories about the nature of the disorder, CHP aims to provide a safe learning environment enhanced by supportive counseling relationships between students and staff. CHP targets the most common areas of impairment associated with ADHD, including social impairment, family conflict, and academic failure. CHP interventions focus on strengthening enabling skills such as organization; specific educational skills such as homework management, studying, and notetaking; socialization; goal setting and self-regulation of behavior; and recreational activities to improve the student's group cooperation, fitness, and sports skills. Parent involvement is encouraged through group parent training and weekly reports. Each intervention is provided in the context of supportive and enjoyable activities.

Two versions of CHP have been evaluated: an after-school model and a consulting model. In the after-school model, a small group of counselors delivers CHP interventions to 5-20 students 2-3 days per week as part of an after-school program. The counselors consult regularly with the students' teachers and hold monthly meetings with the students' parents. In the consulting model, CHP interventions are provided during the school day by school staff (e.g., teachers, school counselors). Staff members volunteer to serve as mentors for individual students, establishing a supportive relationship with the student and providing selected CHP interventions through one-on-one meetings once or twice per week at times established by the mentor (e.g., before school, after school, at lunch, during homeroom, during time assigned for the use of a special education resource room). In the consulting model, up to three CHP interventions are provided to each student at any given time. The after-school model may provide a greater number of concurrent interventions.

CHP is implemented throughout the school year and may be led by a master's or doctoral-level psychologist, social worker, counselor, or behavior specialist, or others who have received training in the CHP. The after-school version of the CHP also requires paraprofessional counselors in a ratio of one staff for every 2 or 3 students.

Descriptive Information

Areas of Interest Mental health treatment
Outcomes Review Date: December 2010
1: ADHD symptoms
2: Social functioning
3: Academic performance
4: School functioning
Outcome Categories Education
Mental health
Social functioning
Ages 6-12 (Childhood)
13-17 (Adolescent)
Genders Male
Female
Races/Ethnicities Black or African American
Hispanic or Latino
White
Race/ethnicity unspecified
Settings School
Geographic Locations Urban
Suburban
Rural and/or frontier
Implementation History The after-school model of CHP was first implemented in 1999 in a middle school in Harrisonburg, Virginia. Since then, it has been implemented in Columbia, South Carolina; Pittsburgh, Pennsylvania; and Athens, Cincinnati, Lancaster, and Logan, Ohio. The consulting model was first implemented in 2003 in Harrisonburg and has been implemented in all four of the Ohio sites. More than 1,500 students have participated in one of the two models of CHP.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: Yes
Evaluated in comparative effectiveness research studies: No
Adaptations No population- or culture-specific adaptations of the intervention were identified by the developer.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories IOM prevention categories are not applicable.

Quality of Research
Review Date: December 2010

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Evans, S. W., Serpell, Z. N., Schultz, B. K., & Pastor, D. A. (2007). Cumulative benefits of secondary school-based treatment of students with ADHD. School Psychology Review, 36(2), 256-273.

Schultz, B. K., Evans, S. W., & Serpell, Z. N. (2009). Preventing failure among middle school students with ADHD: A survival analysis. School Psychology Review, 38(1), 14-27.

Study 2

Langberg, J. M., Smith, B. H., Bogle, K. E., Schmidt, J. D., Cole, W. R., & Pender, C. A. S. (2007). A pilot evaluation of small group Challenging Horizons Program (CHP): A randomized trial. Journal of Applied School Psychology, 23(1), 31-58.

Study 3

Langberg, J. M., Epstein, J. N., Urbanowicz, C. M., Simon, J. O., & Graham, A. J. (2008). Efficacy of an organization skills intervention to improve the academic functioning of students with attention-deficit/hyperactivity disorder. School Psychology Quarterly, 23(3), 407-417.

Supplementary Materials

Psychometric information on measures used.

Outcomes

Outcome 1: ADHD symptoms
Description of Measures ADHD symptoms were assessed using the Behavior Assessment System for Children (BASC). The BASC is a comprehensive behavior rating scale that measures internalizing and externalizing behaviors. The standardized scores on the inattention and hyperactivity subscales were used for analyses of ADHD symptoms. Parents completed the BASC at baseline and at follow-up evaluations.
Key Findings Sixth-grade students were randomly assigned to an intervention group that received CHP or a community care control condition (i.e., students were free to pursue educational, psychosocial, or medication treatment of their choice but were not provided any treatment by the investigators). Students were followed through the end of 7th grade. From the initial evaluation to the final follow-up, CHP participants showed significant improvement in attentiveness, but not in hyperactivity. For inattention, change over time was significantly different for the two groups (p < .05), with the control group getting worse over time and the CHP participants showing improvement; the effect size for this finding was medium (Cohen's d = 0.76). Both the CHP participants and the control group showed a decrease in hyperactivity, with hyperactivity ratings decreasing more sharply over time for the CHP participants compared with the control group, but the difference between groups was not statistically significant.
Studies Measuring Outcome Study 1
Study Designs Experimental
Quality of Research Rating 3.2 (0.0-4.0 scale)
Outcome 2: Social functioning
Description of Measures Social functioning was assessed using the Impairment Rating Scale (IRS). The IRS is a brief rating scale that measures general functioning in various domains (including self-esteem and relationships with peers, siblings, parents, and teachers) and overall problem severity. Parents or teachers indicate the degree to which functioning in the domains is or is not a problem that requires additional treatment. Scores range from 0 (no problem, does not need intervention) to 6 (extreme problem, needs intervention). Higher scores indicate greater impairment and need for additional treatment. Parents and teachers completed the IRS at baseline and then either on a monthly basis (in one study) or one time following completion of the first semester of school (in another study).
Key Findings In one study, 6th-grade students with ADHD were randomly assigned to an intervention group that received CHP or a community care control condition (i.e., students were free to pursue educational, psychosocial, or medication treatment of their choice but were not provided any treatment by the investigators). Students were followed through the end of 7th grade. Parent ratings showed a significantly different change over time in overall social functioning for the two groups, with the control group exhibiting increasing impairment over time and CHP participants improving over time (p < .05). The effect size for this finding was small (Cohen's d = 0.40). Teacher ratings showed no significant between-group difference over time in social functioning.

In another study, 6th- and 7th-graders experiencing a combination of learning and behavior problems were randomly assigned to an intervention group that received CHP or a control condition that included involvement in a district-run after-school program. Parent ratings indicated that CHP participants improved in self-esteem from baseline to the end of the semester while the control group showed a worsening in self-esteem (p = .021). The effect size for this finding was medium (Cohen's d = 0.50). In addition, parent ratings indicated that CHP participants moved from exhibiting overall problem severity at baseline to no longer exhibiting problems at the end of the semester; there was no change for the control group over this period (p = .004). The effect size for this finding was medium (Cohen's d = 0.77). Teacher ratings did not indicate any significant between-group difference over time in self-esteem.
Studies Measuring Outcome Study 1, Study 2
Study Designs Experimental
Quality of Research Rating 3.1 (0.0-4.0 scale)
Outcome 3: Academic performance
Description of Measures Academic performance was assessed using grade point averages (GPAs) for core subjects. GPAs were computed for each grading period as follows: A = 4.0, B = 3.0, C = 2.0, D = 1.0, and F = 0.
Key Findings In one study, 6th-grade students with ADHD were randomly assigned to an intervention group that received CHP or a community care control condition (i.e., students were free to pursue educational, psychosocial, or medication treatment of their choice but were not provided any treatment by the investigators). Students were followed through the end of 7th grade. The investigators' focus in this study was the prevention of academic failure, defined as grading periods for which the student's academic performance was unsatisfactory for most (if not all) core subjects, as agreed on by teachers. By the end of 6th grade, 77% of CHP participants did not experience academic failure in core subjects (reading/English, math, science, and social studies) compared with 59% of students in the control group (p = .03). By the end of 7th grade, 74% of CHP participants did not experience academic failure, compared with 48% of students in the control group (p = .04).

In another study, 6th- and 7th-graders experiencing a combination of learning and behavior problems were randomly assigned to an intervention group that received CHP or a control condition that included involvement in a district-run after-school program. A comparison of mean grades at quarter 1 and quarter 2 in four core subjects (science, history, math, and language arts) revealed that the only significant difference between the intervention and control groups was for science grades at quarter 2: CHP participants had better grades in science than students from the control group (p = .016).

In a third study, students with ADHD in grades 4-7 were randomly assigned to an intervention group that immediately received CHP or a wait-list control group. Results of this study showed a significant improvement in GPAs combined across the four core subjects (math, science, language arts, and history) from pretest to 8-week follow-up for CHP participants but not for the control group (p < .01).
Studies Measuring Outcome Study 1, Study 2, Study 3
Study Designs Experimental
Quality of Research Rating 3.0 (0.0-4.0 scale)
Outcome 4: School functioning
Description of Measures School functioning was assessed using the following:

  • Impairment Rating Scale (IRS): The IRS is a brief rating scale that measures general functioning in various domains including school functioning. The IRS includes six questions pertaining to school, family, self-esteem, and social functioning. Parents or teachers indicate the degree to which functioning in the domains is or is not a problem that requires additional treatment. Scores range from 0 (no problem, does not need intervention) to 6 (extreme problem, needs intervention). Higher scores indicate greater impairment and need for additional treatment. Parents and teachers completed the IRS at baseline and following completion of the first semester of school.
  • Conners Abbreviated Symptom Questionnaire (ASQ): The 10-item Conners Global Index (CGI) portion of the scale evaluates the reported severity of childhood problems related to behaviors important for school functioning. The CGI response format requires parents to answer 10 questions related to their child's behavior (e.g., disturbs other children, restless or overactive, fails to finish things he/she starts) using a 4-point scale (0 = not at all true; 1 = just a little true; 2 = pretty much true; and 3 = very much true). The ASQ (and CGI) was administered to parents at baseline and following the completion of the first semester of school.
  • Homework Problems Checklist (HPC): The HPC is a parent report instrument consisting of 20 items. It is commonly used as a screening tool and outcome measure to assess homework problems. HPC has two subscales: Inattention/Avoidance of Homework (IA) and Poor Productivity/Nonadherence to Homework Rules (NA). The IA and NA factors can be combined to produce an HPC total score. The HPC was completed by parents at baseline and after the intervention.
Key Findings In one study, 6th- and 7th-graders experiencing a combination of learning and behavior problems were randomly assigned to an intervention group that received CHP or a control condition that included involvement in a district-run after-school program. Results of this study showed that:

  • Parent ratings on the IRS showed that CHP participants improved in school functioning from baseline to the end of the semester (from a high level of impairment to a lower level of impairment), whereas for students in the control group, school functioning was highly impaired at baseline and was still highly impaired at the end of the semester (p = .004). The effect size for this finding was medium (Cohen's d = 0.74). Teacher ratings of school functioning using the IRS did not reveal any significant between-group difference in school functioning.
  • Parent reports on the ASQ indicated that CHP participants made large improvements in organizational skills important in school functioning over the semester compared with students in the control group (p = .002). The effect size for this finding was medium (Cohen's d = 0.65). Parent reports on the ASQ revealed no significant between-group difference in homework completion.
  • Parent reports on the CGI showed that CHP participants had fewer behavioral functioning difficulties in school following completion of the intervention, whereas parents of students in the control group reported that behavioral functioning became more impaired over the semester (p < .001). The effect size for this finding was medium (Cohen's d = 0.74).
In another study, students with ADHD in grades 4-7 were randomly assigned to an intervention group that immediately received CHP or a wait-list control group. HPC total scores showed that CHP participants improved overall in homework performance over the course of the 8-week intervention, whereas the control group had no significant change (p < .01). The effect size for this finding was medium (Cohen's d = 0.71). Ratings on the HPC subscales showed that CHP participants made significant improvement in both the IA factor (p < .001) and NA factor (p < .001), whereas the control group had no significant change in either factor.
Studies Measuring Outcome Study 2, Study 3
Study Designs Experimental
Quality of Research Rating 3.3 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 6-12 (Childhood)
13-17 (Adolescent)
77% Male
23% Female
94% White
4% Hispanic or Latino
2% Race/ethnicity unspecified
Study 2 6-12 (Childhood)
13-17 (Adolescent)
66.7% Male
33.3% Female
66% Black or African American
34% White
Study 3 6-12 (Childhood)
13-17 (Adolescent)
83.8% Male
16.2% Female
70% White
30% Black or African American

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures
Validity
of Measures
Fidelity Missing
Data/Attrition
Confounding
Variables
Data
Analysis
Overall
Rating
1: ADHD symptoms 4.0 4.0 2.0 3.0 2.0 4.0 3.2
2: Social functioning 3.5 3.5 2.3 3.5 2.3 3.8 3.1
3: Academic performance 3.0 2.5 2.5 3.5 2.5 4.0 3.0
4: School functioning 3.5 3.5 2.8 3.8 2.5 3.8 3.3

Study Strengths

The studies used measures that have been well researched and have strong psychometric properties. Manualized interventions were used, and training was provided to the interventionists. Some attempts were made to assess intervention fidelity using, for example, counselor's checklists, attendance logs, on-site staff supervision, and training. Attrition was generally low, and the researchers addressed missing data in the analyses. Appropriate statistical analyses were used in all three studies.

Study Weaknesses

Treatment integrity was not adequately addressed. For example, in one study, there was variability in the services received within the intervention group, and only one of the three studies employed an outside observer with a checklist to assess integrity. A number of confounding variables raise concerns about internal validity. Specifically, raters (parents and teachers) were not blinded to condition; analyses did not control for students' medication status, intelligence, and disciplinary status; and there was no external validation of the criteria used to assess students' learning disabilities or other comorbid diagnoses. The small sample size in all three studies limits the ability to evaluate potential moderators of treatment efficacy and may have affected the power of the analyses.

Readiness for Dissemination
Review Date: December 2010

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Embry, D. D., & Biglan, A. (2008). Evidence-based kernels: Fundamental units of behavioral influence. Clinical Child and Family Psychology Review, 11(3), 75-113.  Pub Med icon

Evans, S. W. (2010). Challenging Horizons Program psychosocial service provider manual. Harrisonburg, VA: Steven W. Evans & James Madison University.

Evans, S. W. (2010). Challenging Horizons Program treatment manual. Harrisonburg, VA: Steven W. Evans & James Madison University.

Program Web sites, http://www.scstudentexcellence.org and http://oucirs.org/

Smith, B. H. (2006). Challenging Horizons Program (CHP) [PowerPoint slides].

Smith, B. H., & Challenging Horizons Program. (2007). CHP student manual. Columbia, SC: Author.

Other program materials:

  • 21st Century Grant CHP After-School Program Timeline (2010)
  • Accreditation Visit Procedures and Criteria (2010)
  • Challenging Horizons Program Staff Manual for the Boys and Girls Club at Dent Middle School (2008)
  • Challenging Horizons Program Treatment Fidelity Assessment
  • Challenging Horizons Program Weekly Report Card (2008)
  • CHP Rules Quiz
  • CHP Rules Quiz Answer Key
  • CHP Training Schedule (2010)
  • Classroom Performance Survey (1996)
  • Low-Emotion Reprimand [PowerPoint slides]
  • Low-Emotion Reprimand Worksheet

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials
Training and Support
Resources
Quality Assurance
Procedures
Overall
Rating
3.5 4.0 3.8 3.8

Dissemination Strengths

The curriculum is thorough and supports the program's goals and concepts. Many well-developed forms and protocols are provided to guide implementation of the highly manualized intervention. The Weekly Report Card offers good linkage between staff and parents. Planning materials are customized to each site and include clearly written manuals, useful tools, and a comprehensive list of tasks to be completed; considerable focus is placed on implementation readiness. The program Web sites offer information for a variety of audiences. Detailed and comprehensive fidelity measures are provided for quality assurance. High-quality training materials and other support resources are available, and trainers conduct site visits after implementation begins to determine whether additional training and continued telephone consultation are needed.

Dissemination Weaknesses

The student manual lacks engaging qualities such as color, pictures, and graphics and uses language that may not be ideal for the target age group. Further guidance is needed to aid decisionmakers in justifying the adoption of this program, such as a justification for the high staff-to-student ratio. The Treatment Fidelity Assessment is fairly long and unwieldy.

Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
Manuals Included in the cost of training Yes
Support materials Included in the cost of training Yes
15-hour off-site training $4,000 per site for up to 12 participants, plus travel expenses Yes
Phone consultation Included in the cost of training (up to 16 hours) Yes
Accreditation site visit Included in the cost of training (travel expenses additional) Yes
Contact Information

To learn more about implementation or research, contact:
Steven W. Evans, Ph.D.
(740) 593-2186
CHP@oucirs.org

Consider these Questions to Ask (PDF, 54KB) as you explore the possible use of this intervention.