Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1August, G. J., Hektner, J. M., Egan, E. A., Realmuto, G. M., & Bloomquist, M. L. (2002). The Early Risers longitudinal prevention trial: Examination of 3-year outcomes in aggressive children with intent-to-treat and as-intended analysis. Psychology of Addictive Behaviors, 16(4S), S27-S39. 
August, G. J., Realmuto, G. M., Hektner, J. M., & Bloomquist, M. L. (2001). An integrated components preventive intervention for aggressive elementary school children: The Early Risers program. Journal of Consulting and Clinical Psychology, 69(4), 614-626. 
Supplementary Materials August, G. J., Bloomquist, M. L., Lee, S. S., Realmuto, G. M., & Hektner, J. M. (2006). Can evidence-based prevention programs be sustained in community practice settings? The Early Risers' Advanced-Stage Effectiveness Trial. Prevention Science, 7(2), 151-165. 
August, G. J., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2007). The Early Risers "Skills for Success" program: A targeted intervention for preventing conduct problems and substance abuse in aggressive elementary school children. In P. H. Tolan, J. Szapocznick, & S. Sambrano (Eds.), Preventing youth substance abuse: Science-based programs for children and adolescents (pp. 137-158). Washington, DC: American Psychological Association.
August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003). Four years of the Early Risers early-age-targeted preventive intervention: Effects on aggressive children's peer relations. Behavior Therapy, 34, 453-470.
August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003). Parceling component effects of a multifaceted prevention program for disruptive elementary school children. Journal of Abnormal Child Psychology, 31(5), 515-527. 
August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2003). Dissemination of an evidence-based prevention innovation for aggressive children living in culturally diverse, urban neighborhoods: The Early Risers effectiveness study. Prevention Science, 4(4), 271-286. 
August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2003). Maintenance effects of an evidence-based prevention innovation for aggressive children living in culturally diverse urban neighborhoods: The Early Risers effectiveness study. Journal of Emotional and Behavioral Disorders, 12(4), 194-205.
August, G. J., Realmuto, G. M., Winters, K. C., & Hektner, J. M. (2001). Prevention of adolescent drug abuse: Targeting high-risk children with a multifaceted intervention model--The Early Risers "Skills for Success" program. Applied & Preventive Psychology, 10, 135-154.
Bloomquist, M. L., August, G. J., Lee, S. S., Berquist, B. E., & Mathy, R. (2005). Targeted prevention of antisocial behavior in children: The Early Risers "Skills for Success" program. In R. G. Steele & M. C. Roberts (Eds.), Handbook of mental health services for children, adolescents, and families (pp. 201-214). New York: Kluwer.
Outcomes
| Outcome 1: Academic competence and achievement (performance and behaviors) |
|
Description of Measures
|
Academic achievement in reading and arithmetic was measured using the Woodcock-Johnson Tests of Achievement--Revised. Academic competence was measured using various teacher and parent rating scales. Teachers completed the Learning Problems subscale of the Behavioral Assessment System for Children--Teacher Rating Scale (BASC-TRS), the Cognitive Competence subscale of the Teacher's Scale of Child's Actual Competence and Social Acceptance, and the Concentration Problems subscale of the Teacher Observation of Classroom Adaptation Scale--Revised (TOCA-R). Parents completed the Concentration Problems subscale of the Parent Observation of Classroom Adaptation (POCA). Using factor analysis, the authors created composite scores from the outcome measures to reduce redundancy across measures and create global constructs of the outcomes.
Children were also categorized on severity of aggressiveness based on composite scores from the Aggression scales of the TOCA-R and POCA. Data on all measures were collected at baseline and at the end of 3 consecutive school years (i.e., at pretest and three posttests).
|
|
Key Findings
|
At 2-year follow-up, scores of academic competence improved among children in the intervention group and worsened among children in the control group (p < .018). This finding had a small effect size (Cohen's d = 0.26) but was robust across both genders and all levels of aggressiveness.
At 3-year follow-up, the improvements in academic competence observed at 2 years were maintained (p < .018). Children from intervention families who received at least half of the FLEX program over 3 years exhibited a greater rate of improvement in academic achievement (p < .013) and in concentration problems (p < .036) than children from intervention families who received less than half of the FLEX program. Intervention children whose families participated in FLEX also moved into the normative range for children their age on academic achievement.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 2: Behavioral self-regulation |
|
Description of Measures
|
Behavioral self-regulation was assessed using scales from two parent and two teacher measures: the Aggressive-Disruptive, Hyperactivity, and Impulsivity subscales of the TOCA-R and the POCA and the Aggression and Hyperactivity subscales of the BASC-TRS and BASC--Parent Rating Scale (PRS). Using factor analysis, the authors created composite scores from the outcome measures to reduce redundancy across measures and create global constructs of the outcomes.
Children were also categorized on severity of aggressiveness based on composite scores from the Aggression scales of the TOCA-R and POCA. Data on all measures were collected at baseline and at the end of 3 consecutive school years (i.e., at pretest and three posttests).
|
|
Key Findings
|
At 2-year follow-up, there were no significant overall findings on self-regulation for the entire sample of children. However, among severely aggressive children, those in the intervention group improved in self-regulation, while those in the control group remained relatively stable (p < .008). Severely aggressive children in the intervention group also improved in ratings of aggression, hyperactivity, and impulsivity when compared with similar children in the control group (p < .001).
At 3-year follow-up, there were no statistically significant findings on behavioral self-regulation measures after controlling for baseline scores.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 3: Social competence |
|
Description of Measures
|
Social competence was assessed using the Social Skills, Leadership, and Adaptability subscales of the BASC-TRS and BASC-PRS. As an element of social competence, peer acceptance was measured using the Social Acceptance subscale of the Teacher's Scale of Child's Actual Competence and Social Acceptance. Using factor analysis, the authors created composite scores from the outcome measures to reduce redundancy across measures and create global constructs of the outcomes.
Children were also categorized on severity of aggressiveness based on composite scores from the Aggression scales of the TOCA-R and POCA. Data on all measures were collected at baseline and at the end of 3 consecutive school years (i.e., at pretest and three posttests).
|
|
Key Findings
|
At 2-year follow-up, children in the intervention and control groups improved in social competence, but no overall intervention effects were found for social competence, social skills, or adaptability.
At 3-year follow-up, children in the intervention group exhibited better scores on social competence relative to children in the control group (p = .015). Intervention group children whose families received at least half of the FLEX intervention exhibited more improvement in social skills than control group children (p = .02).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 4: Parental investment in the child |
|
Description of Measures
|
Parental investment in the child was measured using the Involvement, Positive Parenting, and Inconsistent Discipline subscales of the Alabama Parenting Questionnaire; the Depression, Isolation, and Health subscales of the Parenting Stress Index; the Parenting Practices Questionnaire; the Support and Cohesion subscales of the Family Relations Scale; and the teacher and parent versions of the Parent-Teacher Involvement Questionnaire. Using factor analysis, the authors created composite scores from the outcome measures to reduce redundancy across measures and create global constructs of the outcomes.
Children were also categorized on severity of aggressiveness based on composite scores from the Aggression scales of the TOCA-R and POCA. Data on all measures were collected at baseline and at the end of 3 consecutive school years (i.e., at pretest and three posttests).
|
|
Key Findings
|
At 2-year follow-up, parental investment remained stable for both the control and intervention groups, and no significant effects were found. However, intervention and control group parents of the most severely aggressive children showed the lowest level of investment in their children (p < .001).
At 3-year follow-up, parents of severely aggressive children who received at least half of the FLEX intervention improved more rapidly on parental distress than similar control group parents, but the effect was marginally statistically significant (p = .058).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 5: Effective discipline |
|
Description of Measures
|
Effective discipline was measured using the Parenting Practices Questionnaire.
Children were also categorized on severity of aggressiveness based on composite scores from the Aggression scales of the TOCA-R and POCA. Data on all measures were collected at baseline and at the end of 3 consecutive school years (i.e., at pretest and three posttests).
|
|
Key Findings
|
At 2-year follow-up, intervention group parents of severely aggressive children reported improvements in the use of effective discipline, when compared with similar control group parents (p < .003).
At 3-year follow-up, intervention group parents reported improvements in the use of effective discipline relative to control group parents, but the effect was marginally statistically significant (p = .054).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood) 26-55 (Adult)
|
68.6% Male 31.4% Female
|
89% White 11% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Academic competence and achievement (performance and behaviors)
|
4.0
|
4.0
|
3.0
|
2.5
|
3.0
|
4.0
|
3.4
|
|
2: Behavioral self-regulation
|
4.0
|
4.0
|
3.0
|
3.0
|
3.0
|
4.0
|
3.5
|
|
3: Social competence
|
4.0
|
4.0
|
3.0
|
2.5
|
3.0
|
4.0
|
3.4
|
|
4: Parental investment in the child
|
4.0
|
4.0
|
2.5
|
2.0
|
2.5
|
4.0
|
3.2
|
|
5: Effective discipline
|
4.0
|
4.0
|
2.5
|
2.0
|
2.5
|
4.0
|
3.2
|
Study Strengths The research used well-selected measures with strong reliability and validity. To ensure fidelity of implementation, the researchers employed several strategies, including a fidelity checklist, manualized training, logbooks, supervision, rating forms, observations by fidelity technicians, and a parent satisfaction questionnaire.
Study Weaknesses The fidelity was not as rigorous with the parent component as it was with other aspects of the intervention, particularly from a dosage perspective.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Bloomquist, M., Coleman, S., Berquist, B., & August, G. (n.d.). Early Risers Skills for Success program implementation manual with CD-ROM.
Early Risers Skills for Success training slides [PowerPoint slide handout]
The Evolution of a Community Social Services Agency Into a Sustainable Prevention Service Provider: Dissemination of the Early Risers Prevention Program [PowerPoint slide handout]
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.0
|
2.5
|
1.3
|
1.9
|
Dissemination Strengths Program materials include a collection of creative activities, behavior management strategies, handouts, forms, and session guides. The implementation manual is very detailed and contains good supportive material related to youth violence programming and strategies that could assist in training practitioners or supervisors. To support quality assurance, materials include measures that address child and parental attendance and documentation of services. Developers also provide an Internet-based fidelity monitoring procedure and direct observation of interventions to enhance quality assurance.
Dissemination Weaknesses The program manual includes dense material that would benefit from significant reorganization. The individual unit information is provided, but there is no course outline or direction on how the units relate to each other. No program goals or selection criteria for students are provided. Materials also do not include a clear intervention training curriculum. Very little information is available on using quality assurance measures or using data to improve program implementation.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
2-day, on-site training (includes program manual)
|
$7,000 including travel expenses
|
Yes
|
|
PATHS Basic Kit (grades 1-6)
|
$629 each; $719 with PATHS Turtle Unit
|
Contact the developer
|
|
Additional program manuals
|
$75 each
|
No
|
|
Support and quality assurance information
|
Contact the developer
|
Contact the developer
|
Additional Information The total annual cost to administer the program, including the agency's direct and indirect costs, is approximately $1,500-$2,500 per student.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
August, G. J., Bloomquist, M. L., Lee, S. S., Realmuto, G. M., & Hektner, J. M., (2006). Can evidence-based prevention programs be sustained in community practice settings? The Early Risers' Advanced-Stage Effectiveness Trial. Prevention Science, 7(2), 151-165. 
August, G. J., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2007). The Early Risers "Skills for Success" program: A targeted intervention for preventing conduct problems and substance abuse in aggressive elementary school children. In P. H. Tolan, J. Szapocznick, & S. Sambrano (Eds.), Preventing youth substance abuse: Science-based programs for children and adolescents (pp. 137-158). Washington, DC: American Psychological Association.
August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003). Four years of the Early Risers early-age-targeted preventive intervention: Effects on aggressive children's peer relations. Behavioral Therapy, 34, 453-470.
August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003). Parceling component effects of a multifaceted prevention program for disruptive elementary school children. Journal of Abnormal Child Psychology, 31(5), 515-527. 
* August, G. J., Hektner, J. M., Egan, E. A., Realmuto, G. M., & Bloomquist, M. L. (2002). The Early Risers longitudinal prevention trial: Examination of 3-year outcomes in aggressive children with intent-to-treat and as-intended analysis. Psychology of Addictive Behaviors, 16(4S), S27-S39. 
August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2003). Dissemination of an evidence-based prevention innovation for aggressive children living in culturally diverse, urban neighborhoods: The Early Risers effectiveness study. Prevention Science, 4(4), 271-286. 
August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2003). Maintenance effects of an evidence-based prevention innovation for aggressive children living in culturally diverse urban neighborhoods: The Early Risers effectiveness study. Journal of Emotional and Behavioral Disorders, 12(4), 194-205.
* August, G. J., Realmuto, G. M., Hektner, J. M., & Bloomquist, M. L. (2001). An integrated components preventive intervention for aggressive elementary school children: The Early Risers program. Journal of Consulting and Clinical Psychology, 69(4), 614-626. 
August, G. J., Realmuto, G. M., Winters, K. C., & Hektner, J. M. (2001). Prevention of adolescent drug abuse: Targeting high-risk children with a multifaceted intervention model--The Early Risers "Skills for Success" program. Applied & Preventive Psychology, 10, 135-154.
Bloomquist, M. L., August, G. J., Lee, S. S., Berquist, B. E., & Mathy, R. (2005). Targeted prevention of antisocial behavior in children: The Early Risers "Skills for Success" program. In R. G. Steele & M. C. Roberts (Eds.), Handbook of mental health services for children, adolescents, and families (pp. 201-214). New York: Kluwer.
|
|
|