Quality of Research
Review Date: December 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Maguin, E., Nochajski, T., DeWit, D., Macdonald, S., Safyer, A., & Kumpfer, K. (2007). The Strengthening Families Program and children of alcoholic's families: Effects on parenting and child externalizing behavior. Manuscript submitted for publication. Study 2Kumpfer, K. L., Greene, J. A., Bates, R. F., Cofrin, K., & Whiteside, H. (2007). State of New Jersey DHS Division of Addiction Services Strengthening Families Program Substance Abuse Prevention Initiative: Year Three Evaluation Report (Reporting period: July 1, 2004-June 30, 2007). Salt Lake City, UT: LutraGroup.
Supplementary Materials DeMarsh, J., & Kumpfer, K. L. (1986). Family-oriented interventions for the prevention of chemical dependency in children and adolescents. Journal of Children in Contemporary Society: Advances in Theory and Applied Research, 18(122), 117-151.
Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501-517.
Kumpfer, K. L., Alvarado, R., Smith, P., & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3(3), 241-246. 
Kumpfer, K. L., Alvarado, R., Tait, C., & Turner, C. (2002). Effectiveness of school-based family and children's skills training for substance abuse prevention among 6-8-year-old rural children. Psychology of Addictive Behaviors, 16(Suppl. 4), S65-S71.
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341-349.
Sibthorp, J., Paisley, K., Gookin, J., & Ward, P. (n.d.). Addressing response-shift bias: Retrospective pretests in recreation research and evaluation. Manuscript submitted for publication.
Outcomes
| Outcome 1: Children's internalizing and externalizing behaviors |
|
Description of Measures
|
Children's internalizing and externalizing behaviors were assessed using items from the Ontario Child Health Study scale, the Gresham and Elliot Social Skills scale, parental reports of grades, and the Parent Observation of Child Adaptation Scale.
|
|
Key Findings
|
In a study conducted in the United States and Canada, families with a parent who had problems with alcohol in the past 5 years were exposed to SFP or a control condition (receiving free and widely available educational materials on parenting and family life skills to study at home). Children who received SFP had lower scores on conduct disorder symptoms (p < .01), oppositional defiance symptoms (p < .05), and behavior problems (p < .05) than children in the control group. The effects were modest (standardized coefficient = -.096, -.071, and -.078, respectively). Among parents who received SFP, U.S. parents reported significantly larger decreases in conduct disorder symptoms than Canadian parents (p < .025).
In a statewide implementation of SFP, three annual cohorts of families completed retrospective pretests to assess change from baseline. Families who participated in SFP reported improvements in overt aggression, covert aggression, concentration problems, criminal behavior, social skills, alcohol and drug use, and depression (all p values < .01). The effect sizes were very small for covert aggression, criminal behavior, and alcohol and drug use (Cohen's d = 0.14, 0.01, and 0.03, respectively) and small for overt aggression, concentration problems, social skills, and depression (Cohen's d = 0.29, 0.48, 0.29, and 0.27, respectively).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 2: Parenting practices/parenting efficacy |
|
Description of Measures
|
Parenting practices and parenting efficacy were assessed using the Alabama Parenting Questionnaire, the Strengthening Families Parenting Scale, and a 30-day substance use measure used by the Center for Substance Abuse Prevention.
|
|
Key Findings
|
In a study conducted in the United States and Canada, families with a parent who had problems with alcohol in the past 5 years were exposed to SFP or a control condition (receiving free and widely available educational materials on parenting and family life skills to study at home). Parents who received SFP had lower scores on inconsistent discipline (p < .05) and verbal abuse (p < .01) than parents in the control group. The effects were modest (standardized coefficient = -.088 and -.095, respectively).
In a statewide implementation of SFP, four annual cohorts of families completed retrospective pretests to assess change from baseline. Families who participated in SFP reported improvements in positive parenting (p < .01), parental involvement (p < .01), parenting skills (p < .001), parental supervision (p < .01), and parenting efficacy (p < .01). The effect sizes were small for parenting skills (Cohen's d = 0.46) and medium for positive parenting, parental involvement, parental supervision, and parenting efficacy (Cohen's d = 0.54, 0.52, 0.58, and 0.55, respectively).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 3: Family relationships |
|
Description of Measures
|
Family relationships were assessed using the Conflicts Tactics Scale (parent version), the Family Effectiveness Scale, and the Family Strengths Scale.
|
|
Key Findings
|
In a statewide implementation of SFP, four annual cohorts of families completed retrospective pretests to assess change from baseline. Families who participated in SFP reported improvements in family organization (p < .01), family cohesion (p < .01), family communication (p < .001), family conflict (p < .001), and family strengths/resilience (p < .01). Effect sizes were small for family conflict (Cohen's d = 0.20) and medium for family organization, family cohesion, family communication, and family strengths/resilience (Cohen's d = 0.67, 0.50, 0.67, and 0.65, respectively).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Preexperimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
26-55 (Adult)
|
53.3% Male
46.7% Female
|
50% Non-U.S. population
29.7% Black or African American
15.9% White
2.4% American Indian or Alaska Native
1.9% Hispanic or Latino
0.1% Asian
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
|
51% Male
49% Female
|
43% White
36% Black or African American
17% Hispanic or Latino
3% Race/ethnicity unspecified
0.5% American Indian or Alaska Native
0.5% Asian
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Children's internalizing and externalizing behaviors
|
3.3
|
3.3
|
3.0
|
3.0
|
3.0
|
3.0
|
3.1
|
|
2: Parenting practices/parenting efficacy
|
3.3
|
3.3
|
3.0
|
3.0
|
3.0
|
3.0
|
3.1
|
|
3: Family relationships
|
3.3
|
3.3
|
3.0
|
3.0
|
3.0
|
3.0
|
3.1
|
Study Strengths The first study used random assignment to help minimize potential confounds. The psychometric properties of the outcome measures used were generally well established. Use of a manualized curriculum, staff training and supervision, and a fidelity measure and process evaluations helped ensure fidelity. In the second study, which occurred over 4 years in real-world settings, improving fidelity each year produced stronger results over time.
Study Weaknesses Studies relied on parental reports of child behavior with no collateral reports to verify that parents were not answering questions based on how they thought the program staff wanted them to answer. The second study used retrospective pretests to assess program effects (i.e., questionnaires administered following the intervention asked respondents to recall, for example, child behaviors at baseline as a pretest measure). Retrospective pretests are best used as measures of perceived change, rather than actual change, and they tend to inflate program effects.
|
|
Readiness for Dissemination
Review Date: December 2007
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Kumpfer, K. L., & Whiteside, H. (2006). Strengthening Families Program, 3 to 5 years [CD-ROM]. Salt Lake City: University of Utah, Strengthening Families Program Office.
Kumpfer, K. L., & Whiteside, H. (2006). Strengthening Families Program, 6 to 11 years [CD-ROM]. Salt Lake City: University of Utah, Strengthening Families Program Office.
Kumpfer, K. L., & Whiteside, H. (2006). Strengthening Families Program, 12 to 16 years [CD-ROM]. Salt Lake City: University of Utah, Strengthening Families Program Office.
Parent handouts
Program dissemination overview
Program Web site, http://www.strengtheningfamiliesprogram.org
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.8
|
4.0
|
3.8
|
3.8
|
Dissemination Strengths Implementation materials are thoughtfully designed and contain considerable detail on establishing the program and addressing obstacles. Materials also include specific guidance for adapting the model to different cultures and ethnically appropriate artwork for use in the manuals. On-site training, an online supervision course, and implementation consultation and technical assistance are provided to support the delivery of this program. A comprehensive array of instruments is available to support quality assurance.
Dissemination Weaknesses The program objectives outlined in the manuals are not clearly related to program content and evaluation. Additional guidance is needed for administering some of the quality assurance measures and interpreting the results.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
CD containing materials for one age group: 3-5, 6-11, or 12-16 years
|
$450 each (or included in training fee)
|
Yes
|
|
2-day, on-site group leader training and one SFP CD master set
|
$3,650 plus travel expenses for 2 trainers for groups of 35 or fewer
|
No
|
|
2-day, on-site group leader training and one SFP CD master set
|
$3,050 plus travel expenses for 1 trainer for groups of 15 or fewer
|
No
|
|
Technical assistance (basic)
|
Free
|
No
|
|
Technical assistance (advanced)
|
$85 per hour
|
No
|
|
Evaluation services
|
$1,950-$12,000 annually depending on number of participants and number of evaluation reports
|
No
|
|
Fidelity site visits
|
$1,500 plus travel
|
No
|
Additional Information Small agencies may find it economical to attend a training hosted by a nearby agency. Lutra Group, Inc., the entity that coordinates SFP training and technical assistance, can help in locating other trainings. Training in the United States is available in English and Spanish. Implementation requires a minimum of five trained staff: two group leaders for the parents, two group leaders for the children, and a site coordinator.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Aktan, G. B., Kumpfer, K. L., & Turner, C. W. (1996). Effectiveness of a family skills training program for substance abuse prevention with inner city African-American families. Substance Use and Misuse, 31(2), 157-175.
DeWit, D., Nochajski, T., Barlow, K., Safyer, A., Maguin, E., Macdonald, S., et al. (2005, May). An outcome evaluation of the Strengthening Families Program 6-12 years for alcohol abusing parents in Ontario, Canada. Presented at a meeting of the Society for Prevention Research, Washington, DC.
Fox, D. P., Gottfredson, D. C., Kumpfer, K. L., & Beatty, P. D. (2004). Challenges in disseminating model programs: A qualitative analysis of the Strengthening Washington DC Families Program. Clinical Child and Family Psychology Review, 7(3), 165-176.
Gottfredson, D., Kumpfer, K., Polizzi-Fox, D., Wilson, D., Puryear, V., Beatty, P., et al. (2006). The Strengthening Washington D.C. Families project: A randomized effectiveness trial of a family-based prevention. Prevention Science, 7(1), 57-74.
Harrison, S., Boyle, S. W., & Farley, O. W. (1999). Evaluating the outcomes of a family-based intervention for troubled children: A pretest-posttest study. Research on Social Work Practice, 9(6), 640-655.
Hernandez, L., & Lucero, E. (1996). DAYS La Familia Community Drug and Alcohol Prevention Program: Family centered model for working with inner-city Hispanic families. Journal of Primary Prevention, 16(3), 255-272.
Kameoka, V. A. (1996). The effects of a family-focused intervention on reducing risk for substance abuse among Asian and Pacific-Island youths and families. Evaluation of the Strengthening Hawaii's Families Project (1991-1995). Honolulu: University of Hawaii School of Social Work, Social Welfare Evaluation and Research Unit.
Kumpfer, K. L., & Alvarado, R. (2003). Family-strengthening approaches for the prevention of youth problem behaviors. American Psychologist, 58(6-7), 457-465.
Kumpfer, K. L., Alvarado, R., Smith, P., & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3(3), 241-246.
Kumpfer, K. L., Alvarado, R., Tait, C., & Turner, C. (2002). Effectiveness of school-based family and children's skills training for substance abuse prevention among 6-8-year-old rural children. Psychology of Addictive Behaviors, 16(Suppl. 4), S65-S71.
|
|
|
