Quality of Research
Review Date: October 2006
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Bauman, K. E., Foshee, V. A., Ennett, S. T., Pemberton, M., Hicks, K. A., King, T. S., et al. (2001). The influence of a family program on adolescent tobacco and alcohol use. American Journal of Public Health, 91(4), 604-610.  Study 2Bauman, K. E., Ennett, S. T., Foshee, V. A., Pemberton, M., King, T. S., & Koch, G. G. (2002). Influence of a family program on adolescent smoking and drinking prevalence. Prevention Science, 3(1), 35-42.
Supplementary Materials Bauman, K. E., Ennett, S. T., Foshee, V. A., Pemberton, M., & Hicks, K. (2001). Correlates of participation in a family-directed tobacco and alcohol prevention program for adolescents. Health Education & Behavior, 28(4), 440-461.
Bauman, K. E., Ennett, S. T., Foshee, V. A., Pemberton, M., King, T. S., & Koch, G. G. (2000). Influence of a family-directed program on adolescent cigarette and alcohol cessation. Prevention Science, 1(4), 227-237.
Bauman, K. E., Foshee, V. A., Ennett, S. T., Hicks, K., & Pemberton, M. (2001). Family Matters: A family-directed program designed to prevent adolescent tobacco and alcohol use. Health Promotion Practice, 2(1), 81-96.
Ennett, S. T., Bauman, K. E., Foshee, V. A., Pemberton, M., & Hicks, K. A. (2001). Parent-child communication about adolescent tobacco and alcohol use: What do parents say and does it affect youth behavior? Journal of Marriage and Family, 63, 48-62.
Ennett, S. T., Bauman, K. E., Pemberton, M., Foshee, V. A., Chuang, Y.-C., King, T. S., et al. (2001). Mediation in a family-directed program for prevention of adolescent tobacco and alcohol use. Preventive Medicine, 33, 333-346.
Outcomes
| Outcome 1: Prevalence of adolescent cigarette use |
|
Description of Measures
|
Smoking status was determined at baseline and again at 3- and 12-month follow-ups by asking, "How much have you ever smoked cigarettes in your life?" At each time point, adolescents were considered "nonusers" if they reported they had never used cigarettes or "users" if they reported they had used cigarettes (even a single puff).
|
|
Key Findings
|
The intervention reduced prevalence of self-reported smoking among both users and nonusers, after adjusting for demographic variables and use rates at the start of the program. A very small effect size was found 3 months (Cohen's d = 0.19) and 12 months (Cohen's d = 0.17) following the intervention.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 2: Prevalence of adolescent alcohol use |
|
Description of Measures
|
Drinking status was determined at baseline and again at 3- and 12-month follow-ups by asking, "How much alcohol have you ever had in your life?" At each time point, adolescents were considered "nonusers" if they reported they had never used alcohol or "users" if they reported they had used alcohol (even a sip).
|
|
Key Findings
|
The intervention reduced prevalence of self-reported alcohol use among both users and nonusers, after adjusting for demographic variables and use rates at the start of the program. A small effect size (Cohen's d = 0.32) was found at 3 months following the intervention; this was reduced to a very small effect size (Cohen's d = 0.12) at 12 months.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 3: Onset of adolescent cigarette use |
|
Description of Measures
|
Smoking status was determined at baseline and again at 3- and 12-month follow-ups by asking, "How much have you ever smoked cigarettes in your life?" At each time point, adolescents were considered "nonusers" if they reported they had never used cigarettes or "users" if they reported they had used cigarettes (even a single puff).
|
|
Key Findings
|
Data suggest that the intervention reduced smoking onset among adolescents who reported being nonusers at the start of the program. At 12-month follow-up, 16.4% fewer program participants had initiated smoking compared with a control group of adolescents who did not receive the program. Results appeared significantly stronger among non-Hispanic White adolescents than among adolescents of other ethnicities. The effect size was small (Cohen's d = 0.25) for non-Hispanic Whites and very small (Cohen's d = 0.15) for all adolescents.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
51% Female
49% Male
|
73.4% White
12.5% Black or African American
9.2% Hispanic or Latino
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
|
50.7% Female
49.3% Male
|
78% White
9.9% Black or African American
7.6% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Prevalence of adolescent cigarette use
|
2.3
|
2.5
|
3.5
|
3.5
|
3.8
|
3.8
|
3.2
|
|
2: Prevalence of adolescent alcohol use
|
2.3
|
2.5
|
3.5
|
3.5
|
3.8
|
3.8
|
3.2
|
|
3: Onset of adolescent cigarette use
|
2.3
|
2.5
|
3.5
|
3.5
|
3.8
|
3.5
|
3.2
|
Study Strengths The outcome measures used in this study are standardized and pilot-tested for understanding by adolescent respondents; they are comparable to items widely accepted in the field and used in national surveys. The items used have face validity, and concurrent validity is supported by the fact that bivariate associations with relevant variables were in the expected direction, with many reaching statistical significance. There is strong evidence that the intervention was implemented with fidelity: Health educators received 2 full days of training, were monitored during practice, and continued training throughout the study, and intervention procedures followed a standardized written protocol. The authors used a randomized experimental control group design and appropriate controls on confounding variables (e.g., research staff were blinded to intervention condition, adolescents had no contact with intervention staff). Participation and follow-up rates were high for this type of intervention, with complete follow-up data being collected on more than 80% of the sample. Sufficient measures were taken to ensure that there were very few missing data, which were modeled using generalized estimating equations (GEE). Researchers adjusted for baseline variables potentially related to outcomes.
Study Weaknesses It is not clear whether interviewers were directly monitored during phone calls with the adolescents. Only one survey question was used to determine each outcome with no cross-referencing or corroboration to verify the self-report data. Power may not have been adequate to detect the difference in treatment effect at the 3- versus 12-month follow-up. Sample size appears adequate for tests of prevalence outcomes but may have been smaller than necessary to adequately test for effects of the intervention on onset and for baseline variable-by-intervention interactions.
|
|
Readiness for Dissemination
Review Date: October 2006
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Bauman, K., Foshee, V., & Ennett, S. (1996). Book 1: Why families matter [Program booklet]. Chapel Hill: University of North Carolina at Chapel Hill.
Bauman, K., Foshee, V., & Ennett, S. (1996). Book 2: Helping families matter to teens [Program booklet]. Chapel Hill: University of North Carolina at Chapel Hill.
Bauman, K., Foshee, V., & Ennett, S. (1996). Book 3: Alcohol and tobacco rules are family matters [Program booklet]. Chapel Hill: University of North Carolina at Chapel Hill.
Bauman, K., Foshee, V., & Ennett, S. (1996). Book 4: Non-family influences that matter [Program booklet]. Chapel Hill: University of North Carolina at Chapel Hill.
Bauman, K., Foshee, V., Ennett, S., & Hicks, K. (2005). Health educator guidebook: Local implementation version. Chapel Hill: University of North Carolina at Chapel Hill.
Family Matters Program Overview and Training Description
Family Matters Web site, http://www2.sph.unc.edu/familymatters/index.htm
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.3
|
3.0
|
3.5
|
3.3
|
Dissemination Strengths Implementation materials offer comprehensive guidance on how to implement the program, access to the program Web site and original research, and helpful scripts and protocols on how to deal with difficult participant responses. The types of providers and participants targeted by this intervention are clearly defined. Training materials offer step-by-step directions and anticipate a variety of scenarios. Protocols and evaluation instruments with established criteria are provided free on the program Web site to support quality assurance.
Dissemination Weaknesses No information was provided to assist implementers in selecting communities as target participants. Materials would benefit from an implementation matrix or other organizing tool so that users could distinguish background and introductory documents from actual implementation materials. No sample training materials or implementation monitoring checklist was provided for review.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Health Educator Guidebook
|
Free
|
Yes
|
|
Booklets
|
Free
|
Yes
|
|
Trinkets
|
Free
|
Yes
|
|
On-site training
|
$3,000-$5,000 depending on location
|
No
|
|
Consultation
|
Free
|
No
|
|
Quality assurance tools
|
Free
|
Yes
|
Additional Information The cost of implementing Family Matters in a 2001 national evaluation was about $140 per eligible family.
|
Replications
No replications were identified by the developer.
|
|
|
