Quality of Research
Review Date: November 2011
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Brotman, L. M., Calzada, E., Huang, K.-Y., Kingston, S., Dawson-McClure, S., Kamboukos, D., et al. (2011). Promoting effective parenting practices and preventing child behavior problems in school among ethnically diverse families from underserved, urban communities. Child Development, 82(1), 258–276.  Study 2Brotman, L. M., Dawson-McClure, S., Huang, K. Y., Theise, R., Kamboukos, D., Wang, J., et al. (2012). Early childhood family intervention and long-term obesity prevention among high-risk minority youth. Pediatrics, 129(3), e621-e628. (NOTE: At the time of the NREPP review, the manuscript of this article had been submitted for publication but not yet accepted.)
Brotman, L. M., Dawson-McClure, S., Kamboukos, D., Calzada, E. J., Huang, K. Y., Acra, F. C., et al. (2011). Preventing conduct problems and promoting school success among children living in low-income, urban communities: Replication of ParentCorps effects. Manuscript in preparation.
Outcomes
| Outcome 1: Parenting practices |
|
Description of Measures
|
Parenting practices were assessed using the following measures:
- Parenting Practices Interview (PPI). In one study, the Appropriate Discipline (8 items) and Clear Expectations (3 items) scales were combined to create a composite scale of parent-reported effective disciplinary strategies. In another study, the Positive Parenting (15 items) and Clear Expectations (4 items) scales were combined to create a composite scale of effective parenting. Using a scale from 1 to 5, parents rated each item, and their scores were summed. Higher scores indicate better parenting practices.
- Effective Parenting Test (EPT), which measures parents' ability to identify appropriate behavioral practices of preschool-aged children. The EPT presents 10-12 vignettes with 3 or 4 response choices, for example, "Although Marcus knows how to speak properly, he often talks baby talk when he's at home. His mother finds this very annoying. In order to reduce this behavior, she should: a) Ignore his baby talk and talk to him when he makes sense (correct response); b) Scold his baby talk and tell him to act his age; or c) Try to understand his baby talk, translate it into grown-up or normal talk, and encourage him to imitate the correct response." One response choice is correct (scored as 1), and the others are incorrect (scored as 0). Parents' responses were scored, and the scores were summed. Higher scores indicate better knowledge of parenting practices.
- Global Impressions of Parent Child Interactions-Revised (GIPCI-R). Independent evaluators who were blind to the intervention conditions rated videotapes of parent-child interaction that occurred in the home. The 15-minute semistructured play interaction increased in structure and parent directedness (i.e., free play, puzzle task, cleanup). Using a scale ranging from 1 to 5, evaluators rated parenting behaviors. A composite score was developed on the basis of indices assessing parenting effectiveness and scaffolding of child behavior. Higher scores indicate better parenting behavior.
|
|
Key Findings
|
In one study, eight schools with universal prekindergarten (pre-K) programs designated to serve lower income children (relative to the general community) through a federally subsidized program were recruited from one community school district in New York City. The schools were randomly assigned to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which pre-K students and their parents received education as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Two consecutive cohorts were recruited.
At the postintervention assessment, which used an intent-to-treat approach, parents in the intervention condition had higher scores on the PPI and EPT than parents in the control condition (p < .01), after controlling for baseline scores. This finding was associated with a medium effect size (Cohen's d = 0.50).
Also at the postintervention assessment, among parents initially rated by independent evaluators as having low parenting effectiveness (i.e., those with a GIPCI-R score below the median), those in the intervention group had higher scores on the GIPCI-R than parents in the control group (p < .05). This finding was associated with a large effect size (Cohen's d = 1.44). Further, at the postintervention assessment, parents in the intervention group who were initially rated as having low parenting effectiveness had the same rating level as parents who were initially rated as having high parenting effectiveness (i.e., those with a GIPCI-R score above the median).
In another study, 10 public elementary schools in highly disadvantaged neighborhoods in two community school districts in New York City were randomized to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which pre-K students and their parents received education as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Four consecutive cohorts were recruited.
At both the postintervention (p < .001) and 12-month follow-up (p < .001) assessments, parents in the intervention condition had higher scores on the PPI and EPT than parents in the control condition, after controlling for baseline scores. These findings were associated with small effect sizes (Cohen's d = 0.43 and 0.42 for the postintervention and 12-month follow-up assessments, respectively).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 2: Child behavior problems |
|
Description of Measures
|
Child behavior problems were assessed using the following measures:
- Behavior Assessment System for Children-Preschool Version (BASC), which captures children's overall adaptive behavior (e.g., adaptability, social skills) as well as externalizing problems (e.g., aggression, disruptive behaviors, hyperactivity) and internalizing problems (e.g., anxiety, depression, somatization). Using a 4-point scale ranging from 0 (never) to 3 (almost always), pre-K teachers rated how often each child engaged in each behavior during the past 4 weeks. Higher scores indicate more problematic behavior.
- Teacher and parent report versions of the New York Rating Scale (NYRS), which covers the full range of symptoms that correspond to DSM diagnoses of oppositional defiant disorder and conduct disorder. The Defiance scale includes 14 items (e.g., "defiant," "argues, quarrels with adults," "loses temper," "disobedient, difficult to control"), and the Conduct Problems scale includes 7 items of more serious, but less commonly occurring, actions typical of emerging conduct disorder (e.g., "steals on the sly," "cheats at games," "uses obscene language"). Using a scale ranging from 0 (not at all) to 3 (very much), teachers or parents rated child behavior problems during the past 4 weeks. Given that the Conduct Problems scale items tend to be endorsed infrequently, items were rescaled to a dichotomized response to indicate whether the item was endorsed or not endorsed. Higher scores indicate more problematic behavior.
|
|
Key Findings
|
In one study, eight schools with universal pre-K programs designated to serve lower income children (relative to the general community) through a federally subsidized program were recruited from one community school district in New York City. The schools were randomly assigned to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which pre-K students and their parents received services as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Two consecutive cohorts were recruited.
At the postintervention assessment, which used an intent-to-treat approach, externalizing (from a combination of the BASC and NYRS) and internalizing (from the BASC) behavioral scores were lower and adaptive behavioral scores (from the BASC) were higher for children in the intervention condition relative to scores for children in the control condition (p < .05), after controlling for baseline scores. This finding was associated with a medium effect size (Cohen's d = 0.56).
In another study, 10 public elementary schools in highly disadvantaged neighborhoods were randomized to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which pre-K students and their parents received education as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Four consecutive cohorts were recruited.
At both the postintervention (p = .02) and 12-month follow-up (p = .002) assessments, among children who entered school with elevated levels of defiant behavior (i.e., approximately 10% of the sample at baseline as indicated by the Defiance scale of the NYRS), those in the intervention condition had lower scores on the Defiance scale and the Conduct Problems scale of the NYRS than children in the control condition. These findings were associated with small (Cohen's d = -0.48 for the postintervention assessment) and medium (Cohen's d = -0.74 for the 12-month follow-up assessment) effect sizes.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 3: Parent involvement in school |
|
Description of Measures
|
Parent involvement in school was assessed using the Parent Involvement With School/Teacher subscale of the INVOLVE-Teacher measure. Teachers reported the extent to which the parent contacted the teacher, visited the school, and attended scheduled meetings in the previous 2 months.
|
|
Key Findings
|
Ten public elementary schools in highly disadvantaged neighborhoods were randomized to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which the pre-K students and their parents received education as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Four consecutive cohorts were recruited.
Ratings by pre-K teachers at the postintervention assessment (p < .001) and kindergarten teachers at the 12-month follow-up assessment (p < .01) indicated that parents in the intervention condition were more involved with their child's school and teacher than parents in the control condition, after controlling for baseline ratings and using an intent-to-treat approach. These findings were associated with small effect sizes (Cohen's d = 0.46 and 0.33 for the postintervention and 12-month follow-up assessments, respectively).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 4: Academic achievement |
|
Description of Measures
|
Academic achievement was measured by the Kaufman Test of Educational Achievement--II, a standardized test of reading (word recognition and reading comprehension), math (computation and application problems), and written expression (written language and spelling). Higher scores indicate greater academic achievement.
|
|
Key Findings
|
Ten public elementary schools in highly disadvantaged neighborhoods were randomized to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which pre-K students and their parents received education as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Four consecutive cohorts were recruited.
At the 12-month follow-up assessment, children in the intervention condition had higher scores in reading (p = .01) and writing (p = .01) than children in the control condition, after controlling for baseline levels of school readiness. These findings were associated with small effect sizes (Cohen's d = 0.32 and 0.23 for reading and writing scores, respectively).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 5: Body mass index |
|
Description of Measures
|
Body mass index (BMI) was calculated from each child's height and weight, which were measured with a stadiometer and electronic scale, respectively, during a physical examination by a trained medical student or research assistant. A higher BMI indicates a higher level of body fat.
|
|
Key Findings
|
Ten public elementary schools in highly disadvantaged neighborhoods were randomized to the intervention condition, in which pre-K students and their parents received ParentCorps, or the control condition, in which pre-K students and their parents received education as usual (i.e., full- or half-day classes; occasional workshops on topics such as nutrition, domestic violence, and asthma). Four consecutive cohorts were recruited.
At a follow-up assessment at age 8, among children in the first two cohorts who had elevated levels of behavior problems at age 4 (approximately 25% of the sample, as indicated by t-scores greater than 50 on the aggression subscale of the BASC or any physical aggression on the NYRS), those in the intervention condition had a lower BMI than children in the control condition (p = .006), after controlling for pre-K body size ratings (performed by research assistants who used the 9-point standardized Figure Rating Scale).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
0-5 (Early childhood)
26-55 (Adult)
|
56% Female
44% Male
|
39% Black or African American
24% Hispanic or Latino
13% White
12% Asian
12% Race/ethnicity unspecified
|
|
Study 2
|
0-5 (Early childhood)
|
51% Female
49% Male
|
91% Black or African American
9% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Parenting practices
|
3.0
|
3.3
|
3.0
|
3.5
|
3.6
|
3.5
|
3.3
|
|
2: Child behavior problems
|
3.7
|
3.7
|
3.0
|
3.5
|
3.5
|
3.4
|
3.4
|
|
3: Parent involvement in school
|
3.5
|
3.4
|
3.0
|
3.4
|
3.4
|
3.4
|
3.3
|
|
4: Academic achievement
|
3.9
|
3.9
|
3.0
|
3.5
|
3.7
|
3.7
|
3.6
|
|
5: Body mass index
|
2.8
|
3.3
|
3.0
|
3.1
|
3.6
|
3.5
|
3.2
|
Study Strengths Almost all of the measures in both studies have been used extensively and have good reliability and validity. In one study, the researchers attempted to compensate for the limitations of BMI by using body size ratings made by research assistants unaware of the children's height, weight, or BMI. A high level of fidelity to the intervention was established in both studies through the use of a detailed manual and checklists, training, ongoing supervision, and independent observation. Independent raters completed implementation checklists for a subset of videotaped sessions, which demonstrated that more than 90% of the components were implemented. Missing data from most measures were minimized, and both studies had low levels of attrition and no differential attrition between groups. Also in both studies, imputation strategies were used to manage missing data as needed. Potential confounds were managed well. Schools were randomly assigned to one of two conditions through a matched-pairs design, with demonstrated baseline equivalence between the intervention and control groups. The multivariate analyses used were appropriate and sophisticated, and they accounted for treatment effects, dosage effects, and interaction effects.
Study Weaknesses Although BMI is conventionally accepted by experts in the field as a measure of body fat among children, there are recognized limitations to the BMI methodology, including the lack of differentiation between adipose, skeletal, and muscle tissue. A discussion of the reliability and validity of the fidelity instrument is absent. There was no consideration for the low rates of program attendance (40% of the children and their parents attended five or more sessions) in one study, although appropriate intent-to-treat analyses were used.
|
|
