Quality of Research
Review Date: December 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Senft, R. A., Polen, M. R., Freeborn, D. K., & Hollis, J. F. (1997). Brief intervention in a primary care setting for hazardous drinkers. American Journal of Preventive Medicine, 13(6), 464-470.  Study 2Baer, J. S., Kivlahan, D. R., Blume, A. W., McKnight, P., & Marlatt, G. A. (2001). Brief intervention for heavy-drinking college students: Four-year follow-up and natural history. American Journal of Public Health, 91(8), 1310-1316.
Marlatt, G. A., Baer, J. S., Kivlahan, D. R., Dimeff, L. A., Larimer, M. E., Quigley, L. A., et al. (1998). Screening and brief intervention for high-risk college student drinkers: Results from a 2-year follow-up assessment. Journal of Consulting and Clinical Psychology, 66(4), 604-615. Study 3Monti, P. M., Colby, S. M., Barnett, N. P., Spirito, A., Rohsenow, D. J., Myers, M., et al. (1999). Brief intervention for harm reduction with alcohol-positive older adolescents in a hospital emergency department. Journal of Consulting and Clinical Psychology, 67(6), 989-994. Study 4Bernstein, J., Bernstein, E., Tassiopoulos, K., Heeren, T., Levenson, S., & Hingson, R. (2005). Brief motivational intervention at a clinic visit reduces cocaine and heroin use. Drug and Alcohol Dependence, 77(1), 49-59. Study 5Carroll, K. M., Ball, S. A., Nich, C., Martino, S., Frankforter, T. L., Farentinos, C., et al. (2006). Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multisite effectiveness study. Drug and Alcohol Dependence, 81(3), 301-312.
Outcomes
| Outcome 1: Alcohol use |
|
Description of Measures
|
In one randomized controlled trial (RCT) with middle-aged adults, alcohol use was defined as (1) the number of standard drinks--or standard ethanol content units (SECs) as defined in the Brief Drinker Profile (BDP)--consumed during the past 3 months, (2) drinking days per week in the past 6 months, and (3) average drinks per drinking day in the past 6 months.
In another RCT that addressed drinking among college students, alcohol use was measured using (1) the Daily Drinking Questionnaire (DDQ), which is a log of drinks for each day of the week, and (2) typical alcohol quantity, frequency, and peak consumption (Q-F-P) during the past month. Quantity and peak alcohol consumption were measured on a scale from 0 (0 drinks) to 5 (more than 8 drinks). Frequency was measured on a scale from 0 (less than once a month) to 5 (nearly every day).
A third RCT used the Adolescent Drinking Questionnaire (ADQ), which consists of 4 items from the Adolescent Health Behavior Questionnaire that assess drinking quantity and frequency, high-volume drinking, and drunkenness, each rated on an 8-point scale. Scores were summed across the 4 items to derive a total alcohol use outcome score; outcome scores were averaged across the measurement period.
|
|
Key Findings
|
In one RCT, middle-aged adults presenting for medical appointments at three primary care facilities were screened for hazardous drinking patterns in the past 6 months using the Alcohol Use Disorders Identification Test (AUDIT). The participants were then assigned to receive either usual care (no treatment beyond care for the medical problem they sought to have addressed) or a brief intervention. The intervention consisted of a 30-second scripted message from the primary care clinician expressing concern about the drinking pattern, followed by a 15-minute MI counseling session with a trained health counselor. Results of this study indicated:
- Adults assigned to MI averaged lower reported total consumption (SECs, p = .04) and fewer reported drinking days per week (p = .02) at 6-month follow-up relative to adults assigned to usual care.
- Adults assigned to MI continued to average fewer drinking days per week at 12-month follow-up relative to adults assigned to usual care (p = .04). There were no significant differences between groups for total consumption (SECs) and average drinks per drinking day.
- Men assigned to MI averaged lower scores on all three alcohol consumption measures--total average consumption (p = .03), average drinking days per week (p = .04), and average drinks per drinking day (p = .05)--at 6-month follow-up relative to men assigned to usual care. However, the between-group differences were no longer statistically significant at 12-month follow-up.
- Women assigned to MI averaged fewer drinking days per week at 6-month follow-up relative to women assigned to usual care (p = .05), with no significant between-group differences in alcohol consumption measures at 12-month follow-up.
In a second RCT, university-accepted high school seniors were screened for high-risk drinking as defined by self-reported drinking at least monthly and consuming at least 5 or 6 drinks on 1 occasion in the past month, or having experienced 3 alcohol-related problems on 3 to 5 occasions in the past 3 years. As entering college freshmen, the high-risk students were assigned to either a brief intervention or a no-treatment control condition. The intervention condition consisted of a face-to-face feedback interview in the MI counseling style with a personalized summary sheet following a period of 2 weeks in which the student had maintained a daily log of alcohol consumption. Assessments were conducted at baseline, 6 months after baseline, and annually over the next 4 years. Results from this study showed that:
- High-risk students who received MI feedback were more likely than controls to report, on average, greater decreases in drinking quantity, peak amounts of alcohol consumed, and drinking frequency at 6-month follow-up. Except for drinking frequency, all of these outcome differences were statistically significant (p < .02). A very small effect size (Cohen's d = 0.15) was associated with the differences between high-risk MI recipients and high-risk controls.
- High-risk students who received MI feedback continued to be more likely than controls to report, on average, greater decreases in drinking frequency (p < .029), drinking quantity (p < .001), and peak amounts of alcohol consumed (p < .028) across the first 2 years of follow-up. A very small to small effect size (Cohen's d = 0.14-0.20) was associated with the differences between high-risk MI recipients and high-risk controls. At 2-year follow-up, high-risk students who received MI reported, on average, 3.6 (+/- 2.5) drinks per occasion, compared with averages of 4.0 (+/- 2.8) drinks per occasion reported by high-risk controls and 2.19 (+/- 2.5) drinks per occasion reported by a normative comparison group of students.
- All high-risk male students, regardless of condition assignment, reported drinking more frequently (p < .002) and in greater quantity (p < .0001) relative to all high-risk female students at all assessment points (p < .0001). In addition, high-risk male students who had histories of conduct disorder, regardless of condition assignment, reported higher average peak amounts of alcohol consumed relative to all high-risk female students (with or without conduct disorder histories) across the first 2 years of follow-up (p < .012).
- Across multiple measurement scales, high-risk students who received MI continued to show a greater decrease in the average reported quantity of alcohol consumed compared with high-risk controls across all 4 years of follow-up (p < .001). Significant differences between groups were not sustained on other outcome measures.
- The normative comparison group of students reported, on average, increased quantity of alcohol consumed at the year 2 follow-up and increased drinking frequency at the year 3 and year 4 follow-ups compared with their original baseline assessment in their freshman year (p < .05).
In a third RCT, 18- to 19-year-olds presenting to a hospital emergency room for treatment of an alcohol-related event were assigned to receive either a 35- to 40-minute MI intervention or standard care. Participants were assessed for follow-up 3 and 6 months later. The MI intervention consisted of five components: (1) an introduction and review of the event circumstances, (2) pros and cons around alcohol and driving, (3) personalized assessment and feedback, (4) imagining the future, and (5) establishing goals. Standard care, designed to be consistent with general practice for treating alcohol-involved teens in an urgent care setting, consisted of giving the teen a handout on avoiding drinking and driving and a list of local treatment agencies. Results of this study indicated:
- There were no between-group differences; average reported drinking decreased significantly for all participants from baseline to both the 3- and 6-month follow-ups (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 2: Negative consequences/problems associated with alcohol use |
|
Description of Measures
|
In one RCT, problems associated with alcohol use were identified using the Rutgers Alcohol Problem Inventory (RAPI) and the Alcohol Dependence Scale (ADS). The RAPI is a self-report instrument that rates the occurrence and frequency of 23 items reflecting alcohol's impact on social and health functioning during the past 6 months. Scores on the RAPI were calculated as the number of items occurring at least 1 or 2 times and ranged from 0 to 23. The ADS measures the severity of physical dependence symptoms, with scores that range from 0 to 47.
In another RCT, negative consequences and problems associated with alcohol use were measured using 5 items from the Health Behavior Questionnaire (HBQ). Using 5-point Likert scales ranging from 0 (never) to 4 (5 or more times), the HBQ measures the frequency of trouble with parents, school, friends, dates, or the police because of drinking.
|
|
Key Findings
|
In one RCT, university-accepted high school seniors were screened for high-risk drinking, defined by self-reported drinking at least monthly and consuming at least 5 or 6 drinks on 1 occasion in the past month, or having experienced 3 alcohol-related problems on 3 to 5 occasions in the past 3 years. As entering college freshman, the high-risk students from this group were assigned to either a brief intervention or a no-treatment control condition. The intervention consisted of a face-to-face feedback interview in the MI counseling style with a personalized summary sheet following a period of 2 weeks in which the student had maintained a daily log of alcohol consumption. Assessments were conducted at baseline and annually over the next 4 years. Results of this study showed that:
- All high-risk students, regardless of condition assignment, averaged fewer alcohol-related problems over the first 2 years of follow-up (p < .0001). However, at 2-year follow-up, MI recipients averaged fewer alcohol-related problems in the past 6 months (3.3, +/-3.5) compared with high-risk students in the no-treatment control condition (4.7, +/- 4.4; p < .021). This difference was associated with a small effect size (Cohen's d = 0.32).
- Only 11% of high-risk students who received MI showed mild alcohol dependence (based on a cutoff score of 11 on the ADS) at the 2-year follow-up, compared with 27% of the high-risk students who received no treatment (p < .0001).
- All high-risk female students, regardless of condition assignment, showed greater declines in alcohol-related problems over time compared with all high-risk male students (p < .007).
- High-risk students who received MI reported greater declines in negative consequences associated with drinking, on average, at all assessment points relative to high-risk students in the no-treatment condition (p < .05).
- High-risk students assigned to MI experienced a greater average initial decrease in negative consequences from baseline to the first-year assessment compared with high-risk students in the no-treatment control condition (p = .0095).
- Living in a fraternity or sorority house was associated with more reported drinking (p < .0001) and more reported alcohol-related problems (p < .004) in all high-risk students, regardless of condition assignment. In addition, high-risk male students living in fraternity houses reported more alcohol-related problems relative to the high-risk male and female students living elsewhere (p < .037).
In a second RCT, 18- to 19-year-olds presenting to a hospital emergency room for treatment of an alcohol-related event were assigned to receive either a 35- to 40-minute MI intervention or standard care. Participants were assessed for follow-up 3 and 6 months later. The MI intervention consisted of five components: (1) an introduction and review of the event circumstances, (2) pros and cons of drinking and driving, (3) personalized assessment and feedback, (4) imagining the future, and (5) establishing goals. Standard care, designed to be consistent with general practice for treating alcohol-involved teens in an urgent care setting, consisted of giving the teen a handout on avoiding drinking and driving and a list of local treatment agencies. Results of this study indicated:
- Teens who received MI reported fewer alcohol-related problems (with dates, friends, parents, police, and at school) at 6-month follow-up compared with teens who received standard care (p < .05). This difference was associated with a small effect size (eta-squared = 0.05; Cohen's f = 0.23).
|
|
Studies Measuring Outcome
|
Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 3: Drinking and driving |
|
Description of Measures
|
Drinking and driving was measured using 5 items from the Young Adult Drinking and Driving Questionnaire, which asks the number of times the respondent has driven after consuming various amounts of alcohol. Moving violation records from the Department of Motor Vehicles were also examined for all licensed drivers in the study.
|
|
Key Findings
|
In an RCT, 18- to 19-year-olds presenting to a hospital emergency room for treatment of an alcohol-related event were assigned to receive either a 35- to 40-minute MI intervention or standard care. Participants were assessed for follow-up 3 and 6 months later. The MI intervention consisted of five components: (1) an introduction and review of the event circumstances, (2) pros and cons of drinking and driving, (3) personalized assessment and feedback, (4) imagining the future, and (5) establishing goals. Standard care, designed to be consistent with general practice for treating alcohol-involved teens in an urgent care setting, consisted of giving the teen a handout on avoiding drinking and driving and a list of local treatment agencies. Results from this study indicated:
- About 26% of all study participants reported no drinking-and-driving behavior following treatment. Teens assigned to standard care were almost 4 times more likely than MI recipients to report drinking and driving (p < .05). This difference was associated with a medium effect size (odds ratio = 3.92).
- MI recipients were less likely than teens assigned to standard care to have a moving violation in the 6 months following treatment (3% vs. 23%; p < .05).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 4: Alcohol-related injuries |
|
Description of Measures
|
Alcohol-related injuries were assessed using the Adolescent Injury Checklist (AIC), a 14-item, true/false, self-report measure of recent injuries, adapted to measure alcohol involvement.
|
|
Key Findings
|
In an RCT, 18- to 19-year-olds presenting to a hospital emergency room for treatment of an alcohol-related event were assigned to receive either a 35- to 40-minute MI intervention or standard care. Participants were assessed for follow-up 3 and 6 months later. The MI intervention consisted of five components: (1) an introduction and review of the event circumstances, (2) pros and cons of drinking and driving, (3) personalized assessment and feedback, (4) imagining the future, and (5) establishing goals. Standard care, designed to be consistent with general practice for treating alcohol-involved teens in an urgent care setting, consisted of giving the teen a handout on avoiding drinking and driving and a list of local treatment agencies. Study results indicated:
- Teens who received MI were significantly less likely to report having sustained an alcohol-related injury at 6-month follow-up than those who received standard care (21% vs. 50%; p < .01). This difference was associated with a medium effect size (odds ratio = 3.94).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 5: Drug use (cocaine and opiates) |
|
Description of Measures
|
Use of cocaine and opiates in the past 30 days was assessed using an abbreviated version of the Addiction Severity Index (ASI). The ASI provides composite scores for seven domains: medical, legal, employment, drug, alcohol, family, and psychological functioning. Radio Immune Assay (RIA) analysis of hair samples was used to verify self-reported drug use.
|
|
Key Findings
|
Self-reported cocaine and heroin users (confirmed by hair analysis) not currently in treatment who presented for routine health care at three urban walk-in medical clinics were assigned to receive either a brief MI intervention or usual care. The usual care group received an initial screening and a handout stating, "Based on your screening responses, you would benefit from help with your drug use," followed by a list of available treatment options. The intervention group received the same handout along with a 10- to 45-minute MI intervention delivered by "peer" interventionists (experienced substance abuse outreach workers who were themselves in recovery). Ten days later, intervention participants also recieved a 5- to 10-minute "booster" telephone session in which the peer interventionist reviewed the action plan with the participant and negotiated alternative referrals if necessary. Follow-up assessments were conducted at 3 and 6 months after baseline; hair analysis was conducted at baseline and 6-month follow-up. Results of this study showed that:
- MI recipients were more likely to be abstinent from cocaine and heroin 6 months after the intervention than those assigned to usual care. Percentages of abstinent MI and usual care participants at 6-month follow-up, respectively, were 22.3% and 16.9% for cocaine, 40.2% and 30.6% for opiates, and 17.4% and 12.8% for both cocaine and opiates. The effect sizes associated with these differences were small (adjusted odds ratio = 1.51-1.57).
|
|
Studies Measuring Outcome
|
Study 4
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 6: Retention in treatment |
|
Description of Measures
|
Retention in treatment was defined as (1) the number of treatment sessions completed in the 28 and 84 days after randomization and (2) continued enrollment at the same community-based substance abuse treatment clinic 28 and 84 days after randomization.
|
|
Key Findings
|
In an RCT, substance users entering outpatient treatment at five community-based treatment settings were assigned to receive either a 2-hour intake/evaluation session that incorporated MI strategies and the MI style of counseling or a standard 2-hour intake/evaluation session. Results of this study indicated:
- On average across all five treatment settings, participants assigned to the intake with MI intervention completed significantly more sessions in the 28 days after randomization relative to participants assigned to standard intake (five vs. four sessions; p = .05). A small effect size was associated with this difference (Cohen's d = 0.24).
- Participants assigned to the intake with MI intervention were significantly more likely to be enrolled in treatment at the clinic 28 days after randomization than participants assigned to standard intake (84% vs. 75%; p = .05).
- In a subgroup of participants whose principal substance of abuse was alcohol, those assigned to the intake with MI intervention completed significantly more sessions on average in the 28 days following randomization (p ≤ .01) and more sessions in the 84 days following randomization (p = .05) relative to participants recieving standard intake. Small to medium effect sizes were associated with these differences (Cohen's d = 0.32-0.56).
|
|
Studies Measuring Outcome
|
Study 5
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.9
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
55+ (Older adult)
|
70.5% Male
29.5% Female
|
82% White
18% Race/ethnicity unspecified
|
|
Study 2
|
18-25 (Young adult)
|
54% Female
46% Male
|
83.9% White
16.1% Race/ethnicity unspecified
|
|
Study 3
|
18-25 (Young adult)
|
63.8% Male
36.2% Female
|
79.8% White
12.8% Black or African American
5.3% Asian
2.1% Hispanic or Latino
|
|
Study 4
|
26-55 (Adult)
|
70.6% Male
29.4% Female
|
60% Black or African American
21.5% Hispanic or Latino
12.5% White
6% Race/ethnicity unspecified
|
|
Study 5
|
18-25 (Young adult)
26-55 (Adult)
|
56.7% Male
43.3% Female
|
71.6% White
15.8% Race/ethnicity unspecified
9.9% Black or African American
2.6% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Alcohol use
|
3.3
|
3.5
|
3.5
|
3.3
|
3.5
|
3.6
|
3.4
|
|
2: Negative consequences/problems associated with alcohol use
|
3.3
|
3.3
|
4.0
|
3.5
|
3.4
|
3.8
|
3.5
|
|
3: Drinking and driving
|
3.8
|
3.8
|
4.0
|
3.5
|
2.5
|
3.5
|
3.5
|
|
4: Alcohol-related injuries
|
3.5
|
3.5
|
4.0
|
3.5
|
2.5
|
3.5
|
3.4
|
|
5: Drug use (cocaine and opiates)
|
3.3
|
3.3
|
4.0
|
3.0
|
2.5
|
3.5
|
3.3
|
|
6: Retention in treatment
|
4.0
|
4.0
|
4.0
|
4.0
|
3.5
|
4.0
|
3.9
|
Study Strengths These studies were rigorous clinical trials of high methodological quality. They used well-known, widely accepted assessment instruments with demonstrated reliability and validity. Four of the five studies had large sample sizes, two were multisite trials, and all used random assignment with few exclusionary criteria. Most of the studies were characterized by strong intervention fidelity. Analyses were largely state of the art.
Study Weaknesses Some of the measurement instruments have limited published psychometrics associated with their use. Three of the five studies did not examine developmental trends that potentially could have affected outcome results over time. The studies inconsistently handled potential confounding variables.
|
|
