Quality of Research
Review Date: September 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Hoffman, J. A., Caudill, B. D., Koman, J. J., III, Luckey, J. W., Flynn, P. M., & Hubbard, R. L. (1994). Comparative cocaine abuse treatment strategies: Enhancing client retention and treatment exposure. Co-published simultaneously in the Journal of Addictive Diseases, 13(4), 115-128; and In S. Magura & A. Rosenblum (Eds.), Experimental Therapeutics in Addiction Medicine (pp. 115-128). New York: The Haworth Press. 
Hoffman, J. A., Caudill, B. D., Koman, J. J., III, Luckey, J. W., Flynn, P. M., & Mayo, D. W. (1996). Psychosocial treatments for cocaine abuse: 12-month treatment outcomes. Journal of Substance Abuse Treatment, 13(1), 3-11.
Supplementary Materials Flynn, P. M., Hubbard, R. L., Luckey, J. W., Forsyth, B. H., Smith, T. K., Phillips, C. D., et al. (1995). Individual Assessment Profile (IAP): Standardizing the assessment of substance abusers. Journal of Substance Abuse Treatment, 12(3), 213-221.
Koba Associates, Inc. (1996). Phase II final report: NIDA SBIR Grant No. 5 R44 DA08136-03 entitled "Living in Balance (LIB): A cocaine abuse treatment manual." Unpublished manuscript.
Outcomes
| Outcome 1: Treatment retention |
|
Description of Measures
|
Treatment retention was defined as the number of days between the first and last treatment sessions attended by cocaine-abusing clients, with a maximum of 120 days or 4 months of active treatment.
|
|
Key Findings
|
A randomized clinical trial (RCT) compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Cocaine-abusing clients were much more likely to remain in treatment longer if assigned to any of the enhanced LIB or enhanced usual group therapy conditions as opposed to usual group therapy only (p < .001). Clients assigned to any of the enhanced LIB or enhanced usual group therapy conditions also were less likely to drop out in the first week compared with clients in usual group therapy only (28% to 34% vs. 56%, respectively; p < .005).
Treatment completion rates (having completed at least 90 days in treatment) were 45.2% for the LIB only condition, compared with 19.1% for usual group therapy only and 38% across all treatment groups.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 2: Treatment exposure |
|
Description of Measures
|
Treatment exposure was defined as the number of sessions attended by cocaine-abusing clients.
|
|
Key Findings
|
An RCT compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Clients assigned to any of the three LIB conditions attended an average of 25.2 sessions, compared with an average of 11.7 sessions for clients assigned to the usual group therapy conditions (p < .0001).
Even the LIB alone condition was associated with a higher treatment exposure rate (23.4 sessions attended, on average) relative to the exposure rate (16.5 sessions) for clients assigned to the most enhanced usual group therapy condition (p < .05).
Seventy-four percent of clients assigned to usual group therapy only dropped out prior to their sixth therapy session. Dropout rates for the other five treatment conditions during the same period ranged from 34% to 46% (p < .001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 3: Regular cocaine use |
|
Description of Measures
|
Regular cocaine use was defined as at least weekly use during the past year and was measured by the Individual Assessment Profile (IAP). The IAP is a structured, multidimensional intake assessment interview that covers eight major life domains: (1) background, (2) admissions information, (3) living arrangements, (4) drug, alcohol, and tobacco use, (5) illegal activities, (6) sources of support, (7) health, and (8) treatment history/mental health. The IAP self-report data included an in-treatment validation by random urinalysis on a subset of the original client sample at intake. Regular cocaine use was compared with treatment retention (number of days between the first and last session attended) and treatment exposure (number of sessions attended) to provide evidence of association.
|
|
Key Findings
|
An RCT compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Since there were no significant differences by condition, all six treatment conditions were collapsed and analyzed as one group from intake to the 12-month follow-up interview.
At the 12-month follow-up interview, only 23% of the 178 clients for whom both pre- and posttreatment data were available reported regular (at least weekly) cocaine use in the past year, compared with 84% at the intake interview (p < .05).
Clients who reported they were still regular users of cocaine at the 12-month follow-up were more likely to have dropped out of treatment earlier (39 days after intake, on average) compared with clients who were no longer regular users at follow-up (59 days, on average; p < .05). Clients who reported regular use of cocaine at the 12-month follow-up also were more likely to have attended fewer treatment sessions overall (15 sessions, on average) compared with clients who did not report regular use of cocaine at follow-up (25 sessions, on average; p < .05).
Clients who reported at intake that they used cocaine regularly (at least weekly) were five times more likely to be weekly users of cocaine at follow-up (odds ratio = 4.88); the effect size for this finding was medium. The more treatment sessions clients attended, the less likely they were to be weekly cocaine users at the 12-month follow-up (odds ratio = 0.98); the effect size for this finding was very small.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 4: Regular alcohol use |
|
Description of Measures
|
Regular alcohol use, defined as use occurring at least 3-4 days per week in the past year, was measured by the IAP, a structured, multidimensional intake assessment interview that covers eight major life domains. This outcome was compared with treatment retention (number of days between the first and last session attended) to provide evidence of association.
|
|
Key Findings
|
An RCT compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Since there were no significant differences by condition, all six treatment conditions were collapsed and analyzed as one group from intake to the 12-month follow-up interview. Twelve-month follow-up self-report data (IAP) on regular alcohol use were analyzed for the 60% of the clients (181 of the 303 at intake) for whom both pretreatment (intake) and posttreatment (12 months after treatment discharge) data were available.
At the 12-month follow-up interview, only 16% of clients across all treatment conditions reported regular alcohol use in the past year, compared with 31% at intake (p < .01). Regular alcohol users at the 12-month follow-up were more likely to have dropped out of treatment earlier--an average of 32 days after intake, compared with 59 days for those not reporting regular use (p < .05).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 5: Regular use of other drugs |
|
Description of Measures
|
Regular use of other drugs, defined as at least weekly use during the past year, was measured by the IAP, a structured, multidimensional intake assessment interview that covers eight major life domains. This outcome was compared with treatment retention (number of days between the first and last session attended) and treatment exposure (number of sessions attended) to provide evidence of association.
|
|
Key Findings
|
An RCT compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Since there were no significant differences by condition, all six treatment conditions were collapsed and analyzed as one group from intake to the 12-month follow-up interview. Twelve-month follow-up self-report data (IAP) on regular use of other drugs were analyzed for the 59% of clients (180 of the 303 at intake) for whom both pretreatment (intake) and posttreatment (12 months after treatment discharge) data were available.
At the 12-month follow-up interview, only 7% of clients across all treatment conditions reported at least weekly use of other drugs in the past year, down from 21% at intake (p < .05). These 7% of clients were more likely to have left treatment earlier and to have attended fewer treatment sessions (19 days after intake and 5 sessions, on average) compared with clients not reporting regular use of other drugs (58 days after intake and 24 sessions, on average; p < .01 and p < .05, respectively).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 6: Illegal activities |
|
Description of Measures
|
Illegal activities (drug-related or non-drug-related) were measured by the IAP, a structured, multidimensional intake assessment interview that covers eight major life domains. Illegal activities were defined as arrests, convictions, and incarcerations in the past year for drug-related activities and non-drug-related activities such as aggravated assault, burglary, theft, robbery, stolen property/fencing, forgery/embezzlement, and gambling. Excluded activities included selling illegal drugs (measured separately) and driving while intoxicated. Illegal activities were compared with treatment retention (number of days between the first and last session attended) and treatment exposure (number of sessions attended) to provide evidence of association.
|
|
Key Findings
|
An RCT compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Since there were no significant differences by condition, all six treatment conditions were collapsed and analyzed as one group from intake to the 12-month follow-up interview. Twelve-month follow-up self-report data (IAP) on illegal activities were analyzed for the 61% of clients (184 of the 303 at intake) for whom both pretreatment (intake) and posttreatment (12 months after treatment discharge) data were available.
At the 12-month follow-up interview, 22% of clients across all treatment conditions reported having engaged in drug-related illegal activities, compared with 31% at intake (p < .01). The percentage of clients reporting non-drug-related illegal activities also declined (from 19% at intake to 14% at the 12-month follow-up interview), but this decrease was not statistically significant.
Clients who reported engaging in illegal drug-related or non-drug-related activities at the follow-up interview were more likely to have left treatment earlier--an average of 35 and 30 days after intake, respectively, compared with 59 and 60 days among clients who reported not engaging in illegal activities (p < .01).
Reports of illegal behavior, both drug-related and non-drug-related, were associated with fewer treatment sessions (14 and 12, respectively). In comparison, clients who reported no such illegal behaviors in the 12 months between intervention discharge and the follow-up interview attended 24-25 sessions on average (p < .05).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 7: Drug sales |
|
Description of Measures
|
Active participation in drug sales over the past 12 months was measured by the IAP, a structured, multidimensional intake assessment interview that covers eight major life domains.
|
|
Key Findings
|
An RCT compared the effects of six treatment conditions, each 4 months in duration:
- LIB only (five 2-hour sessions occurring weekly)
- LIB enhanced with individual psychotherapy
- LIB enhanced with individual psychotherapy plus family therapy
- Usual group therapy only (two 90-minute sessions occurring weekly)
- Usual group therapy enhanced with individual psychotherapy
- Usual group therapy enhanced with individual psychotherapy plus family therapy
Since there were no significant differences by condition, all six treatment conditions were collapsed and analyzed as one group from intake to the 12-month follow-up interview. Twelve-month follow-up self-report data (IAP) on drug sales were analyzed for the 60% of clients (181 of the 303 at intake) for whom both pretreatment (intake) and posttreatment (12 months after treatment discharge) data were available.
At the 12-month follow-up interview, 9% of all clients reported participation in drug sales in the previous year, down from 13% at intake (p < .01).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
68% Male
32% Female
|
93% Black or African American
7% White
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Treatment retention
|
3.8
|
3.8
|
1.5
|
3.8
|
3.8
|
3.5
|
3.3
|
|
2: Treatment exposure
|
3.8
|
3.8
|
1.5
|
3.8
|
2.8
|
2.5
|
3.0
|
|
3: Regular cocaine use
|
3.7
|
2.6
|
1.5
|
2.5
|
3.3
|
2.5
|
2.7
|
|
4: Regular alcohol use
|
3.7
|
3.3
|
1.5
|
2.8
|
3.3
|
2.5
|
2.8
|
|
5: Regular use of other drugs
|
3.7
|
3.4
|
1.5
|
2.5
|
3.3
|
2.5
|
2.8
|
|
6: Illegal activities
|
3.7
|
2.8
|
1.5
|
2.8
|
3.3
|
2.5
|
2.7
|
|
7: Drug sales
|
3.5
|
2.8
|
1.5
|
2.8
|
3.3
|
2.5
|
2.7
|
Study Strengths In the applicant's published validation study, with the exception of cocaine, concordance between self-reported use of alcohol and other drugs and urinalysis measures was high (ranging from 88.3% to 98.5%), so the validity of self-report is likely to be acceptable. Random assignment, adequate sample size, and appropriate analyses based on predictions and literature were present.
Study Weaknesses No attempts were made to validate the self-report data on illegal activities with, for example, arrest records or court data. There is no mention of intervention fidelity, and no information was provided about either the training or monitoring/supervision of therapists. Treatment exposure was confounded with treatment conditions because the various conditions offered different amounts of treatment, putting an artificial limit on the number of sessions that could be attended where less treatment was offered. The 12-month follow-up rate--66% of the original sample--was acceptable, but low.
|
|
