Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Milby, J. B., Schumacher, J. E., McNamara, C., Wallace, D., Usdan, S., McGill, T., et al. (2000). Initiating abstinence in cocaine abusing dually diagnosed homeless persons. Drug and Alcohol Dependence, 60(1), 55-67. 
Milby, J. B., Schumacher, J. E., Wallace, D., Frison, S., McNamara, C., Usdan, S., et al. (2003). Day treatment with contingency management for cocaine abuse in homeless persons: 12-month follow-up. Journal of Consulting and Clinical Psychology, 71(3), 619-621. Study 2Milby, J. B., Schumacher, J. E., Wallace, D., Freedman, M. J., & Vuchinich, R. E. (2005). To house or not to house: The effects of providing housing to homeless substance abusers in treatment. American Journal of Public Health, 95(7), 1259-1265. Study 3Milby, J. B., Schumacher, J. E., Vuchinich, R. E., Freedman, M. J., Kertesz, S., & Wallace, D. (2008). Toward cost-effective initial care for substance-abusing homeless. Journal of Substance Abuse Treatment, 34(2), 180-191.
Supplementary Materials Drake, R. E., McHugo, G. J., & Biesanz, J. C. (1995). The test-retest reliability of standardized instruments among homeless persons with substance use disorders. Journal of Studies on Alcohol, 56(2), 161-167.
Milby, J. B., & Schumacher, J. E. (2011). Quality assurance protocol. Unpublished manuscript.
Milby, J. B., Wallace, D., Ward, C. L., Schumacher, J. E., & Michael, M. (2005). Towards a more sensitive assessment of homelessness: The Homelessness Severity Scale. Journal of Social Distress and Homeless, 14(3&4), 151-169.
Schumacher, J. E., Milby, J. B., Wallace, D., Meehan, D.-C., Kertesz, S., Vuchinich, R., et al. (2007). Meta-analysis of day treatment and contingency-management dismantling research: Birmingham Homeless Cocaine Studies (1990-2006). Journal of Consulting and Clinical Psychology, 75(5), 823-828.
Schumacher, J. E., Milby, J. B., Wallace, D., Simpson, C., Frison, S., McNamara, C., et al. (2003). Diagnostic compared with abstinence outcomes of day treatment and contingency management among cocaine-dependent homeless persons. Experimental and Clinical Psychopharmacology, 11(2), 146-157.
Outcomes
| Outcome 1: Drug and alcohol abstinence |
|
Description of Measures
|
Drug and alcohol abstinence was measured with urinalysis, and participants' urine samples were tested for cocaine, marijuana, and alcohol. Two studies also tested urine samples for amphetamines, morphine, and benzodiazepines. If a participant admitted to using any other drugs at baseline, then all of the participant's subsequent urine samples were tested for those drugs as well.
All studies included the following measures of abstinence, which were constructed from participants' urine samples that tested negative for drugs and alcohol:
- Percentage of participants in each condition who had all negative urine samples for the week
- Longest period of consecutive negative urine samples (measured in weeks) achieved by each participant
In addition to these measures, one study also included the percentage of negative urine samples for each participant in the prior 60 days, and another study included the percentage of negative urine samples for each participant across a treatment timeframe.
If a participant failed to provide a urine sample for drug and alcohol testing, he or she was categorized as having a positive result--unless the participant was excused from that test for a valid reason (e.g., medical or social service appointment, illness).
|
|
Key Findings
|
In a 6-month randomized clinical trial (RCT), participants who were homeless and met DSM-III-R criteria for co-occurring cocaine abuse or dependence and nonpsychotic mental disorders were assigned to one of two treatment conditions: the intervention group, which received 2 months of Behavioral Day Treatment and Contingency Managed Housing followed by 4 months of weekly aftercare group meetings and Contingency Managed Housing and Work Therapy, or the comparison group, which received 2 months of Behavioral Day Treatment followed by 4 months of weekly aftercare group meetings. Urine samples were collected from participants twice weekly at random during months 1-2 of the study (excluding weekends and not on consecutive days). During months 3-6 of the study, urine samples were collected twice weekly: once during the regularly scheduled aftercare meeting and once on a random day (but not on consecutive days). Positive urine samples disputed by the participant were subjected to confirmation testing. To remain in housing and work in months 3-6, participants in the intervention group had to maintain negative urine samples (i.e., abstain from drug and alcohol use). Assessments occurred at baseline, at the end of Behavioral Day Treatment (2 months after baseline), and at the end of the study (6 months after baseline). Findings included the following:
- The percentage of days of abstinence from drug and alcohol use in the prior 60 days was higher for the intervention group than the comparison group at 2 months after baseline (71% vs. 41%; p < .0001) and at 6 months after baseline (41% vs. 15%; p = .0009).
- The average weekly percentage of participants who were abstinent from drug and alcohol use was higher for the intervention group than the comparison group across months 1-2 (69% vs. 42%; p < .0001) and across months 3-6 (54% vs. 18%; p = .007).
- At 2 months after baseline, the average number of consecutive weeks of abstinence from drug and alcohol use was higher for the intervention group than the comparison group (4.87 weeks vs. 2.84 weeks; p = .0004). This group difference was associated with a medium effect size (Cohen's d = 0.74).
- At 6 months after baseline, the average number of consecutive weeks of abstinence from drug and alcohol use was higher for the intervention group than the comparison group (9.51 weeks vs. 3.88 weeks; p = .0001). This group difference was associated with a large effect size (Cohen's d = 1.06).
- From 2 to 6 months after baseline, participants in the intervention group had a greater increase in consecutive weeks of abstinence from drug and alcohol use relative to participants in the comparison group (4.87 to 9.51 weeks vs. 2.84 to 3.88 weeks; p = .0001).
A 12-month RCT was conducted with participants who were homeless and met DSM-III-R criteria for co-occurring cocaine abuse and dependence and nonpsychotic mental disorders. During the 6-month treatment period, all participants received Behavioral Day Treatment (months 1-2) and weekly aftercare group meetings and non-abstinence-contingent work therapy (months 4-6); participants also received posttreatment follow-up, which consisted of continuing weekly aftercare group meetings (months 7-12). Urine samples from all participants were randomly collected and tested for drug and alcohol use twice weekly during months 1-6 and once weekly during months 7-12. If a test result was inconclusive, the original urine sample was retested once or twice, and the two most consistent results were used to determine the data point. Participants were assigned to one of three housing conditions:
- The Contingency Managed Housing group, which received rent-free housing in months 1-2 after two consecutive negative urine samples, followed by low-rent housing in months 3-6. Participants paid monthly rent with money earned through non-abstinence-contingent work therapy or other employment. If units were still available, participants could continue in the low-rent housing in months 7-12. To remain in housing in months 1-6, participants in this group had to maintain negative urine samples (i.e., abstain from drug and alcohol use); however, during months 7-12, housing was no longer contingent on the participants maintaining negative urine samples.
- The non-abstinence-contingent housing group, which received rent-free housing in months 1-2 after submitting two urine samples, followed by low-rent housing in months 3-6 with an option to continue in the low-rent housing in months 7-12. Participants paid monthly rent with money earned through non-abstinence-contingent work therapy or other employment. If units were still available, participants could continue in the low-rent housing in months 7-12.
- The no-housing group, which remained in shelters or other temporary housing throughout the 12-month study.
Assessments occurred at baseline, at the end of Behavioral Day Treatment (2 months after baseline), at the end of treatment (6 months after baseline), and at the end of the 6-month posttreatment follow-up (12 months after baseline). Findings included the following:
- Across months 1-2, the average weekly percentage of participants who were abstinent from drug and alcohol use was higher for both housing groups than the no-housing group (69% of the Contingency Managed Housing group and 60% of the non-abstinence-contingent housing group vs. 41% of the no-housing group; p < .0001). Also across months 1-2, the average weekly percentage of participants who were abstinent from drug and alcohol use was higher for each of the housing groups than the no-housing group: 69% of the Contingency Managed Housing group vs. 41% of the no-housing group (p < .0001) and 60% of the non-abstinence-contingent housing group vs. 41% of the no-housing group (p = .0015). However, no significant difference in the average weekly percentage of participants who were abstinent was found between the two housed groups across months 1-2.
- Across months 3-6, the average weekly percentage of participants who were abstinent from drug and alcohol use was higher for both housing groups than the no-housing group (46% of the Contingency Managed Housing group and 40% of the non-abstinence-contingent housing group vs. 18% of the no-housing group; p = .0001). Also across months 3-6, the average weekly percentage of participants who were abstinent from drug and alcohol use was higher for each of the housing groups than the no-housing group: 46% of the Contingency Managed Housing group vs. 18% of the no-housing group (p < .0001) and 40% of the non-abstinence-contingent housing group vs. 18% of the no-housing group (p = .0003). Although the average weekly percentage of participants who were abstinent was higher for the Contingency Managed Housing group than the non-abstinence-contingent housing group (46% vs. 40%), this finding was not significant.
- Across months 1-6, the average number of consecutive weeks of abstinence from drug and alcohol use was higher for the Contingency Managed Housing group than the no-housing group (8.32 weeks vs. 4.48 weeks; p = .0004). Although the average number of consecutive weeks of abstinence was higher for the non-abstinence-contingent housing group than the no-housing group (6.60 weeks vs. 4.48 weeks), this finding was not significant. In addition, although the average number of consecutive weeks of abstinence was higher for the Contingency Managed Housing group than the non-abstinence-contingent housing group (8.32 weeks vs. 6.60 weeks), this finding also was not significant.
An 18-month RCT was conducted with participants who were homeless and met DSM-IV criteria for co-occurring cocaine abuse and dependence and nonpsychotic mental disorders. Participants were assigned to one of two treatment conditions:
- The intervention group, which received Behavioral Day Treatment and Contingency Managed Housing and Work Therapy (months 1-6), which required participants to maintain negative urine samples (i.e., abstain from drug and alcohol use).
- The comparison group, which did not receive Behavioral Day Treatment but did receive Contingency Managed Housing and Work Therapy (months 1-6), which required participants to maintain negative urine samples (i.e., abstain from drug and alcohol use).
All participants received posttreatment follow-up, which consisted of continuing weekly aftercare group meetings during months 7-12 and recommended aftercare group meetings during months 13-18. Starting in month 7, participants had the opportunity to transfer to federally funded Shelter Plus Care housing, which provided low-rent housing on a space-available basis. Although the aftercare and posttreatment follow-up services were not contingent on participants maintaining negative urine samples, a missed or positive sample could result in the participant's removal from the Shelter Plus Care housing or placement at the bottom of the waiting list. Urine samples were collected at baseline, three times weekly (Monday, Wednesday, and Friday) during months 1-6, randomly once every week during months 7-12, and randomly once every 2 months during months 13-18. Findings included the following:
- The average percentage of participants who were abstinent from drug and alcohol use was higher for the intervention group than the comparison group across months 7-12 (49% vs. 34%; p < .005) and across months 13-18 (40% vs. 27%; p < .014). However, no significant difference in the average percentage of participants who were abstinent was found between the groups across months 1-6.
- Across months 1-12, the average number of consecutive weeks of abstinence from drug and alcohol use was higher for the intervention group than the comparison group (19.2 weeks vs. 13.9 weeks; p = .009).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 2: Homelessness |
|
Description of Measures
|
Homelessness was measured using a section of the Retrospective Interview for Housing, Employment, and Treatment History (RHETRO), which was adapted from the recent housing history section of the Personal History Form used by the Columbia University Center for Homelessness Prevention Studies. Participants were asked to provide the number of days they spent in the following four housing domains during the past 60 days:
- Literal homeless situations (i.e., indoor public place, subway or bus, abandoned building, car or other private vehicle, outdoor place, emergency shelter)
- Marginal homeless situations (i.e., hotel/motel, someone else's room, someone else's apartment or house, transitional housing)
- Institutional housing (i.e., long-term alcohol- and drug-free facility, hospital, nursing home, treatment or recovery program, jail or prison, corrections halfway house)
- Stable housing (i.e., own apartment or house, parent's/guardian's apartment or house, group home, boarding house or board-and-care facility, own room in someone else's home or apartment)
The percentage of days in stable housing in the past 60 days was calculated by dividing the total number of days in stable housing by 60 days (i.e., the sum of days in stable housing, days in institutional housing, and days homeless, including literal and marginal homeless situations).
|
|
Key Findings
|
In a 6-month RCT, participants who were homeless and met DSM-III-R criteria for co-occurring cocaine abuse or dependence and nonpsychotic mental disorders were assigned to one of two treatment conditions: the intervention group, which received 2 months of Behavioral Day Treatment and Contingency Managed Housing followed by 4 months of weekly aftercare group meetings and Contingency Managed Housing and Work Therapy, or the comparison group, which received 2 months of Behavioral Day Treatment followed by 4 months of weekly aftercare group meetings. Assessments occurred at baseline, at the end of Behavioral Day Treatment (2 months after baseline), and at the end of the study (6 months after baseline). At 6 months after baseline, the percentage of days in stable housing in the past 60 days was higher for the intervention group than the comparison group (59% vs. 4%; p = .016).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
76.4% Male
23.6% Female
|
82.7% Black or African American
17.3% White
|
|
Study 2
|
26-55 (Adult)
|
75.4% Male
24.6% Female
|
90.8% Black or African American
9.2% White
|
|
Study 3
|
26-55 (Adult)
|
72.8% Male
27.2% Female
|
94.2% Black or African American
5.8% White
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Drug and alcohol abstinence
|
3.2
|
2.9
|
3.2
|
2.3
|
2.6
|
3.5
|
2.9
|
|
2: Homelessness
|
2.8
|
2.7
|
3.2
|
2.5
|
2.1
|
3.5
|
2.8
|
Study Strengths The urine testing used in all three studies is an objective index of drug abstinence and has a long history of use in forensic settings; it has strong reliability and validity, with procedural checks that add to the validity of the test results. In all three studies, the use of several different, but convergent, abstinence measures added to the validity. In one study, evidence of criterion validity was provided for two of the abstinence measures derived from urine test data for participants with a global diagnosis of psychoactive substance use disorder by DSM-III-R criteria: longest period of consecutive negative urine samples (measured in weeks) achieved by each participant, and percentage of negative urine samples for each participant in the prior 60 days. In one study, homelessness was measured by self-report using a slightly modified interview (RHETRO) from a standardized assessment instrument (Personal History Form) that was previously used in multisite studies of homeless populations and had good overall reliability and face validity. Research assistants collecting the assessment data were blind to the treatment conditions, and intervention fidelity was supported by the use of highly structured manuals and a fidelity instrument (Behavior Criteria Observation Checklist), which were used by trained clinicians to rate intervention sessions at random on a quarterly basis for immediate corrective action and retraining of therapists if needed. The studies were randomized, longitudinal, and well designed to systematically test different components of a theory-based, multisystemic intervention with abstinence-contingent incentives. Random assignment in all three studies controlled for many confounds, and multiple statistical analyses were performed for remaining confounds to determine their impact on outcomes. All three studies used an intent-to-treat approach with appropriate statistical modeling of longitudinal data by generalized and mixed linear models.
Study Weaknesses The use of infrequent and slightly different random sampling protocols for urinalysis across time periods within each study and across related studies measuring the same outcome introduces a degree of error and reduces the measure's reliability (stability) in determining sustained drug abstinence. No sample test-retest or interrater reliability was presented for the RHETRO, and no independent or collateral verification of validity was presented for its use with a population with a co-occurring disorder diagnosis (including high psychiatric severity). In regard to urine testing, there was a moderate to high amount of missing data, varying by treatment group across time periods in all three studies. The statistical approach to incorporating varying levels of missing data into the analyses was limited to simple imputation, and there was no correction in the experimentwise alpha level for significant differences between groups as recommended for the numerous statistical comparisons performed in each of the three studies.
|
|
