Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Amaro, H., Dai, J., Arevalo, S., Acevedo, A., Matsumoto, A., Nieves, R., et al. (2007). Effects of integrated trauma treatment on outcomes in a racially/ethnically diverse sample of women in urban community-based substance abuse treatment. Journal of Urban Health, 84(4), 508-522. 
Amaro, H., Larson, M. J., Zhang, A., Acevedo, A., Dai, J., & Matsumoto, A. (2007). Effects of trauma intervention on HIV sexual risk behaviors among women with co-occurring disorders in substance abuse treatment. Journal of Community Psychology, 35(7), 895-908.
Supplementary Materials Amaro, H., McGraw, S., Larson, M. J., Lopez, L., Nieves, R., & Marshall, B. (2004). Boston Consortium Services for Families in Recovery: A trauma-informed intervention model for women's alcohol and drug addiction treatment. Published simultaneously in Alcoholism Treatment Quarterly, 22(3/4), 95-119; and In B. M. Veysey & C. Clark (Eds.), Responding to physical and sexual abuse in women with alcohol and other drug and mental disorders: Program building (pp. 95-119). New York: Haworth Press.
McHugo, G. J., Kammerer, N., Jackson, E. W., Markoff, L. S., Gatz, M., Larson, M. J., et al. (2005). Women, Co-Occurring Disorders, and Violence Study: Evaluation design and study population. Journal of Substance Abuse Treatment, 28(2), 91-107.
Morrissey, J. P., Ellis, A. R., Gatz, M., Amaro, H., Reed, B. G., Savage, A., et al. (2005). Outcomes for women with co-occurring disorders and trauma: Program and person-level effects. Journal of Substance Abuse Treatment, 28(2), 121-133.
Morrissey, J. P., Jackson, E. W., Ellis, A. R., Amaro, H., Brown, V. B., & Najavits, L. M. (2005). Twelve-month outcomes of trauma-informed interventions for women with co-occurring disorders. Psychiatric Services, 56(10), 1213-1222.
Noether, C. D., Finkelstein, N., VanDeMark, N. R., Savage, A., Reed, B. G., & Moses, D. J. (2005). Design strengths and issues of SAMHSA's Women, Co-Occurring Disorders, and Violence Study. Psychiatric Services, 56(10), 1233-1236.
Outcomes
| Outcome 1: Substance use and related problem severity |
|
Description of Measures
|
Substance use and related problem severity were evaluated using the Addiction Severity Index (ASI), a semistructured interview that assesses seven domains: medical, legal, employment, drug, alcohol, family, and psychological functioning. Items addressing substance use ask the client about use of individual drugs in the past 30 days. Substance use measures from the ASI included illicit drug use, drug abstinence rate, and problem composite scores for the alcohol and drug domains, which range from 0.0 (no symptoms) to 1.0 (highest severity of symptoms). The drug abstinence rate was a measure derived from a self-report of 0 days of use for each of the substances addressed in the ASI.
|
|
Key Findings
|
Women receiving substance abuse treatment at one of five sites--three residential, one outpatient, and one methadone outpatient--were assigned to BCM and were compared with women receiving usual substance abuse treatment services at one of four comparable sites. Usual substance abuse treatment services typically consisted of weekly individual and daily group sessions and relapse prevention services for residential clients, a weekly group or individual session for outpatient clients, and daily dosing and a weekly individual or group session for methadone clients. Findings included the following:
- The proportion of women reporting any illicit drug use was lower in the BCM group than the comparison group at both the 6-month (21.3% vs. 48.2%; p < .001) and 12-month (17.3% vs. 40%; p < .001) follow-ups after controlling for baseline values.
- Women in both conditions reported decreased drug use problem severity (ASI drug composite score) from baseline across the 12-month follow-up period (p < .001). However, women receiving BCM had higher rates of reported abstinence than comparison women at the 6-month (67% vs. 38%; p < .0001) and 12-month (75% vs. 40%; p < .0001) follow-ups.
- Women in both conditions reported decreased alcohol use problem severity (ASI alcohol composite score) from baseline across the 12-month follow-up period (p < .0001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
| Outcome 2: Mental health symptomatology |
|
Description of Measures
|
Mental health symptomatology was evaluated using the Global Severity Index (GSI) from the Brief Symptom Inventory (BSI), a 53-item self-report checklist of symptoms grouped into depression, anxiety, and somatization subscales. Each item is rated for the prior week across a 5-point Likert scale from 0 (not at all) to 4 (extremely bothersome). The GSI score is the sum of the 53 item ratings, with higher scores indicating more severe mental health symptoms.
|
|
Key Findings
|
Women receiving substance abuse treatment at one of five sites--three residential, one outpatient, and one methadone outpatient--were assigned to BCM and were compared with women receiving usual substance abuse treatment services at one of four comparable sites. Usual substance abuse treatment services typically consisted of weekly individual and daily group sessions and relapse prevention services for residential clients, a weekly group or individual session for outpatient clients, and daily dosing and a weekly individual or group session for methadone clients. Women receiving BCM reported fewer mental health symptoms than comparison group women at the 12-month follow-up (p = .01), a difference associated with a small effect size (Cohen's d = 0.32).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 3: Posttraumatic stress symptoms |
|
Description of Measures
|
Posttraumatic stress symptoms were measured using the Posttraumatic Symptom Scale (PTSS), a semistructured interview of 17 items corresponding to symptoms associated with a DSM-IV clinical diagnosis of posttraumatic stress disorder (PTSD). The PTSS comprises three subscales to evaluate the level of reexperiencing, avoidance, and hyperarousal symptoms. Each item is rated on two 4-point scales, one for frequency and one for severity. Ratings range from 0-3, from "not at all" to "5 or more times per week" for frequency and from "not at all" to "very much" for severity.
|
|
Key Findings
|
Women receiving substance abuse treatment at one of five sites--three residential, one outpatient, and one methadone outpatient--were assigned to BCM and were compared with women receiving usual substance abuse treatment services at one of four comparable sites. Usual substance abuse treatment services typically consisted of weekly individual and daily group sessions and relapse prevention services for residential clients, a weekly group or individual session for outpatient clients, and daily dosing and a weekly individual or group session for methadone clients. Although reported PTSD symptoms decreased for both groups from baseline to the 6-month follow-up, these symptoms continued to decrease for BCM women from the 6- to the 12-month follow-up, while they increased during this time for comparison group women (p = .01). This difference was associated with a small effect size (Cohen's d = 0.35).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 4: HIV sexual risk behaviors |
|
Description of Measures
|
HIV sexual risk behaviors were measured using one item asking whether the client had engaged in unprotected sex during the 30-day period prior to assessment. "Unprotected sex" was defined as vaginal, oral, or anal sex without the use of a condom or other latex barrier with a main partner, another person or persons other than the main partner but not known to be in a high-risk sex group, or a known "risky" person or persons. A "risky" person was defined as someone who was HIV positive; someone with AIDS; someone who used injection drugs, used other drugs such as cocaine, or was high on any substance; or someone with whom the participant engaged in sex in exchange for money or drugs.
|
|
Key Findings
|
Women receiving substance abuse treatment at one of five sites--three residential, one outpatient, and one methadone outpatient--were assigned to BCM and were compared with women receiving usual substance abuse treatment services at one of four comparable sites. Usual substance abuse treatment services typically consisted of weekly individual and daily group sessions and relapse prevention services for residential clients, a weekly group or individual session for outpatient clients, and daily dosing and a weekly individual or group session for methadone clients. Findings included the following:
- From baseline to the 6-month follow-up, the percentage of women who reported engaging in unprotected sex decreased in the BCM group (34% to 29%) and increased in the comparison group (19% to 44%; p = .004).
- At the 6- and 12-month follow-up, comparison group women were, respectively, 2.8 and 4.5 times as likely as women receiving BCM to report engaging in unprotected sex (p values < .01), after adjusting for baseline demographics, participation in alcohol or drug treatment, illicit drug use, and residential/correctional living environment. This group difference was associated with medium effect sizes (odds ratio = 2.8 at 6 months and 4.5 at 12 months).
- African American and Hispanic women were nearly 3 times as likely to report engaging in unprotected sex as White women at the 6-month follow-up (p < .01), regardless of group assignment. This difference was associated with a medium effect size (odds ratio = 2.97). No differences by ethnicity were found at 12 months.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.1
(0.0-4.0 scale)
|
| Outcome 5: Perceived power in one's relationship |
|
Description of Measures
|
Perceived power in one's relationship was measured using the Sexual Relationship Power Scale (SRPS). This 23-item questionnaire comprises two subscales, Relationship Control and Decision-Making Dominance, and asks about relationships in the past 6 months. The Relationship Control subscale includes items such as "If I asked my partner to use a condom, he would get angry" and "I feel trapped or stuck in our relationship" and uses a 4-point Likert scale that ranges from 1 (strongly agree) to 4 (strongly disagree). The Decision-Making Dominance subscale includes items such as "Who usually has more to say about what you do together?" and "Who usually has more to say about how often you see one another?" and uses a 3-point categorical scale with ratings of 1 (your partner), 2 (both of you equally), and 3 (you). Scores on the two subscales are added to generate a total score, with higher total scores indicating more perceived power in the relationship.
|
|
Key Findings
|
Women receiving substance abuse treatment at one of five sites--three residential, one outpatient, and one methadone outpatient--were assigned to BCM and were compared with women receiving usual substance abuse treatment services at one of four comparable sites. Usual substance abuse treatment services typically consisted of weekly individual and daily group sessions and relapse prevention services for residential clients, a weekly group or individual session for outpatient clients, and daily dosing and a weekly individual or group session for methadone clients. Among women who reported being in at least one sexual relationship in the 6 months prior to follow-up:
- Women receiving BCM reported having greater power in their relationships than comparison group women at the 6-month (p < .01) and 12-month (p < .001) follow-ups.
- Women who said they had engaged in unprotected sex in the 6 months prior to follow-up reported lower perceived relationship power compared with women who said they had not engaged in unprotected sex at both the 6-month (p < .001) and the 12-month (p = .002) follow-ups, regardless of condition assignment.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.5
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
100% Female
|
35% Hispanic or Latino
35% White
26% Black or African American
5% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use and related problem severity
|
2.5
|
2.5
|
2.7
|
2.5
|
2.2
|
2.0
|
2.4
|
|
2: Mental health symptomatology
|
3.7
|
3.7
|
2.7
|
2.5
|
2.0
|
2.0
|
2.8
|
|
3: Posttraumatic stress symptoms
|
3.7
|
3.7
|
2.7
|
2.5
|
2.0
|
2.0
|
2.8
|
|
4: HIV sexual risk behaviors
|
1.7
|
1.7
|
2.8
|
2.5
|
2.0
|
2.0
|
2.1
|
|
5: Perceived power in one's relationship
|
2.9
|
2.9
|
2.8
|
2.5
|
2.0
|
2.0
|
2.5
|
Study Strengths The reliability and validity of the ASI have been established with different substance abuse treatment populations, and study sample reliabilities were provided for both the drug and alcohol composite score measures. The Brief Symptom Inventory and the Posttraumatic Symptom Scale have excellent psychometric properties, and 1-week test-retest sample reliabilities were high for both instruments. Interventionists received extensive standardized training with weekly reviews and ongoing supervision from senior clinicians throughout the study period to support intervention fidelity. All assessment interviews were administered by trained research assistants, not the interventionists. The number of clients enrolled in the study was sufficient to provide adequate statistical power to detect main group differences.
Study Weaknesses No study sample psychometrics were presented for self-reported abstinence, a measure derived from the ASI that may have been influenced by the higher proportion of court-ordered participants in the intervention than comparison group. The absence of sample psychometrics for the unprotected sex measure is an issue, since internal reliability and validity vary widely with different study populations. No data on sample reliability were presented for the Sexual Relationship Power Scale, a scale for which the underlying psychometrics are not well developed in the field. The study lacked a measurement instrument to quantify fidelity, which was weakened by the multidimensional nature of the intervention and multiple treatment and comparison sites. Lack of randomization, variability of available treatment services across the comparison sites, and length of the follow-up window (a 12-week period) created the possibility of confounding variables and challenged the underlying assumptions necessary to support the data analysis models employed.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Amaro, H. (2009). Client tracking system data elements and timetable. Boston, MA: Author.
Amaro, H. (2009). Co-morbidity screening form and protocol. Boston, MA: Author.
Amaro, H. (2009). Follow-up interviews with consumers. Boston, MA: Author.
Amaro, H. (2009). Job descriptions/staff qualifications. Boston, MA: Author.
Amaro, H. (2009). Protocol for integrating mental health and trauma clinical services into substance abuse treatment. Boston, MA: Author.
Amaro, H. (2009). Record abstraction form. Boston, MA: Author.
Amaro, H. (2009). TREM group facilitator preparation and supervision protocol. Boston, MA: Author.
Amaro, H., Melendez, M. P., & Melnick, S. (2005). Integrated substance abuse, mental health and trauma treatment with women, August 2005: A case study workbook for staff training. Boston, MA: The Institute on Urban Health Research.
Amaro, H., & Nieves, R. L. (2004). Economic success in recovery, June 2004: An educational group curriculum for women in recovery. Boston, MA: The Institute on Urban Health Research.
Amaro, H., & Nieves, R. L. (2004). Pathways to family reunification and recovery, June 2004: An educational group curriculum for women in recovery. Boston, MA: The Institute on Urban Health Research.
Amaro, H., Nieves, R. L., Paris, I., & Lopez, L. (2000). Service provider cross-training in substance abuse, mental health and trauma: A woman's trauma-informed service integration model. Boston, MA: Author.
Amaro, H., Nieves, R. L., & Saunders, L. (2004). Women's leadership training institute, June 2004: An educational group curriculum for women in recovery. Boston, MA: The Institute on Urban Health Research.
Educational materials and other products:
- Client resource card
- Fact sheet for consumers on co-occurring disorders
- Fact sheet for providers on co-occurring disorders
- PowerPoint training slides
- Women's Integrated Trauma and Substance Abuse Treatment Model Annual Families in Recovery Event booklet
- Women's Integrated Trauma and Substance Abuse Treatment Model newsletter
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.5
|
3.5
|
1.8
|
2.6
|
Dissemination Strengths Manuals for individual program components are well organized and include practical and detailed guidance for implementers. Customizable and flexible training options, as well as ongoing implementation support, are available. Training materials are comprehensive, informative, and well written, providing clear objectives and goals for each session. Program materials and training curricula establish clear standards for implementation. Suggestions for ensuring fidelity are provided to support quality assurance.
Dissemination Weaknesses While manuals for the individual program components are provided, there is little information articulating a clear plan for overall implementation. The developer does not provide continuing education to support ongoing proficiency in using the intervention. No comprehensive quality assurance protocol is available to assist new sites in monitoring target outcomes or implementation fidelity.
|
|
