Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Dennis, M. (2002, August). 30 month findings from the Cannabis Youth Treatment (CYT) randomized field experiment. Presented at the 110th annual conference of the American Psychological Association, Chicago, IL.
Dennis, M., Godley, S. H., Diamond, G., Tims, F. M., Babor, T., Donaldson, J., et al. (2004). The Cannabis Youth Treatment (CYT) Study: Main findings from two randomized trials. Journal of Substance Abuse Treatment, 27(3), 197-213. 
Supplementary Materials Dennis, M. L., Funk, R., Godley, S. H., Godley, M. D., & Waldron, H. (2004). Cross-validation of the alcohol and cannabis use measures in the Global Appraisal of Individual Needs (GAIN) and Timeline Followback (TLFB; Form 90) among adolescents in substance abuse treatment. Addiction, 99(Suppl. 2), 120-128.
Dennis, M. L., Titus, J. C., Diamond, G., Donaldson, J., Godley, S. H., Tims, F. M., et al. (2002). The Cannabis Youth Treatment (CYT) experiment: Rationale, study design, and analysis plans. Addiction, 97(Suppl. 1), 16-34.
Diamond, G. S., Liddle, H. A., Wintersteen, M. B., Dennis, M. L., Godley, S. H., & Tims, F. (2006). Early therapeutic alliance as a predictor of treatment outcome for adolescent cannabis users in outpatient treatment. American Journal on Addictions, 15(Suppl. 1), 26-33.
French, M. T., Roebuck, M. C., Dennis, M. L., Diamond, G., Godley, S. H., Tims, F., et al. (2002). The economic cost of outpatient marijuana treatment for adolescents: Findings from a multi-site field experiment. Addiction, 97(Suppl. 1), 84-97.
Garcia, B. C. (Moderator), Leveille, T., Hamilton, N., Tims, F., & Vargo, M. (2006, March). Implementation of the Family Support Network in an operating program: A replication. Presented at the Joint Meeting on Adolescent Treatment Effectiveness, Washington, DC.
Hamilton, N., & Tims, F. (2008). The Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Substance Abuse Treatment (CSAT) Grant Programs Final Report for Family Support Network (FSN) of Juvenile Outpatient Program (JOP), Grant Number T113190 (Rev. ed.).
Tims, F. M., Dennis, M. L., Hamilton, N., Buchan, B. J., Diamond, G., Funk, R., et al. (2002). Characteristics and problems of 600 adolescent cannabis abusers in outpatient treatment. Addiction, 97(Suppl. 1), 46-57.
Outcomes
| Outcome 1: Abstinence from substance use |
|
Description of Measures
|
Abstinence from substance use was measured using the Global Appraisal of Individual Needs (GAIN) and was defined as the total number of abstinent days over four 3-month follow-up periods (3, 6, 9, and 12 months after intake). Abstinence was also defined as the percentage of participants who reported, at 30 months after intake, that they had either no past-month symptoms (short-term remission) or no past-year symptoms (sustained remission) of any substance abuse or dependence. The GAIN is a standardized, semistructured interview with eight main sections (background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational) that is designed to support the diagnosis, placement, and outcome monitoring of patients and the economic analysis of an intervention.
Self-report data were confirmed by urinalysis at intake and at follow-up assessments 3, 6, and 30 months after intake.
|
|
Key Findings
|
In two trials of a randomized controlled study, five manual-driven treatment interventions for adolescents with cannabis-related disorders were compared. Trial 1 compared the following interventions at two sites:
- FSN
- Motivational Enhancement Therapy/Cognitive Behavior Therapy with 5 sessions (MET/CBT5)
- MET/CBT with 12 sessions (MET/CBT12)
Trial 2 compared the following interventions at two sites:
- Adolescent Community Reinforcement Approach (A-CRA)
- MET/CBT5
- Multidimensional Family Therapy (MDFT)
In both trials, assessments were conducted at intake and at 3-, 6-, 9-, and 12-month follow-ups. Trial 1 additionally included a 30-month follow-up as part of the Persistent Effects of Treatment Studies--Adolescents (PETS-A) study. Findings from these trials included:
- Days of total substance abstinence reported by all participants increased 24% (from 52 to 65 days) between intake and the 3-month follow-up, with no significant differences by intervention. Similar gains were seen at each of the subsequent follow-ups (6, 9, and 12 months after intake), regardless of intervention, site, or trial participation. Across both trials, total days reported abstinent from all substances over 12 months ranged from 251 to 269 days, with no significant differences by intervention.
- At the 30-month follow-up, 60% of FSN participants reported no past-month substance abuse or dependence symptoms, compared with 51% for MET/CBT5 (trial 1) participants and 56% of MET/CBT12 participants, but this finding was not statistically significant.
- At the 30-month follow-up, 33% of FSN participants reported no past-year substance abuse or dependence symptoms, compared with 23% of MET/CBT5 (trial 1) participants and 27% of MET/CBT12 participants, but this finding was not statistically significant.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.7
(0.0-4.0 scale)
|
| Outcome 2: Recovery from substance use |
|
Description of Measures
|
Recovery from substance use was measured by the percentage of adolescents living in the community (as opposed to living in a correctional facility, inpatient treatment program, or other controlled environment) and reporting no past-month substance use, abuse, or dependence problems at follow-up interviews 3, 6, 9, 12, and 30 months after intake. For the 6% of adolescents who did not complete a 12-month follow-up, data from their last follow-up were used to determine their recovery status.
|
|
Key Findings
|
In two trials of a randomized controlled study, five manual-driven treatment interventions for adolescents with cannabis-related disorders were compared. Trial 1 compared the following interventions at two sites:
- FSN
- Motivational Enhancement Therapy/Cognitive Behavior Therapy with 5 sessions (MET/CBT5)
- MET/CBT with 12 sessions (MET/CBT12)
Trial 2 compared the following interventions at two sites:
- Adolescent Community Reinforcement Approach (A-CRA)
- MET/CBT5
- Multidimensional Family Therapy (MDFT)
In both trials, assessments were conducted at intake and at 3-, 6-, 9-, and 12-month follow-ups. Trial 1 additionally included a 30-month follow-up as part of the Persistent Effects of Treatment Studies--Adolescents (PETS-A) study. Findings from these trials included:
- Across interventions and sites in both trials, the percentage of adolescents reporting being in recovery increased from an average of 3% at intake to an average of 24% at the 3-month follow-up, where it remained across the 6-, 9-, and 12-month follow-ups.
- The percentage of participants who reported being in recovery at the 12-month follow-up showed small differences by intervention assignment, but after controlling for site and recovery status in the month prior to intake, the differences were not significant. Twenty-two percent of FSN participants reported that they were in recovery at the 12-month follow-up, compared with 28% of MET/CBT5 (trial 1) participants, 17% of MET/CBT12 participants, 34% of A-CRA participants, 23% of MET/CBT5 (trial 2) participants, and 19% of MDFT participants.
- At the 30-month follow-up, 20% of FSN participants reported being in recovery during the previous month, compared with 25% of MET/CBT5 (trial 1) participants and 26% of MET/CBT12 participants, but this finding was not statistically significant.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.7
(0.0-4.0 scale)
|
| Outcome 3: Cost effectiveness |
|
Description of Measures
|
Cost effectiveness was measured by the average daily cost of an adolescent achieving abstinence from substance use and the cost of an adolescent being in recovery 12 months after intake. Abstinence from substance use was measured using the GAIN and was defined as the total number of abstinent days during the 12-month period following intake. Recovery was defined as living in the community (as opposed to living in a correctional facility, inpatient treatment program, or other controlled environment) and reporting no past-month substance use, abuse, or dependence problems at the 12-month follow-up. Cost estimates were based on data collected using the Drug Abuse Treatment Cost Analysis Program (DATCAP), which measures both the accounting and opportunity costs of a substance abuse treatment program based on standard economic principles. DATCAP was supplemented with service contact logs completed by therapists and case managers.
|
|
Key Findings
|
In two trials of a randomized controlled study, five manual-driven treatment interventions for adolescents with cannabis-related disorders were compared. Trial 1 compared the following interventions at two sites:
- FSN
- Motivational Enhancement Therapy/Cognitive Behavior Therapy with 5 sessions (MET/CBT5)
- MET/CBT with 12 sessions (MET/CBT12)
Trial 2 compared the following interventions at two sites:
- Adolescent Community Reinforcement Approach (A-CRA)
- MET/CBT5
- Multidimensional Family Therapy (MDFT)
In both trials, assessments were conducted at intake and at 3-, 6-, 9-, and 12-month follow-ups. Findings from these trials included:
- In trial 1, the average cost per day of abstinence over the 12 months of follow-up was $8.79 and varied significantly by intervention assignment, with FSN costing $15.13 per participant, significantly more than either MET/CBT5 or MET/CBT12 ($4.91 and $6.15, respectively, p < .05). This cost difference was associated with a large effect size (Cohen's f = 0.48). The higher cost per day of abstinence for an FSN participant compared with either an MET/CBT5 or MET/CBT12 participant was consistent across both sites, with site 1 costs being $17.04 versus $5.75 and $7.67, respectively (p < .05), and site 2 costs being $13.80 versus $4.17 and $5.00, respectively (p < .05). These cost differences by site were associated with large effect sizes (Cohen's f = 0.40 for site 1 and Cohen's f = 0.63 for site 2).
- In trial 1, the average cost per participant in recovery at the 12-month follow-up was $8,846 and varied significantly by intervention assignment, with FSN costing $15,116 per participant, significantly more than either MET/CBT5 or MET/CBT12 ($3,958 and $7,377, respectively, p < .05). This cost difference was associated with a large effect size (Cohen's f = 0.72). The higher cost per day of recovery for an FSN participant compared with either an MET/CBT5 or MET/CBT12 participant was consistent across both sites, with site 1 costs being $18,284 versus $3,495 and $9,257, respectively (p < .05), and site 2 costs being $12,899 versus $4,369 and $5,914, respectively (p < .05). These cost differences by site were associated with large effect sizes (Cohen's f = 0.67 for site 1 and Cohen's f = 0.81 for site 2).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
83% Male
17% Female
|
61% White
30% Black or African American
5% Race/ethnicity unspecified
4% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Abstinence from substance use
|
4.0
|
3.8
|
3.9
|
3.9
|
3.1
|
3.8
|
3.7
|
|
2: Recovery from substance use
|
3.9
|
3.8
|
3.9
|
3.9
|
3.1
|
3.6
|
3.7
|
|
3: Cost effectiveness
|
3.5
|
3.3
|
3.9
|
3.5
|
3.0
|
3.7
|
3.5
|
Study Strengths The self-report measure for abstinence, the GAIN, has robust psychometric properties and was supplemented by collateral reports, urinalysis, and treatment records at baseline and at the 3-, 6-, and 30-month follow-ups. The study had strong intervention and assessment fidelity and achieved a complete dataset through excellent follow-up rates at 12 and 30 months after intake. The analysis thoroughly addressed the limited data loss.
Study Weaknesses The absence of any type of control group makes it difficult to interpret the effect of the intervention. Cost outcome measures were restricted to a narrowly defined assessment over less than 4 months and did not take into consideration the differential costs of expert judgments, professional personnel, or region-specific cost of living expenses.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Certification overview
Coleman, M., & Hamilton, N. (2008). MET/CBT clinical supervisors training. Largo, FL: Operation PAR.
Coleman, M., Welsh, T., & Hamilton, N. (2008). Motivational interviewing, motivational enhancement therapy, cognitive behavioral therapy (MET/CBT) training. Largo, FL: Operation PAR.
Cost sheet for training
Definition of completion
Hamilton, N. (2008). MET/CBT post training and coaching evaluation questions: CEO version. Largo, FL: Operation PAR.
Hamilton, N. (2008). MET/CBT post training and coaching evaluation questions: Counselor/therapist version. Largo, FL: Operation PAR.
Hamilton, N. (2008). MET/CBT post training and coaching evaluation questions: Supervisor version. Largo, FL: Operation PAR.
Hamilton, N. (2008). Motivational interviewing: Effective communication to achieve goals [PowerPoint slides]. Largo, FL: Operation PAR.
Hamilton, N. (2008). Family Support Network [PowerPoint slides]. Largo, FL: Operation PAR.
Hamilton, N. (2008). Goals of cognitive behavioral therapy: Restructuring thoughts, perceptions, & beliefs [PowerPoint slides]. Largo, FL: Operation PAR.
Hamilton, N. (2008). Motivational enhancement therapy & cognitive behavioral therapy: A brief introduction [PowerPoint slides]. Largo, FL: Operation PAR.
Hamilton, N. (2008). Motivational enhancement therapy: Facilitating change [PowerPoint slides]. Largo, FL: Operation PAR.
Hamilton, N. (2008). Sample training contract. Largo, FL: Operation PAR.
Hamilton, N. L., Brantley, L. B., Tims, F. M., Angelovich, N., & McDougall, B. (2001). Cannabis Youth Treatment (CYT) Series: Volume 3. Family Support Network for adolescent cannabis users [DHHS Pub. No. SMA 05-4103]. Rockville, MD: Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration.
Supervisor Ratings of Therapist Skillfulness. (1999).
Therapist/Counselor Competency Model
Therapist Skillfulness Rating Scale Definitions
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.3
|
3.8
|
3.5
|
3.5
|
Dissemination Strengths The program manual is highly detailed and includes tips, goals, and instructions for each component of the intervention. Initial, booster, and organizational supervisor training support the implementation of this intervention, with additional coaching also available. Organizational supervisors are trained to use videotapes or audiotapes to evaluate adherence to the model and improve clinical quality. A certification process is also available to support quality assurance.
Dissemination Weaknesses Some implementation materials appear to be in draft form. It is unclear how organizational readiness is assessed. Some training slides are wordy and redundant. The overall training content addressing individual intervention components may be difficult to grasp in the short training time allotted. The overall plan for quality assurance is unclear.
|
|
