Quality of Research
Review Date: December 2006
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1McFarlane, W. R., Link, B., Dushay, R., Marchal, J., & Crilly, J. (1995). Psychoeducational multiple family groups: Four-year relapse outcome in schizophrenia. Family Process, 34(2), 127-144.  Study 2McFarlane, W. R., Lukens, E., Link B., Dushay R., Deakins, S. A., Newmark, M., et al. (1995). Multiple-family groups and psychoeducation in the treatment of schizophrenia. Archives of General Psychiatry, 52(8), 679-687. Study 3McFarlane, W. R., Dushay, R. A., Stastny, P., Deakins, S. M., & Link, B. (1996). A comparison of two levels of Family-Aided Assertive Community Treatment. Psychiatric Services, 47(7), 744-750. Study 4McFarlane, W. R., Dushay, R. A., Deakins, S. M., Stastny, P., Lukens, E. P., Toran, J., et al. (2000). Employment outcomes in Family-Aided Assertive Community Treatment. American Journal of Orthopsychiatry, 70(2), 203-214. Study 5Dyck, D. G., Short, R. A., Hendryx, M. S., Norell, D., Myers, M., Patterson, T., et al. (2000). Management of negative symptoms among patients with schizophrenia attending multiple-family groups. Psychiatric Services, 51(4), 513-519. Study 6Dyck, D. G., Hendryx, M. S., Short, R. A., Voss, W. D., & McFarlane, W. R. (2002). Service use among patients with schizophrenia in psychoeducational multiple-family group treatment. Psychiatric Services, 53(6), 749-754.
Supplementary Materials Dixon, L., McFarlane, W. R., Lefley, H., Lucksted, A., Cohen, M., Falloon, I., et al. (2001). Evidence-based practices for services to families of people with psychiatric disabilities. Psychiatric Services, 52(7), 903-910.
Family Psychoeducation: Implementation Resource Kit. (n.d.). Developed through a contract from the Substance Abuse and Mental Health Services Administration's Center for Mental Health Services and a grant from The Robert Wood Johnson Foundation. Provided to NREPP in hard copy; available online at http://mentalhealth.samhsa.gov/cmhs/communitysupport/toolkits/family/default.asp
McFarlane, W. R. (2002). Empirical studies of outcome in multifamily groups. In W. R. McFarlane (Ed.), Multifamily groups in the treatment of severe psychiatric disorders (pp. 49-70). New York: Guilford.
McFarlane, W. R., Dixon, L., Lukens, E., & Lucksted, A. (2003). Family psychoeducation and schizophrenia: A review of the literature. Journal of Marital and Family Therapy, 29(2), 223-245.
McFarlane, W. R., Dushay, R., Lukens, E., Stastny, P., Deakins, S., & Link, B. (1999). Work outcomes in family-aided assertive community treatment: Vocational rehabilitation for persons with psychotic disorders. Epidemiologia e Psichiatria Sociale, 8(3), 174-182.
McFarlane, W. R., McNary, S., Dixon, L., Hornby, H., & Cimett, E. (2001). Predictors of dissemination of family psychoeducation in community mental health centers in Maine and Illinois. Psychiatric Services, 52(7), 935-942.
Outcomes
| Outcome 1: Employment among persons who have schizophrenia |
|
Description of Measures
|
Employment data were collected with the use of a dichotomous scale ("unemployed" or "engaged in work-related activities") for 2 months preceding the index admission and at 1- and 2-year follow-ups. This outcome was also measured using the Employment Tracking Form (ETF), a checklist developed for the intervention. The ETF was administered every 3 months, recording employment activity over the prior month. For analytic purposes, employment was collapsed into four categories: unemployed, sheltered work, unpaid work, and competitive work.
|
|
Key Findings
|
Two of three studies, all randomized controlled trials, reviewed found better employment outcomes for patients who were involved in PMFG when compared with patients who received other forms of treatment.
In the first study, no statistically significant differences were found in employment rates between patients in PMFG therapy and the single-family psychoeducation comparison group at baseline or 2 years later.
In a second study, PMFG participants had a higher rate of sheltered employment than Assertive Community Treatment (ACT) participants (18% vs. 6%, p < .05) over a period of 4 to 20 months after they became involved in PMFG, but not at 24 months. No differences were found in rates of competitive employment.
In a third study, PMFG treatment included Family-Aided Assertive Community Treatment (FACT), in which vocational specialists worked with patients to obtain competitive employment. Individuals involved in PMFG plus FACT were compared with individuals who received conventional assertive community treatment (CVR). PMFG plus FACT participants were more likely to be competitively employed than CVR participants at 12 months (37.1% vs. 7.7%, p < .01), at 18 months (26.5% vs. 8%, p < .05), and when data were averaged across 18 months (27.8% vs. 9.8%, p < .01). PMFG plus FACT participants also had higher average earnings than CVR participants over 18 months ($755 vs. $214 for CVR participants, p = .019). For individuals who had any competitive employment experience during the 18 months of the study, PMFG/FACT participant earnings averaged $1,448, compared with $320 for CVR participants (p = .038).
|
|
Studies Measuring Outcome
|
Study 2, Study 3, Study 4
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.0
(0.0-4.0 scale)
|
| Outcome 2: Psychiatric relapse |
|
Description of Measures
|
Relapse was defined in one study as the reemergence of symptoms of major psychosis and/or schizophrenia that persisted continuously for a minimum of 7 days over the 4 years of data collection; independent raters made the judgment of relapse. In another study, relapse was defined as the reemergence of psychotic symptoms, whether or not the episode required hospitalization.
|
|
Key Findings
|
Patients involved in PMFG demonstrated lower relapse and/or hospitalization rates than patients involved in other treatments across three studies reviewed.
In one study, PMFG patients had lower relapse rates than patients in single-family treatment (p < .05).
In a second study, PMFG patients who completed care had a lower relapse rate than patients who completed single-family treatment over a 2-year period (30% vs. 48%, p < .05).
A third study examined service use of individuals in PMFG treatment compared with individuals who received standard care. The results indicate that PMFG patients had lower rates of hospitalization in the year following baseline (p = .03) but did not differ in minutes of outpatient services used or the use of crisis care when compared with individuals receiving standard care (i.e., service costs did not simply shift from inpatient to outpatient care).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 6
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.5
(0.0-4.0 scale)
|
| Outcome 3: Symptoms of schizophrenia |
|
Description of Measures
|
Interviewers assessed positive and negative symptoms each month for 12 months using two instruments: the Brief Psychiatric Rating Scale (BPRS) and the Modified Scale for the Assessment of Negative Symptoms (MSANS). The BPRS contains 24 items assessing both positive and negative symptoms; the MSANS contains 30 items that assess negative symptoms. Positive symptoms include exaggerations of normal functions; negative symptoms include losses of normal functions. Interviewers participated in training sessions for rating accuracy and reliability.
|
|
Key Findings
|
PMFG patients had fewer negative symptoms of schizophrenia than patients receiving other forms of treatment in one of two studies reviewed for this outcome.
In one study, PMFG patients had lower levels of negative symptoms than patients receiving single-family psychoeducational treatment when discharged from the hospital. However, individuals in single-family treatment improved over the course of the 2-year study, while PMFG patients remained at the lower level of negative symptoms at discharge. There was no statistically significant difference between the two groups at 2-year follow-up, when both psychoeducational groups had very low levels of negative symptoms.
In another study, PMFG patients had lower levels of negative symptoms when compared with usual care patients (p < .05).
|
|
Studies Measuring Outcome
|
Study 2, Study 5
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.3
(0.0-4.0 scale)
|
| Outcome 4: Family stress |
|
Description of Measures
|
Family stress was measured as family burden and family well-being using the Social Adjustment Scale-III (SAS-III), Family Version.
|
|
Key Findings
|
Families involved in PMFG reported reduced family stress in a variety of areas. For example, 24 months after becoming involved in PMFG, families reported improvements in objective burden (p = .001) subjective burden (p < .001), dissatisfaction with the patient (p <.01), and lower levels of friction between the patient and others (p < .05). Twenty-two of 68 families did not provide data on these measures at baseline or follow-up.
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
1.8
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
70.7% Male
29.3% Female
|
Data not reported/available
|
|
Study 2
|
18-25 (Young adult)
26-55 (Adult)
|
73.3% Male
26.7% Female
|
52.9% White
40.7% Black or African American
4.7% Hispanic or Latino
1.7% Asian
|
|
Study 3
|
18-25 (Young adult)
26-55 (Adult)
|
64.7% Male
35.3% Female
|
77.9% White
14.7% Black or African American
5.9% Hispanic or Latino
1.5% Race/ethnicity unspecified
|
|
Study 4
|
18-25 (Young adult)
26-55 (Adult)
|
70% Male
30% Female
|
87% White
10% Race/ethnicity unspecified
3% Black or African American
|
|
Study 5
|
18-25 (Young adult)
26-55 (Adult)
|
73% Male
27% Female
|
95% White
5% Race/ethnicity unspecified
|
|
Study 6
|
18-25 (Young adult)
26-55 (Adult)
|
77% Male
23% Female
|
Data not reported/available
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Employment among persons who have schizophrenia
|
2.0
|
2.0
|
0.5
|
3.0
|
2.3
|
2.0
|
2.0
|
|
2: Psychiatric relapse
|
2.8
|
2.3
|
2.5
|
2.5
|
2.8
|
2.0
|
2.5
|
|
3: Symptoms of schizophrenia
|
2.0
|
2.0
|
2.0
|
2.5
|
2.8
|
2.5
|
2.3
|
|
4: Family stress
|
3.0
|
2.5
|
0.0
|
2.3
|
2.5
|
0.8
|
1.8
|
Study Strengths In some studies reviewed, implementation fidelity was addressed by videotaping the intervention and providing staff training and supervision. Attrition and missing data were high, but this is typical for the population studied. Relapse criteria were clearly articulated. The discussion sections of articles provided a thoughtful presentation of the limitations of the studies. The authors appear to have addressed possible confounds quite well, and the analyses selected were more than adequate for the time of publication.
Study Weaknesses Steps were taken to enhance fidelity of implementation in some studies, but there did not appear to be measures of fidelity pertaining to the employment interventions used. In addition, while some of the psychiatric measures are well known, the psychometric properties of the scales were not discussed. The Employment Tracking Form developed for this outcome had face validity, but its reliability and validity were not discussed; employment could have been verified by outside sources.
The authors provided a thoughtful discussion of potential confounds, but more could have been done to handle confounds. For example, in one study, the caseload was higher in the comparison group, and vocational specialists were not used as they were in a later study. In addition, the impact of out-of-group socializing for the families involved in PMFG was not fully discussed. More information about the types of jobs obtained, how family helped participants obtain jobs, and verification of jobs would have improved the studies.
The studies may have been underpowered to find some effects. Attrition was high in one study, and once relapse was accounted for, little was done to address attrition. In one study, for one measure, the problem of missing data was considerable, with 22 of 68 families not providing data at baseline or follow-up on family stress measures. In one study, it was not clear if drug abuse, emergency department visits, or social service use were counted as service use.
|
|
Readiness for Dissemination
Review Date: December 2006
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Family Psychoeducation: Implementation Resource Kit. (n.d.). Developed through a contract from the Substance Abuse and Mental Health Services Administration's Center for Mental Health Services and a grant from The Robert Wood Johnson Foundation. Provided to NREPP in hard copy; available online at http://mentalhealth.samhsa.gov/cmhs/communitysupport/toolkits/family/default.asp
Family Psychoeducation: National Implementing Evidence-Based Practices Project [DVD]
McFarlane, W., Downing, D., Lapin, M., Jacobson, L., Perry, K., & Amenson, C. (2002). Family Psychoeducation Workbook.
Psychoeducational Family Therapy Trainer Resource Disk [CD-ROM]
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
4.0
|
4.0
|
3.5
|
3.8
|
Dissemination Strengths Program materials are well organized and easy to understand. Materials provide thorough implementation guidance and examples of cultural adaptations. High-quality, audience-appropriate training materials are provided. Weekly consultation and assistance for organizational clinical supervisors are available. Fidelity and outcome measures are provided as part of the implementation resource kit to support ongoing quality assurance.
Dissemination Weaknesses The interview process for program fidelity measures seems complicated enough to require training and ongoing technical assistance in order to be used reliably.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Implementation resource kit
|
Free
|
Yes
|
|
Multifamily Groups in the Treatment of Severe Mental Illness (book)
|
About $20 each
|
Yes
|
|
3-day, on-site intensive training (includes Family Psychoeducation: National Implementing Evidence-Based Practices Project DVD, multifamily group training manual, clinical supervision for 1 year, videotape review for 1 year, monthly phone consultation for 1 year, competency assessment, and certification)
|
$1,200 per participant
|
Yes
|
Additional Information Direct financial costs of providing the intervention are about $350 per year per consumer in staff time for an ongoing multifamily group, assuming the use of a master's-level practitioner. Initial implementation costs are about $1,200 per practitioner for staff recruitment, preparation, and associated costs for training.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Asen, E., & Schuff, H. (2006). Psychosis and multiple family group therapy. Journal of Family Therapy, 28(1), 58-72.
Chien, W. T., & Chan, S. W. (2004). One-year follow-up of a multiple-family-group intervention for Chinese families of patients with schizophrenia. Psychiatric Services, 55(11), 1276-1284.
* Dyck, D. G., Hendryx, M. S., Short, R. A., Voss, W. D., & McFarlane, W. R. (2002). Service use among patients with schizophrenia in psychoeducational multiple-family group treatment. Psychiatric Services, 53(6), 749-754.
* Dyck, D. G., Short, R. A., Hendryx, M. S., Norell, D., Myers, M., Patterson, T., et al. (2000). Management of negative symptoms among patients with schizophrenia attending multiple-family groups. Psychiatric Services, 51(4), 513-519.
Kearney, M. S. (1985). A comparative study of multiple family group therapy and individual conjoint family therapy within an outpatient community chemical dependency treatment program. Dissertation Abstracts International, 45(12-B, Pt 1).
McDonell, M. G. (2005). Treatment integrity and client outcomes in multiple family group treatment. Dissertation Abstracts International, 66(1-B).
McDonell, M. G., Short, R. A., Berry, C. M., & Dyck, D. G. (2003). Burden in schizophrenia caregivers: Impact of family psychoeducation and awareness of patient suicidality. Family Process, 42(1), 91-103.
McFarlane, W. R., Dunne, E., Lukens, E., Newmark, M., McLaughlin-Toran, J., Deakins, S., et al. (1993). From research to clinical practice: Dissemination of New York State's family psychoeducation project. Hospital & Community Psychiatry, 44(3), 265-270.
Mullen, A., Murray, L., & Happell, B. (2002). Multiple family group interventions in first episode psychosis: Enhancing knowledge and understanding. International Journal of Mental Health Nursing, 11(4), 225-232.
Ruffolo, M. C., Kuhn, M. T., & Evans, M. E. (2005). Support, empowerment, and education: A study of multiple family group psychoeducation. Journal of Emotional and Behavioral Disorders, 13(4), 200-212.
Voss, W. D. (2003). Multiple family group treatment and negative symptoms in schizophrenia: Two-year outcomes. Dissertation Abstracts International, 63(11-B).
Norway and Denmark trials:
Jeppesen, P., Petersen, L., Thorup, A., Abel, M. B., Ohlenschlaeger, J., Christensen, T. O., et al. (2005). Integrated treatment of first-episode psychosis: Effect of treatment on family burden: OPUS trial. British Journal of Psychiatry, 187(Suppl. 48), s85-90.
Johannessen, J. O., Larsen, T. K., McGlashan, T., & Vaglum, P. (2000). Early intervention in psychosis: The TIPS project, a multi-centre study in Scandinavia. In B. Martindale, A. Bateman, M. Crowe, & F. Margison (Eds.), Psychosis: Psychological approaches and their effectiveness (pp. 210-234). London: Gaskell.
Nordentoft, M., Jeppesen, P., Abel, M., Kassow, P., Petersen, L., Thorup, A., et al. (2002). OPUS study: Suicidal behaviour, suicidal ideation and hopelessness among patients with first-episode psychosis. One-year follow-up of a randomised controlled trial. British Journal of Psychiatry, 181(Suppl. 43), s98-106.
Petersen, L., Jeppesen, P., Thorup, A., Abel, M.B., Ohlenschlaeger, J., Christensen, T. O., et al. (2005). A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ (British Medical Journal), 331(7517): 602.
Petersen, L., Nordentoft, M., Jeppesen, P., Ohlenschlaeger, J., Thorup, A., Christensen, T. O., et al. (2005). Improving 1-year outcome in first-episode psychosis: OPUS trial. British Journal of Psychiatry, 187(Suppl. 48), s98-103.
Thorup, A., Petersen, L., Jeppesen, P., Ohlenschlaeger, J., Christensen, T., Krarup, G., et al. (2005). Integrated treatment ameliorates negative symptoms in first episode psychosis--Results from the Danish OPUS trial. Schizophrenia Research, 79(1), 95-105.
|
|
|
