Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Caplan, R. D., Vinokur, A. D., Price, R. H., & van Ryn, M. (1989). Job seeking, reemployment, and mental health: A randomized field experiment in coping with job loss. Journal of Applied Psychology, 74(5), 759-769. 
Price, R. H., van Ryn, M., & Vinokur, A. D. (1992). Impact of a preventive job search intervention on the likelihood of depression among the unemployed. Journal of Health and Social Behavior, 33(2), 158-167. Study 2Vinokur, A. D., Price, R. H., & Schul, Y. (1995). Impact of the JOBS intervention on unemployed workers varying in risk for depression. American Journal of Community Psychology, 23 (1), 39-74.
Vinokur, A. D., Schul, Y., Vuori, J., & Price, R. H. (2000). Two years after a job loss: Long-term impact of the JOBS program on reemployment and mental health. Journal of Occupational Health Psychology, 5(1), 32-47.
Supplementary Materials Little, R. J., & Yau, L. H. Y. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin's causal model. Psychological Methods, 3(2), 147-159.
Vinokur, A. D., & Price, R. H. (1999). JOBS II Preventive Intervention for Unemployed Job Seekers, 1991-1993: Southeast Michigan. Codebook for all parts. Volumes 1, 2, and 3. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. Retrieved from http://www.icpsr.umich.edu/icpsrweb/ICPSR/studies/2739/documentation
Vinokur, A. D., Price, R. H., & Caplan, R. D. (1991). From field experiments to program implementation: Assessing the potential outcomes of an experimental intervention program for unemployed persons. American Journal of Community Psychology, 19(4), 543-562.
Vuori, J., Price, R. H., Mutanen, P., & Malmberg-Heimonen, I. (2005). Effective group training techniques in job-search training. Journal of Occupational Health Psychology, 10(3), 261-275.
Outcomes
| Outcome 1: Social support |
|
Description of Measures
|
Social support was measured with an 8-item scale developed by Abbey, Abramis, and Caplan. Respondents were asked to characterize their relationship with their spouse or a person they see often and feel close to. They rated how much the following forms of social support were provided through the relationship, on a scale from 1 (not at all) to 5 (a great deal): "encouragement," "caring," "useful information," "listening," "saying things that raise self-confidence," "talking with the person when upset," "understanding," and "direct help."
|
|
Key Findings
|
Participants who received the JOBS training and remained unemployed reported no decline in social support from a significant other from 4 weeks to 4 months after the intervention, while unemployed participants in the comparison group, who received a self-help booklet, did report a decline in social support (p < .05).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 2: Sense of personal mastery |
|
Description of Measures
|
Sense of personal mastery consisted of the following measures:
- Job-seeking self-efficacy: Respondents rated, on a scale from 1 (not at all confident) to 5 (a great deal confident), how they felt they were at performing behaviors required for getting a job, such as using their social networks to obtain job leads, preparing resumes, and interviewing.
- Self-esteem: Respondents rated their degree of agreement or disagreement on a scale from 1 (strongly agree) to 5 (strongly disagree) with statements such as "I am able to do things as well as most other people," and "On the whole I am satisfied with myself." Items were based on Rosenberg's self-esteem scale.
- Locus of control: Using a 10-item index, respondents rated their degree of agreement or disagreement on a scale from 1 (strongly agree) to 5 (strongly disagree) with statements such as "What is going to happen will happen," "Trusting fate has never turned out well for me," and "Decisions could be made just as well by flipping a coin." Items were based on the original Rotter Internal-External Locus of Control scale.
|
|
Key Findings
|
In one study, at both 4 weeks (p = .02) and 4 months (p = .02) after the intervention, unemployed participants who received the JOBS training reported higher perceived self-efficacy in their job-seeking ability than unemployed comparison group participants, who received a self-help booklet. Additionally, among reemployed participants who received the JOBS training, the more sessions they attended, the greater was the increase in job-seeking self-efficacy (p < .01) and self-esteem (p < .05) from pretest to 4 weeks after the intervention.
In a second study, a personal mastery measure was constructed by computing the mean scores of the above three measures (job-seeking self-efficacy, self-esteem, and locus of control), following a confirmatory factor analysis. Participants who received the JOBS training reported significantly higher levels of personal mastery at both 2 months and 6 months after the intervention than unemployed comparison group participants, who received a self-help booklet (p < .01). These findings were associated with very small (Cohen's d = 0.19) and small (Cohen's d = 0.21) effect sizes, respectively.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 3: Mental health |
|
Description of Measures
|
Mental health was assessed with the following measures:
- Depression: Using an 11-item scale based on the depression subscale of the Hopkins Symptom Checklist, respondents indicated how much, on a scale from 1 (not at all) to 5 (extremely), they had been bothered or distressed in the past 2 weeks by various depression symptoms such as feeling blue, having thoughts of ending one's life, and crying easily.
- Quality of life: Using an 8-item scale, respondents indicated their satisfaction with each item (e.g., "How do you feel about your life as a whole?") on a scale from 1 to 7, where 1 = terrible, 2 = unhappy, 3 = mostly dissatisfied, 4 = mixed, 5 = mostly satisfied, 6 = pleased, and 7 = delighted.
- Distress symptoms: Using an 18-item index, respondents and their spouses/significant others rated on a scale from 1 (none of the time) to 5 (all of the time) how much of the time during the past 2 weeks the respondent experienced various symptoms such as restlessness, anxiety, and inattentiveness.
- Role and emotional functioning: Using a 15-item index, respondents and their spouses/significant others rated on a scale from 1 (very poorly) to 5 (exceptionally well) how well the respondent did in the last 2 weeks with respect to various roles and emotional tasks, such as handling responsibilities and daily demands, staying level-headed, and making the right decisions.
|
|
Key Findings
|
In two studies, a subgroup of participants was identified as being at higher risk for elevated depressive symptoms at follow-up based on higher combined scores on depressive symptoms, financial strain, and low assertiveness at pretest.
In the first study, controlling for participants' depression at pretest and the number of hours worked during the follow-up period, among participants identified as being at higher risk for a depression episode, those who received JOBS training had significantly lower levels of depression 4 weeks (p = .01), 4 months (p = .001), and 28 months (p = .01) after the intervention than high-risk participants in the comparison group, who received a self-help book . Further, among high-risk participants receiving the intervention, there was a lower number of participants who experienced a severe episode of depression during any follow-up period (p < .02) and a lower number of severe episodes observed (p < .01) than among the high-risk participants in the comparison group. Additionally, among reemployed participants who received the JOBS training, the more sessions they attended, the greater was the increase in quality of life (p < .05) from pretest to 4 weeks after the intervention.
In the second study, at both 2 months and 6 months after the intervention, high-risk respondents who received the JOBS training exhibited significantly lower levels of depressive symptomatology (p < .05 and p < .01, respectively) and distress symptoms (p < .06, measured at 6-month follow-up only) and higher levels of role and emotional functioning scores (p < .01, measured at 6-month follow-up only) than their high-risk counterparts in the comparison group, who received a self-help book.
The moderating effects of risk for depression that were found at 2- and 6-month follow-up were not found at 2-year follow-up. Compared with participants assigned to the comparison condition, those who received the JOBS training, regardless of risk for depression, had a lower level of depressive symptoms (p < .05) and higher level of role and emotional functioning (p < .05). Additionally, significantly fewer actually met a stringent criterion for a major depressive episode (MDE) diagnosis (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 4: Reemployment status and quality |
|
Description of Measures
|
In one study, participants were classified as reemployed if they reported working at least 20 hours per week and characterized the number of hours employed as "working enough." Persons working fewer than 20 hours per week and characterizing that amount as "not working enough" were categorized as not reemployed. All other responses were dropped from the analysis.
In a second study, reemployment was defined as described above, using both objective and subjective criteria; additionally, separate analyses were conducted on all participants, categorizing those working 20 hours per week or more as reemployed and those working fewer than 20 hours per week as unemployed.
|
|
Key Findings
|
In one study, at both 4 weeks and 4 months after the intervention, JOBS participants had a significantly higher reemployment rate than participants in the comparison group, who received a self-help booklet (p = .04 and p = .025, respectively). Furthermore, the percentage of persons who had found reemployment in what they characterized as their main occupation was higher for the experimental group at both 4-week (p = .004) and 4-month (p = .008) follow-up.
In a second study, participants who received the JOBS training had a significantly higher reemployment rate 2 months after the intervention, based on both objective criteria alone and the combination of objective and subjective criteria, relative to comparison group participants who received a self-help booklet (p < .05). By 6-month follow-up, only those participants in the JOBS intervention group who were at high risk for increased depression still had a significantly higher reemployment rate (based on both objective criteria alone and the combination of objective and subjective criteria) relative to the comparison group (p < .05).
High-risk participants who received JOBS training reported significantly higher incomes than high-risk participants in the comparison group at both 2-month and 6-month follow-up (p < .05). These findings were associated with small effect sizes (Cohen's d = 0.22 and 0.26, respectively).
At 2-year follow-up, relative to the comparison group, the JOBS training group had a significantly higher percentage of respondents who were currently employed for 20 or more hours per week (p < .01), working more hours per week (p < .01), earning a higher income per month (p < .01), and working more months during the past 12 months (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
54% Female
46% Male
|
85% Race/ethnicity unspecified
15% Black or African American
|
|
Study 2
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
55% Female
45% Male
|
76% White
21.5% Black or African American
2.5% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Social support
|
3.5
|
3.5
|
3.3
|
2.3
|
2.8
|
2.8
|
3.0
|
|
2: Sense of personal mastery
|
3.8
|
3.5
|
3.4
|
2.6
|
2.9
|
3.8
|
3.3
|
|
3: Mental health
|
3.8
|
3.6
|
3.4
|
2.9
|
2.9
|
3.8
|
3.4
|
|
4: Reemployment status and quality
|
2.6
|
2.8
|
3.4
|
2.5
|
2.8
|
3.8
|
3.0
|
Study Strengths For most outcomes, the authors used multi-item measures from standardized survey instruments with strong psychometric properties. The validity of the mental health measures was further demonstrated as both the participant and the participant's significant other provided ratings. The JOBS trainers delivering the intervention received extensive formal training using a standardized manual developed for the intervention; they were observed regularly and received immediate feedback. Follow-up rates were well within expectations for this type of study. Attrition analyses conducted by the authors suggest that differential attrition rates did not compromise the internal validity of the results. Threats to internal validity also were controlled through the experimental design (randomized controlled trial) by controlling for some variables in analyses (i.e., age, sex, occupation, education, income, and psychosocial state), and by including a random sample of dropouts. The types of analyses conducted (e.g., ANOVA, logistic regression, ordinary least squares regression, dose analysis) and sample size are both more than adequate. The use of an intent-to-treat approach maintained the integrity of the randomized study design.
Study Weaknesses One of the indicators of reemployment status was problematic in that it combined two different concepts: reported number of hours working and the respondent's perception of whether he or she was working enough. The latter of these measures is prone to bias, and the authors provided no reliability data for it. Respondents were included in analyses only if they were either working 20 or more hours and indicated this was enough (reemployed) or working fewer than 20 hours and indicated this was not enough (not reemployed). Implementation fidelity was measured using participants' perceptions of the training process, an approach that is less than ideal. Approximately half of those who were recruited failed to attend the intervention. There was no discussion of how, if at all, the authors treated missing data. It appears that the authors conducted analyses only on those participants for whom they had complete data.
|
|
