Course Map

Course Overview | Why We Evaluate | Types of Evaluation | Evaluation Designs | Research Quality | Three Stages of Evaluation | Hiring an External Evaluator | Managing an Evaluation | Evaluation Reporting | Summary

Course Overview

About This Course (page 1)

Intended Audience (page 1)

Course Technical Requirements (page 2)

Course Topics (page 3)

Learning Objectives (page 4)

↑ Top

Why We Evaluate

What Is Program Evaluation? (page 5)

Goals of Evaluation (pages 6-7)

Final Thoughts on Why We Evaluate (page 8)

↑ Top

Types of Evaluation

Determining Your Focus (page 9)

Process Evaluation (page 10)

Process Evaluation Questions (page 11)

Outcome Evaluation (page 12)

Outcome Evaluation Questions (page 13)

Impact Evaluation (page 14)

Impact Evaluation Questions (page 15)

↑ Top

Evaluation Designs

What Is Evaluation Design? (page 16)

Categories of Evaluation Design (page 17)

Preexperimental Design (page 18)

Posttest-Only Preexperimental Design (page 19)

Pretest and Posttest Preexperimental Design (page 20)

Pretest and Posttest Preexperimental Design With Follow-Up (page 21)

Quasi-experimental Design (page 22)

Types of Control Conditions in a Quasi-experimental Design (page 23)

Benefits and Challenges of Quasi-experimental Design (page 24)

Experimental Design (page 25)

Benefits and Challenges of Experimental Design (page 26)

How To Choose the Right Evaluation Design (page 27)

Design Considerations for NREPP Acceptance (page 28)

↑ Top

Research Quality

Why Research Quality Matters (page 29)

The NREPP Model (page 30)

About NREPP (page 31)

NREPP Rating Criteria (page 32)

NREPP Submission Requirements (page 33)

↑ Top

Three Stages of Evaluation

Beyond Exploration (page 34)

Planning (page 35)

Deciding on Evaluation Questions (page 36)

Defining the Study Sample and Group Assignment (page 37)

Choosing Outcomes (page 38)

Choosing Outcome Measures (page 39)

Using Existing Measures Vs. Home-Grown (page 40)

Measuring Fidelity (page 41)

Identifying Data Sources (page 42)

Quantitative Data (page 43)

Qualitative Data (page 44)

Calculating Statistical Power (page 45)

Factors That Affect Power (page 46)

Obtaining Institutional Review Board Approval (pages 47-48)

Data Collection (page 49)

Obtaining Participant Consent (page 50)

Informed Consent (page 51)

Assent (page 52)

Retaining Participants (page 53)

Managing and Securing Data (page 54)

The Data Collection Plan (page 55)

Data Management Activities and Responsibilities (page 56)

The Confidentiality Plan (page 57)

Tips for Maintaining Confidentiality (page 58)

Data Analysis (page 59)

Analytical Techniques (page 60)

Quantitative Analysis (page 61)

Qualitative Analysis (page 62)

Confounding Variables (page 63)

↑ Top

Hiring an External Evaluator

Do I Need an External Evaluator? (page 64)

Assess Your Readiness To Evaluate (page 65)

Advantages to Hiring an External Evaluator (page 66)

Challenges in External Evaluations (page 67)

Assessing Evaluator Credentials (page 68)

Ongoing Assessment (page 69)

Finding a Qualified Evaluator (page 70)

Potential Sources for Finding an External Evaluator (page 71)

↑ Top

Managing an Evaluation

Planning and Management (page 72)

Setting a Budget: Labor Costs (page 73)

Setting a Budget: Non-Labor Costs (page 74)

Soliciting Cost Proposals (page 75)

Funding Sources (page 76)

Timelines (page 77)

Factors Affecting the Timeline (page 78)

Monitoring Plan (page 79)

Contracts (page 80)

Creating a Positive Relationship (page 81)

Tips for Managing External Evaluations (page 82)

↑ Top

Evaluation Reporting

Purpose of Evaluation Reporting (page 83)

The Final Evaluation Report (page 84)

Tailoring the Report for Different Audiences (page 85)

Components of an Evaluation Report (page 86)

Dissemination of Findings (page 87)

↑ Top

Summary

Brief Summary of Concepts (page 88)

Concept Review (pages 89-90)

Conclusion (page 91)

↑ Top