•  

Home > About NREPP > FAQs

Frequently Asked Questions (FAQs)

General Information

What is NREPP?

What does Quality of Research mean?

What does Readiness for Dissemination mean?

What is comparative effectiveness research?

Are environmental interventions/community-level interventions included in NREPP?

What is the recommended format for referencing an NREPP intervention summary?

How do I become a reviewer for NREPP?

How do I contact NREPP with a question or comment?

Using the NREPP Site

How do I decide which intervention to use?

How do I find an intervention?

How do I get more information on an intervention I found on the site?

Why are Spanish-language summaries provided for only some interventions?

Why are there two review dates in some intervention summaries?

The NREPP Review Process

Who rates the interventions reviewed?

How are interventions reviewed?

Does NREPP share the intervention summary with the program developer before publishing it on the NREPP Web site?

Can the program developer decline publication of the intervention summary on the NREPP Web site?

Can an intervention be reviewed more than once?

My intervention has been accepted for review. How can I refer to its NREPP status in my program materials?

Submitting an Intervention for Review

How do I find out if my intervention qualifies for submission?

Is there any cost to submit an intervention to NREPP?

Where can I find information on preparing a submission to NREPP?

Will every intervention that is submitted be reviewed?

Will there be another open submission period?

Can I submit multiple interventions for review?

Can interventions developed outside the United States be reviewed by NREPP?

Why aren't prevention and treatment interventions considered under separate criteria?

Information for Developers of NREPP-Reviewed Interventions

My intervention is included in the registry. How can I refer to the intervention's NREPP status in program materials?

Can I update my intervention summary?

General Information

What is NREPP?

The National Registry of Evidence-based Programs and Practices (NREPP) is a searchable online registry of interventions that support mental health promotion, substance abuse prevention, and mental health and substance abuse treatment. Through NREPP, users can identify and learn more about interventions that may meet their needs. All interventions that are included in NREPP have been voluntarily submitted, have met a set of minimum requirements, and have been assessed by independent reviewers.

back to top

What does Quality of Research mean?

Quality of Research (QOR) is one of two main categories of NREPP ratings. There are six QOR rating criteria: reliability of measures; validity of measures; intervention fidelity; missing data and attrition; potential confounding variables; and appropriateness of analysis. Together, these criteria assess the strength of evidence supporting the results or outcomes of the intervention. Because interventions may target multiple outcomes, and the evidence supporting the different outcomes may vary, each outcome is rated separately. Each intervention summary contains QOR ratings as well as a description of the study strengths and weaknesses that may have contributed to those ratings. For more information on the QOR ratings, see the Quality of Research page.

back to top

What does Readiness for Dissemination mean?

Readiness for Dissemination (RFD) is one of two main categories of NREPP ratings. There are three RFD rating criteria: availability of implementation materials; availability of training and support resources; and availability of quality assurance procedures. Together, these criteria assess the quality and availability of an intervention's training and implementation materials, that is, how easily the intervention can be implemented with fidelity using the materials and services that are currently available to the public. Each intervention summary contains RFD ratings as well as a description of the dissemination strengths and weaknesses that may have contributed to those ratings. For more information on the RFD ratings, see the Readiness for Dissemination page.

back to top

What is comparative effectiveness research?

The Federal Coordinating Council for Comparative Effectiveness Research defined comparative effectiveness research, or CER, as follows:

"Comparative effectiveness research is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in 'real world' settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.

  • To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations and subgroups.
  • Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, diagnostic testing, behavioral change, and delivery system strategies.
  • This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results."

This definition was published on page 16 of the Federal Coordinating Council's June 30, 2009, Report to the President and the Congress (PDF, 3.6MB).

Each intervention summary included in NREPP indicates whether or not the intervention was evaluated in CER studies (i.e., whether or not at least one of the studies included in the NREPP review met the definition of CER given above).

back to top

Are environmental interventions/community-level interventions included in NREPP?

Yes.

back to top

What is the recommended format for referencing an NREPP intervention summary?

NREPP recommends the following format for references:

[Program name]. Intervention summary retrieved on [date] from the Substance Abuse and Mental Health Services Administration's National Registry of Evidence-based Programs and Practices, [URL of summary]

For example:
Motivational Interviewing. Intervention summary retrieved on April 15, 2013, from the Substance Abuse and Mental Health Services Administration's National Registry of Evidence-based Programs and Practices, http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=130

back to top

How do I become a reviewer for NREPP?

Potential reviewers are recruited, selected, and approved by SAMHSA on the basis of their experience and areas of expertise. Quality of Research reviewers and Readiness for Dissemination reviewers are required to have specific expertise. Potential reviewers also must complete training on the procedures and criteria they will use to rate interventions. For more information, see the Reviewer Selection & Training page.

back to top

How do I contact NREPP with a question or comment?

Please email the NREPP staff at nrepp@samhsa.hhs.gov, call 1-866-436-7377, or visit the Contact Us page to use our online contact form.

back to top

Using the NREPP Site

How do I decide which intervention to use?

First, identify your community's needs so you can determine which type of intervention will be the most appropriate. Next, assess your organization's capacity (e.g., financial resources, organizational commitment, community buy-in) to implement an intervention with fidelity. Once you have this information, search NREPP or another registry to find an intervention that matches your community's needs and that can be implemented given your organization's capacity. Information on selecting and implementing an intervention can be found in NREPP's Learning Center — A Road Map for Implementing Evidence-Based Programs.

back to top

How do I find an intervention?

If you know the name of the intervention, you can either type it in the keyword Search box or select it from the View All Interventions list. From these pages on the Web site, you can access summaries describing each of the interventions in the registry.

You can also view a list of interventions that have been Accepted for Review. These interventions do not have summaries because their review process is not complete. If you sign up for e-updates, you will receive announcements with links to interventions recently added to the site.

If you do not know the name of the intervention, try entering keywords in the Search box or use the Advanced Search to select search criteria such as areas of interest, outcome categories, geographic location, age, race/ethnicity, setting, gender, and/or keyword.

back to top

How do I get more information on an intervention I found on the site?

For more information about an intervention included on this site, please contact the developer directly, using the contact information provided in the intervention summary. To assist you in gathering additional information, NREPP has compiled a list of Questions To Ask as You Explore the Possible Use of an Intervention (PDF, 54KB).

back to top

Why are Spanish-language summaries provided for only some interventions?

Spanish translations are provided for summaries of interventions for which the materials needed to deliver the intervention (e.g., treatment manuals, curricula, training guides, information for administrators) are available for dissemination in Spanish. To see a list of the summaries that have been translated into Spanish, go to the View All Interventions page, click Language Translations in the list of search criteria on the left, select Spanish, and click Search. You also can use the Language Translations criterion to refine a search after using the Basic Search or Advanced Search.

back to top

Why are there two review dates in some intervention summaries?

Interventions may undergo a new review if they have been included in the registry for 5 years and if new research and/or updated dissemination materials are available (see Can an intervention be reviewed more than once?). The new review may include a new Quality of Research (QOR) review, a new Readiness for Dissemination (RFD) review, or both. When an intervention undergoes a new review, the summary is updated to display the results of both the original and the new QOR and RFD reviews. Review dates are included to differentiate the information displayed from the two reviews.

back to top

The NREPP Review Process

Who rates the interventions reviewed?

NREPP contract staff assign two sets of two reviewers to independently review each intervention: one set for the Quality of Research review and one set for the Readiness for Dissemination review. For more information, see the Reviewer Selection & Training page.

back to top

How are interventions reviewed?

Each intervention is reviewed by two Quality of Research reviewers and two Readiness for Dissemination reviewers. Each reviewer receives a review packet of materials and information that has been assembled by NREPP staff and includes copies of studies and supporting documents (QOR reviews) or dissemination materials (RFD reviews). Working independently, each reviewer assesses the information presented in the packet using NREPP’s rating criteria. Each reviewer then completes a form in the packet, providing numerical QOR or RFD ratings as well as written comments discussing the strengths and weaknesses of the studies or the dissemination materials and processes. Completed packets are returned to NREPP staff, who prepare an intervention summary that includes the reviewers’ ratings and comments.

back to top

Does NREPP share the intervention summary with the program developer before publishing it on the NREPP Web site?

Yes. At the conclusion of the review, NREPP sends the program developer a draft intervention summary and asks him or her to provide comments. NREPP's response to these comments may include making factual corrections or other adjustments to clarify the results of the review. The summary is not published until the developer has approved the final version.

back to top

Can the program developer decline publication of the intervention summary on the NREPP Web site?

Yes, program developers have the option to decline publication of the summary. However, each NREPP review represents a considerable investment of time and public funds, and developers who agree to participate in a review are expected to authorize NREPP to publish the summary. In the event that a developer decides not to provide this approval, NREPP will include a notation on the Web site, indicating the intervention was reviewed but the developer declined to authorize publication of the summary. Although reviewed, these interventions will not be considered included in the registry.

back to top

Can an intervention be reviewed more than once?

An intervention reviewed by NREPP may be eligible for a new review in 5 years if certain requirements are met. The new review may include a new Quality of Research review, a new Readiness for Dissemination review, or both.

To qualify for a new Quality of Research review, one of the following must apply:

  • A new study or studies has been conducted to evaluate the same intervention with the same target population or a slightly modified intervention with the same or different target population.
  • A study or studies previously reviewed by NREPP has been updated with results from secondary analysis of the original dataset and/or analysis of longitudinal data.

All studies used in a Quality of Research review must demonstrate significant (p ≤ .05) findings related to substance abuse and/or mental health behavioral outcomes. The studies must be described in a peer-reviewed journal article or other professional publication (e.g., a book volume) or documented in a comprehensive evaluation report.

To qualify for a new Readiness for Dissemination review, the intervention's dissemination materials must have changed markedly, or new materials must have been developed, since the previous review was conducted.

NREPP plans to periodically invite cohorts of previously reviewed interventions to participate in new reviews, subject to the availability of funding.

back to top

My intervention has been accepted for review. How can I refer to its NREPP status in my program materials?

Program materials, including Web sites, can state that the intervention has been accepted for review by NREPP. A list of interventions that have been Accepted for Review appears on the NREPP Web site, and an intervention's inclusion on this list can be noted.

Please note that the acceptance of interventions for review and the inclusion of reviewed interventions on the NREPP Web site are not intended to convey endorsement, recommendation, or approval of these interventions by SAMHSA.

back to top

Submitting an Intervention for Review

How do I find out if my intervention qualifies for submission?

Requirements for submission are determined for each individual open submission period and announced in a Federal Register notice several months prior to the start of the open submission period. These notices are posted on the NREPP Newsroom page.

To receive announcements about future open submission periods, please sign up for our Submissions mailing list.

back to top

Is there any cost to submit an intervention to NREPP?

There is no fee to submit an intervention to NREPP. However, submitters must provide all the materials to be used in the review, including three copies of all dissemination materials, at their own expense.

back to top

Where can I find information on preparing a submission to NREPP?

This information can be found in NREPP's Learning Center — Preparing an Intervention for NREPP Submission and Potential Review. This course was created for developers of mental health or substance abuse programs, including principal investigators of relevant research studies, who are working toward submission to NREPP.

back to top

Will every intervention that is submitted be reviewed?

No. Each submission undergoes a preliminary assessment to determine whether it has met minimum requirements. Submissions that meet the minimum requirements may receive an invitation for review from SAMHSA, depending on the availability of current funding.

back to top

Will there be another open submission period?

Yes. Open submission periods are generally held annually. To receive announcements about open submission periods, please sign up for our Submissions mailing list.

back to top

Can I submit multiple interventions for review?

Yes. Each intervention must be submitted separately.

back to top

Can interventions developed outside the United States be reviewed by NREPP?

Yes, if they meet the minimum requirements and have implementation materials that can be disseminated in the United States.

back to top

Why aren't prevention and treatment interventions considered under separate criteria?

The developers and Federal sponsors of NREPP believe that the scientific standards for the evidence-based practice assessments contained within NREPP apply equally to the prevention and treatment components of behavioral health care.

back to top

Information for Developers of NREPP-Reviewed Interventions

My intervention is included in the registry. How can I refer to the intervention's NREPP status in program materials?

Program materials, including Web sites, can state that the intervention has been reviewed by NREPP and is included in SAMHSA's National Registry of Evidence-based Programs and Practices.

Please note that the acceptance of interventions for review and the inclusion of reviewed interventions on the NREPP Web site are not intended to convey endorsement, recommendation, or approval of these interventions by SAMHSA.

back to top

Can I update my intervention summary?

Developers may request updates to some parts of the summary after its publication on the NREPP Web site.

Updates are generally limited to the following sections:

  • Contact information
  • Costs
  • Implementation history
  • Adaptations
  • Replications

Updates cannot be made to any of the following: ratings, description of measures, key findings, strengths and weaknesses, outcomes, outcome categories, study populations, settings, and geographic locations.

To request an update, please contact the NREPP staff by email at nrepp@samhsa.hhs.gov. Please note that the request must come from the Principal designated for the NREPP review, or the Principal's authorization for the update must be documented in writing.

back to top