The NREPP Open Submission Process
The next NREPP open submission period is expected to take place in late 2017 or early 2018.
As detailed in the July 7, 2015 Federal Register notice
, SAMHSA generally holds one NREPP open submission period a year, but may hold more or less, depending on the number of reviews in progress and resources available. The next NREPP open submission period is expected to take place in late 2017 or early 2018. The dates of all open submission periods will be announced on the NREPP website and via NREPP’s listserv. Anyone wishing to join the NREPP listserv can subscribe
on the NREPP’s Contact Us page.
It is extremely important that those wishing to submit a program for review become familiar with the minimum requirements for submission. The minimum requirements, announced in the Federal Register
Notice of July 7, 2015, were revised in September 2017 to provide clarity. These requirements supersede those published in 2015.
To be considered for review, interventions must meet the following four minimum requirements. Applicants should be aware that meeting these criteria is necessary but not sufficient to ensure their program is eligible for review.
Outcomes and Populations. For a program to be eligible for review, it must assess a substance use or mental health outcome OR assess a behavioral health outcome of an intervention that targets a mental health or substance-using population. An outcome is the observed characteristic of the target population that an intervention is expected to change. For a list of eligible outcomes, please see the NREPP Outcomes Taxonomy.
Study Designs. A study design describes how the experimental and control groups are formed. To be eligible for NREPP, an intervention must have been evaluated with at least one study that is conducted using an experimental or quasi-experimental design. An experimental design requires that individuals or clusters (e.g., clinics) be randomly assigned to two or more groups. Similarly, a quasi-experimental design compares two or more groups but lacks the random assignment component to sort the units into groups. For NREPP, quasi-experimental designs are required to collect and report (at a minimum) baseline and posttest data for the originally assigned groups. Studies utilizing single-group, time series, or regression discontinuity designs are ineligible at this time, but will be considered under the emerging programs and practices section of the NREPP Learning Center.
Study Comparisons. Study comparisons refer to the comparison between intervention and control/comparison group(s). To rate the impact of a program, NREPP seeks comparisons that allow us to approximate the absolute effect of an intervention. Therefore, to be eligible for NREPP, a study must compare the intervention to an inactive control/comparison. Inactive comparisons include the following: no-treatment, wait-list group, treatment-as-usual, or an intervention that is presumed to be ineffective or substantially less effective than the intervention. In contrast, NREPP does not review active comparisons, where the comparison is expected to yield a relative effect. Active comparisons include comparison/control groups that receive an intervention expected to cause some change (e.g., an intervention rated as promising or effective on an evidence-based registry).
Other ineligible comparisons include those that compare (1) the effects of the same intervention on various subgroups (e.g., boys vs. girls) or subsets (e.g., pregnant women who are only part of the originally assigned sample) or (2) various (nonzero) doses or components of the same intervention.
Publication Quality. The results of these studies must be published in a peer-reviewed journal or professional publication, or documented in a comprehensive evaluation report, published in 1995 or after. Comprehensive evaluation reports must include, at a minimum, an introduction, a description of research methods (including information about study design, measures, and analysis plan), results (including test statistics), and discussion. Applicants are required to provide full-text documents at the time of submission that demonstrate the intervention meets these minimum requirements. Other research articles, published or unpublished evaluation reports, grant final reports, and replication studies may be submitted as additional supporting documentation. Note: Abstracts, posters, or links to partial articles are regarded as incomplete and will not be considered.
It should be noted that meeting minimum requirements does not automatically qualify a program for review. Additional screening criteria may reduce the range of evidence considered or exclude all research evidence from consideration.
NREPP further limits the review of interventions in some of the following ways.
Timeframe: Given that many interventions are associated with dozens of studies over decades of research, NREPP, with few exceptions, limits the evidence base for an intervention to no more than a 10-year span. The endpoint of the span is determined by the publication date of the most recent eligible study.
Outcome Measures: NREPP excludes measures that rely on single, subjective items. NREPP also excludes measures for which a standardized mean difference cannot be calculated (e.g., incident rates). NREPP excludes measures for which effect size data cannot be calculated from the information reported in the study or through data obtained directly from the study authors. NREPP excludes duplicate measures (e.g., subscale scores may be excluded if the total scale score is reported and the subscales measure the same outcome as the total scale).
Applicants are required to provide full-text documents at the time of submission that demonstrate the intervention meets these minimum requirements. Other research articles, published or unpublished evaluation reports, grant final reports, and replication studies may be submitted as additional supporting documentation.
Abstracts or links to partial articles are regarded as incomplete and will not be considered.
NREPP no longer requires programs to have developed implementation materials, training and support resources, and quality assurance procedures. However, programs and practices with such dissemination and/or implementation resources will be considered for prioritized review if the program is within the priorities established by the SAMHSA review process. The materials, along with the location of their availability, will be listed in the program or practice profile.
During the open submission process, applicants will be asked to submit dissemination and implementation information through the completion of a Resources for Dissemination and Implementation (RFDI) checklist. This information should include a brief narrative description of the materials that are being submitted. These materials may include, but are not limited to, treatment manuals, information for administrators, training for direct service staff, tested training curricula, training videos, brochures, presentations, mechanisms for ongoing supervision and consultation, protocols for gathering process and outcome data, ongoing monitoring of intervention fidelity, and processes for gathering feedback. Applicants should also provide the location (such as website url or address) where the materials can be obtained, as well as any costs, and whether the information is required or optional for implementation.
For more information on preparing a submission and to learn more about the new NREPP review process, see the July 7, 2015 Federal Register
notice and the Reviews and Submissions pages