The Practice-Based Evidence Corner
The Practice-based Evidence (PBE) Corner is intended to be a forum in which researchers, practitioners, program developers, and evaluators highlight programs and interventions that have some evidence of effectiveness, but which have not been evaluated with methods that assess the impact of different confounding variables and factors through the use of control groups. One purpose of the PBE Corner is to provide a forum to highlighting and discussing behavioral health programs that address high priority issues for SAMHSA and its stakeholders, but which rely on PBE research to support their claims. The PBE Corner is a place where important programs can receive an open hearing that balances the program's claims of effectiveness, with the limitations of the evidence. The PBE Corner is intended as a place programs that have a strong historical roots in a particular community but have, for valid reasons, never been evaluated using mainstream scientific methods, i.e., RCTs and QEDs, and it is a place where innovative programs that target special or underserved populations can make a case for why their approach is effective and deserves more attention.
The PBE corner is not designed as a repository or database for any program that does not meet the minimum criteria of the registry. Instead, the PBE Corner is intended to be a place in which the stakeholders can present finding and evidence on issues that are of critical importance, in part because they address the limitations of existing registries, and in part because the address the deficiencies in our broaden systems of behavioral health care.
The PBE Corner format will be similar to that of the other corners of the Learning Center. We will use a variety of resources, highlighted case studies, discussion forums, and a limited inventory of key programs designed to illustrate critical issues of equity and effectiveness in behavioral health programming.
What is Practice-based Evidence?
PBE has developed as both a complement to and a criticism of the rise of evidence-based programs and practices, and their influence on policy. PBE is complementary in that it often provides information on the operations and mechanisms of an intervention that conventional outcome evaluation ignores. PBE opens up the "black box" of an intervention to describe and assess how programs work in practice. PBE is also a criticism of EBPs, because promoters of PBE point out while RCTs and QEDs are able to limit potential biases in data collection and analysis, they also create serious ethnical issues when used to evaluate programs that target smaller underserved populations. Swisher (2010), in describing PBE, quotes Einstein: "Not everything that can be counted counts and not everything that counts can be counted."
As Swisher1 described it, with PBE, “the real, messy, complicated world is not controlled. Instead, real world practice is documented and measured, just as it occurs, ‘warts’ and all. It is the process of measurement and tracking that matters, not controlling how practice is delivered.” As such, PBE is bound to time and context. It describes characteristics of a specific population, a specific location or organizational setting, and a specific set of practitioners, and the results of a specific intervention or mechanism. PBE is not meant to be statistically representative of some broader universe, nor does it enable researchers to isolate causal variables using statistical hypothesis testing.
That being said, it would incorrect to assume that PBE does not lend itself to any generalizations or insights beyond context specific descriptions of activities. PBE explores the internal workings of a program and collects data on how programs affect participants. It integrates qualitative data with measurable outcome data to construct cases or sets of cases and enables researchers and practitioners to better understand a program in terms of recognized theories and change mechanisms.
PBE data is used for formative assessment as opposed to summative assessment; it is suggestive of an intervention’s effectiveness. It may be strongly suggestive, as when a particular pattern or link between variables is repeated frequently to the point that it seems obvious, or it may be weakly suggestive, as when a particular outlier case describes a pattern that deserves further exploration. For example, an evaluator might document the perceptions of key stakeholders, collect pre and post test data from program participants, and conduct ethnographic research, and then marshal and interpret the available evidence to thoughtfully argue how and why a program is or is not working. Others then judge the program based on that information and may then offer alternative interpretations of how the program works, based on alternative analyses of the data.
As a concept, PBE includes a broad array of data types, from qualitative data gathered through interviews, document review, non-systematic surveys, and ethnography, to descriptive statistics that describe the demographic characteristics of a target population, practitioners, and organizations using categories that are broadly accepted in social science research. For example, a researcher may describe a target population using standard taxonomies for race and ethnicity, socio-economic status, behavioral health diagnoses, and some accepted treatment outcome measures. In other instances, however, a researcher might collaborate with practitioners and program clients to develop outcomes, measures and characteristics that are specific to that program, and community context and which would not be recognized at meaningful to a broader community of practitioners.
Different types of PBE
In the PBE Corner, we will provide a roadmap through these different types of PBE, while situating the discussion of PBE within a broader behavioral health evaluation framework.
Return here in the future for more on types of PBE.
Resources for PBE and Cultural Competence in Evaluation
The Road to Evidence: The Intersection of Evidence-Based Practices and Cultural Competence in Children's Mental Health
This policy paper on the intersection of evidence-based practices (EBPs) and cultural competence grows out of recent research findings and policy trends that encourage greater use of EBPs in clinical programs with children, adolescents, and their families to improve the quality of care and outcomes of mental health interventions.
Using Practice-Based Evidence to Complement Evidence-Based Practice in Children's Mental Health
Practice-based evidence holds promise for expanding the realm of interventions that have objective support based in community values. However the construct needs more rigorous definition and parameters.
A Compendium of Proven Community-Based Prevention Programs
Highlights 79 evidence-based disease and injury prevention programs that have saved lives and improved health.
Systematic Review Methods
The Community Guide conducts systematic reviews of interventions in many topic areas to learn what works to promote public health.
The Effectiveness of Universal School-Based Programs for the Prevention of Violent and Aggressive Behavior
The results of this review provide strong evidence that universal school-based programs decrease rates of violence and aggressive behavior among school-aged children.
Evidence-Based Practice and Practice-Based Evidence
Understanding Cultural Conditions
Evaluators Paul Florin, Phillip Graham, and Roy Gabriel discuss the importance of understanding cultural context and conditions when designing and delivering technical assistance to support substance misuse prevention services.
Evidence Based Practices, Practice Evidence Based Practices, Practice Based Evidence and Community Based Evidence and Community Defined Evidence in Multicultural Defined Evidence in Multicultural Mental Health
Statistics, culture, practice definitions.
Community Defined Practices Webinar Series
The webinar series focuses on community-defined practices (CDPs) and promising practices in racial and ethnic communities (Asian/Pacific Islander, Native American, African American, and Latino).
What is Community Defined Evidence?
Evidence-Based Practices, Practice-Based Evidence, Community-Defined Evidence Practices Currently Being Used by Systems of Care
Practices in use by type and state.
1 Swisher, A. K. (2010). Practice-Based Evidence. Cardiopulmonary Physical Therapy Journal, 21(2), 4.