The Race Matters Consortium web-site is made possible, in part, by a grant from the Casey Alliance on Racial Equity to the Rockville Institute.

Examining an evidence base

Evidence-based policy and practice is a concept that has received great attention in medicine and health care. According to the Institute of Medicine: Evidence-based medicine describes a diverse array of health care initiatives that seek to ensure that medical care received by patients is grounded in the best scientific knowledge and is appropriate for a given individual.

The Institute of Medicine goes on to identify such “evidence based practice” as a combination of the following three factors: (1) best research evidence, (2) best clinical experience, and (3) consistent with patient values.

This definition is easily applied to child welfare.

Evidence-based child welfare policy and practice seeks to ensure that children and families that are engaged with the child welfare system receive services that are based upon the best available scientific evidence and clinical experiences and are applied with recognition of and respect for the values of individual children and families (of all races, ethnicities and cultures).

According to NAPCWA, the Institute of Medicine definition builds “on a foundation of scientific research while honoring the clinical experience of child welfare practitioners, and being fully cognizant of the values of the families we serve”.

What is the Evidence Base?

Adapting a model developed by NAPCWA , there are several things one should consider before adopting a new practice, policy or intervention:

  1. Is the policy, practice or intervention under consideration based on a solid conceptual/theoretical framework?

    • Is the theory upon which it is based widely accepted?
    • Is there a logic model that makes sense in child welfare?

  2. What is the extent to which the policy, practice, or intervention can be replicated?

    • Are there practice manuals, protocols, and related written materials?
    • Is training/consultation available?
    • What races/ethnicities/culture or communities is the policy, practice or intervention designed for?
    • Does the practice lend itself to application in other race/ethnic or cultural communities or with other populations?

  3. How well is it supported by research?


    • What is the purpose of the research?
    • What is/was the impetus for the research who developed the research question and how (ie. researcher, researcher and community, community, managing or oversight board, etc).
    • How rigorous is the design?
    • How closely did the research subjects relate to a child welfare population?
    • What are the findings? How strong are the results?
    • How many evaluations have been conducted?
    • Are the results superior to other “usual services”?
    • Have there been systematic reviews/meta analyses?

  4. Does the policy, practice or intervention pose an acceptable risk to the child or family?

    • What is the benefit to risk of harm ratio?
    • Is it implemented within the context of the values of the race/ethnicity/culture of the children and families it serves?
    • Does this practice place children at undue risk?
    • Is this practice potentially more toxic than the
    • circumstances it tries to prevent or ameliorate?
Each policy, practice and intervention that is reviewed by the Race Matters Consortium will be examined on each of these four axes.
  1. Is the policy, practice or intervention under consideration based on a solid conceptual/theoretical framework?


  2. What is the extent to which the policy, practice, or intervention can be replicated?


  3. How well is it supported by research?


  4. Does the policy, practice or intervention pose an acceptable risk to the child or family?

Best available scientific evidence is a key concept and involves identifying and selecting interventions that have the strongest research evidence for their effectiveness. This recognizes that research varies in quality and this is often based on a number of factors including knowledge in the field at the time of the study and available resources. The following scheme for classifying interventions used by Barbara Thomlison will be used here. While we will classify the research as accurately as possible, users who are seriously considering implementing an intervention are encouraged to go to the source studies and double check our judgments.

Level 1: Well supported, efficacious treatment with positive evidence from more than two randomized clinical trials.


Level 2: Supported and probably efficacious treatment with positive evidence from two or more quasi-experimental studies, or where researchers found positive evidence from only one randomized clinical trial.


Level 3: Supported and acceptable treatment with positive evidence from comparative studies, correlations studies, and case control studies: one nonrandomized study; or any type of quasi-experimental study.


Level 4: Promising and acceptable treatment with evidence from experts or clinical experience or respected authority or both.


We use the concept of evidence-based policy and practice to identify and classify interventions so that users of this website can quickly judge the strength of the evidence and applicability of identified research. Users are strongly encouraged to double check the information presented to verify our reading of the research. To add in this we provide bibliographic information for all presented studies.


Research Bibliography

Roundtable on Evidence-Based Medicine. [On-line] available http://www.iom.edu/CMS/28312/RT-EBM.aspx

Institute of Medicine (2001). Crossing the Quality Chasm: A New Health System for the 21st Century, Washington DC: National Academy Press.

NAPCWA. (2005). Guide for Child Welfare Administrators on Evidence Based Practice. Washington DC: APHSA.

Thomlison, B. (2003). Characteristics of Evidence-Based Child Maltreatment Interventions. Child Welfare, 82 (5), 541-569.

 
UIUC School of Social Work