Select an Evidence Based Program

Section 1.0

What Does it Mean for a Program to be Evidence-based?

Health promotion programs that have been found to produce positive outcomes based on the results of rigorous evaluations are often termed “evidence-based.” To be identified as an evidence-based program (EBP), an intervention or program must be thoroughly evaluated by researchers who are able to attribute positive outcomes to the intervention itself.

When you look at various programs to see if they are evidence-based, you will come across many evaluation study designs (see Table 1). You do not need to be an expert in research methods to understand these study designs, but it is useful to understand some basic terms. The following terms are used when describing participants in studies.

  • Experimental group — Individuals in the experimental group are taking part in the program that is being evaluated in the study.
  • Comparison group — Individuals in the comparison group are not taking part in the program that is being evaluated. They may not be enrolled in any program or they may be enrolled in some alternative program. Members of the comparison group may or may not be similar in characteristics to the members of the experimental group (see Table 1 for an example).
  • Control group — Individuals in the control group are not taking part in the program that is being evaluated; however, they may be enrolled in some alternative program. Members of the control group are likely similar in characteristics to members of the experimental group (see Table 1 for an example).
Table 1 - Evaluation study designs
Type and description Example
Non-experimental
Includes an experimental group, but no control or comparison group.
Researchers develop a new exercise program for seniors. They enroll all of the seniors at Senior Center A in the program (this is an experimental group). Both before and after the program, the researchers administer a survey that assesses the seniors’ confidence in regards to exercising. The researchers find that the seniors have greater exercise confidence after completing the program.

Because the researchers did not use a comparison or control group, they do not know if the same outcome (i.e., increased exercise confidence) would have occurred if they had used a traditional exercise program.
Quasi-experimental
Includes an experimental group and a comparison group.
Researchers develop a new exercise program for seniors. They enroll all of the seniors at Senior Center A in the new program (this is the experimental group). They enroll all of the seniors at Senior Center B in a traditional exercise program (this is the comparison group). The researchers administer a survey that assesses confidence in regards to exercising to all of the seniors both before and after the programs. The researchers find that the seniors at Senior Center A see a greater increase in their exercise confidence than the seniors at Senior Center B.

At first glance, it appears as though the new exercise program produced a superior outcome. However, this may not be the case. The seniors enrolled in the exercise programs were not randomly assigned to the new or traditional program; they were assigned based on which senior center they attended. If the seniors at Senior Center A are younger or have fewer medical conditions than those at Senior Center B, then these group differences may account for the superior outcome at Senior Center A.
Experimental
Includes an experimental group and a control group.
Researchers develop a new exercise program for seniors. The names of all of the seniors at Senior Center A and Senior Center B are placed in a large bowl. The names are mixed up and then half of the names are drawn out of the bowl. These seniors are enrolled in the new exercise program (this is the experimental group). The seniors whose names remain in the bowl are enrolled in a traditional exercise program (this is the control group). The researchers administer a survey that assesses confidence in regards to exercising to all of the seniors both before and after the programs. The researchers find that the seniors enrolled in the new exercise program see a greater increase in exercise confidence than the seniors enrolled in the traditional exercise program.

The research indicates that the new exercise program produces a superior outcome. Because the seniors were randomly assigned to the new or traditional program, it is unlikely the participants in the two programs differed significantly in terms or their ages or medical conditions. Therefore, group differences likely did not play a role in the outcomes. Researchers can attribute the greater increase in exercise confidence among those enrolled in the new exercise program to the components of the program.
Figure 1.
Basic criteria for evidence-based programs

1. Researchers find the program to have positive outcomes in an evaluation study

2. The positive outcomes can be attributed to components of the program itself

3. The evaluation study is peer-reviewed by experts who are knowledgeable about the topic

4. The program is approved or endorsed by a research organization or federal agency

When evaluation researchers have identified evidence supporting a particular program, they will often publish their findings in peer-reviewed scientific journals. Publishing their findings allows experts in the field who are not associated with the evaluation to examine the evaluation and determine if they agree with the methods used and with the conclusions drawn about the effects of the program. Evaluation researchers may also submit evidence to research organizations and federal agencies that will examine the evidence and approve or endorse the programs they find to have solid bases of evidence (see Figure 1). This approval or endorsement communicates to others in the field that these programs have met various standards of effectiveness (see Identifying Evidence-Based Interventions for more information).

Glossary of Terms

Toolkit Glossary

scroll to see all

Adaptation– Making changes to an evidence-based program in order to make it more suitable for a particular organization and/or audience.

Baseline – A starting point. In evidence-based programming, the term “baseline” is usually used in the context of data collection, where baseline data is data collected before a program is implemented.

Credentials – A testimony of qualification, competence, or authority issued to an individual by a third party. Examples of credentials include academic diplomas, academic degrees (e.g., MSW, MPH, PhD), licenses (e.g., MD, RN, LCSW), certifications (e.g., CHES, CPR, first aid), and security clearances.

Evidence – Facts or testimony in support of a conclusion, statement, or belief. In some settings, individuals may refer to “levels of evidence” or “types of evidence.” These terms will have specific definitions unique to the setting in which they are used. When referring to evidence-based programs, the term “evidence” is generally used to describe the findings or results of program evaluation studies.

Evidence-based program – A program that has been thoroughly evaluated by researchers who determined it produces positive outcomes.

Fidelity – The extent to which a program is being implemented as its developers intended for it to be.

Implementation – Putting into action or carrying out a program.

Instrument – A measurement tool. Instruments can take many forms including biomedical equipment (e.g., glucometer, blood pressure monitor, weight scale), pencil and paper tests, questionnaires, and interviews. A thermometer is an instrument used to measure body temperature. Likewise, a survey is an instrument that can be used to measure anxiety.

Medicaid – A publically-funded health insurance program for individuals who have low incomes and fall into certain categories of eligibility.

Objectives – Specific, measurable steps that can be taken to achieve goals.

Peer review – When experts review a professional’s performance, research, or writings. Peer review is a way that qualified professionals self-regulate their professions. Performance, research, or writings that pass the peer review process have increased credibility or trustworthiness.

Program champion – An individual who advocates for a program.

Quality assurance- A collection of planned, systematic activities applied to ensure that the components of a program are being implemented well.

Secondary data – Previously collected data that is being used for a purpose other than that for which it was originally collected.

Theory of behavior change – An attempt to explain how and why people change their behaviors. Researchers typically generate theories of behavior change from research in the fields of psychology, education, health, and other social sciences. When developing evidence-based programs, researchers will select a theory or components from several theories to guide program development.

Audience – The individuals for whom you implement your program. Depending on your setting, these individuals may also be referred to as a target population, population of interest, or clientele.

Buy-in – Typically used in the business world, buy-in refers to a financial exchange. In the context of health programs, the buy-in of stakeholders (community members, organizational leaders, participants, etc.) is generally non-financial. It involves their acceptance of a concept, idea, or proposal.

Data – A collection of facts, such as measurements and statistics.

Evidence-based practice – When clinicians (e.g., doctors, nurses) base their healthcare treatment decisions on the findings of current research, their clinical expertise, and the values/preferences of their patients.

Evidence-informed practice or program – A practice or program that is guided by theories and preliminary research. While there is some indication that these practices and programs produce positive outcomes, the evidence is too weak to refer to them as evidence-based. These are sometimes referred to as “promising” or “emerging” practices and programs.

Goals – General, non-measurable intentions or outcomes.

Incentives for participation – Factors that motivate an individual to take part in a program. Organizations sometimes provide incentives to encourage participants to begin and/or remain enrolled in a program. Common incentives include gift cards and program t-shirts.

Intervention – Organized efforts to promote health and prevent disease. This term is used because the efforts intervene, or come between, an individual and a negative health outcome in an attempt to prevent or delay the negative outcome. “Intervention” and “program” are often used interchangeably.

Interventionist – An individual who implements or carries out the components of a program.

Lay leaders – Individuals who do not have formal healthcare credentials who are trained to lead evidence-based programs.

Medicare – A publically-funded health insurance program for adults over age 65 and individuals with certain disabilities or health conditions.

Partnership – A cooperative relationship between two or more organizations that collaborate to achieve a common goal through the effective use of knowledge, personnel, and other resources.

Primary data – Original or new data being collected for a specific research goal.

Protocols - Predefined procedural methods. Examples include detailed program implementation procedures, required equipment, required data collection instruments with detailed instructions for administration, and recommended safety precautions.

Readiness – The degree to which an organization is prepared or ready for something.

Stakeholder – Any individual or group that has a stake or interest in a program.

Translation – The process of taking a program originally implemented in a controlled, laboratory-like setting and making it suitable for implementation in the community.

Table of Contents