Implement an Evidence Based Program

Section 5.4

Quality Assurance

Additional Resource

The National Council on Aging provides a brief on the Administration on Aging recommendations for quality assurance.

National Council on Aging offers RE-AIM for Program Planning: Overview and Applications.

Quality assurance (QA) is a collection of planned, systematic activities applied to ensure that program components are being implemented well. Initiatives to improve program activities and prevent mistakes or defects also constitute QA. Despite the initial appearance of similarity, QA differs from evaluation in that QA focuses on ongoing work instead of final outcomes at a particular point in time. In an era where funding opportunities are often competitive, QA is critical because it demonstrates to funders and stakeholders that EBPs are credible, worthwhile investments.

Continuous Quality Improvement (CQI), an ongoing quality improvement process that is closely related to QA, involves four steps: (1) planning (e.g., selecting implementation objectives and methods of monitoring program delivery), (2) monitoring (e.g., collecting process and outcome data and obtaining feedback from stakeholders), (3) evaluating (e.g., analyzing data collected during monitoring to identify challenges and develop solutions for them), and (4) making corrective changes (e.g., implementing solutions to identified challenges in order to improve overall program performance and participant satisfaction).

A QA program that integrates CQI will include the following components.

  • A specified timeline for QA activities and designated roles and responsibilities for personnel.
  • Personnel and stakeholder orientation to the QA plan and procedures.
  • Designated performance indicators including measures of participant reach, organizational capacity, and program delivery.
  • Mechanisms for personnel to conduct periodic reviews of fidelity and assess performance using indicators.
  • Protocols for taking corrective action when fidelity is jeopardized or performance requires improvement.

The concept of QA can initially seem abstract and difficult to apply, but frameworks such as RE-AIM (described in the subsequent section) can help organizations implement QA in logical, manageable pieces.

RE-AIM framework

The RE-AIM framework was developed as a model for the planning, evaluation, reporting, and review of translational research and practice. (Translation is the process of taking a program originally implemented in a controlled “laboratory-like” setting and making it suitable for implementation in the community.) The framework has five components from which the framework name is drawn: Reach, Effectiveness, Adoption, Implementation, and Maintenance. Each component and its significance are described in Table 5. Information about the development and application of RE-AIM can be found at www.re-aim.org.

Table 5 - RE-AIM components
Component Definition Significance
Reach The extent to which a program reaches the intended audience. Assesses the adequacy of program marketing, participant recruitment, and participant retention strategies.
Effectiveness The impacts and outcomes of an intervention in terms of measures such as disease rates, behaviors, knowledge, quality of life, and healthcare expenses. Evaluates whether a program is producing positive changes in audience members’ well-being.
Adoption The extent to which organizations and implementation sites deliver and embed the program into their ongoing activities and the level of organizational support provided for it. Investigates the adequacy of program partners, resources, and the implementation process (e.g., session locations, frequency, skill of personnel).
Implementation Fidelity monitoring. Determines if the program is being implemented as designed, regardless of the setting or which staff member is delivering the program.
Maintenance The extent to which a program becomes institutionalized or a routine part of an organization’s activities. Assesses the extent to which the program is accessible to new participants and the sustainability infrastructure.

Glossary of Terms

Toolkit Glossary

scroll to see all

Adaptation– Making changes to an evidence-based program in order to make it more suitable for a particular organization and/or audience.

Baseline – A starting point. In evidence-based programming, the term “baseline” is usually used in the context of data collection, where baseline data is data collected before a program is implemented.

Credentials – A testimony of qualification, competence, or authority issued to an individual by a third party. Examples of credentials include academic diplomas, academic degrees (e.g., MSW, MPH, PhD), licenses (e.g., MD, RN, LCSW), certifications (e.g., CHES, CPR, first aid), and security clearances.

Evidence – Facts or testimony in support of a conclusion, statement, or belief. In some settings, individuals may refer to “levels of evidence” or “types of evidence.” These terms will have specific definitions unique to the setting in which they are used. When referring to evidence-based programs, the term “evidence” is generally used to describe the findings or results of program evaluation studies.

Evidence-based program – A program that has been thoroughly evaluated by researchers who determined it produces positive outcomes.

Fidelity – The extent to which a program is being implemented as its developers intended for it to be.

Implementation – Putting into action or carrying out a program.

Instrument – A measurement tool. Instruments can take many forms including biomedical equipment (e.g., glucometer, blood pressure monitor, weight scale), pencil and paper tests, questionnaires, and interviews. A thermometer is an instrument used to measure body temperature. Likewise, a survey is an instrument that can be used to measure anxiety.

Medicaid – A publically-funded health insurance program for individuals who have low incomes and fall into certain categories of eligibility.

Objectives – Specific, measurable steps that can be taken to achieve goals.

Peer review – When experts review a professional’s performance, research, or writings. Peer review is a way that qualified professionals self-regulate their professions. Performance, research, or writings that pass the peer review process have increased credibility or trustworthiness.

Program champion – An individual who advocates for a program.

Quality assurance- A collection of planned, systematic activities applied to ensure that the components of a program are being implemented well.

Secondary data – Previously collected data that is being used for a purpose other than that for which it was originally collected.

Theory of behavior change – An attempt to explain how and why people change their behaviors. Researchers typically generate theories of behavior change from research in the fields of psychology, education, health, and other social sciences. When developing evidence-based programs, researchers will select a theory or components from several theories to guide program development.

Audience – The individuals for whom you implement your program. Depending on your setting, these individuals may also be referred to as a target population, population of interest, or clientele.

Buy-in – Typically used in the business world, buy-in refers to a financial exchange. In the context of health programs, the buy-in of stakeholders (community members, organizational leaders, participants, etc.) is generally non-financial. It involves their acceptance of a concept, idea, or proposal.

Data – A collection of facts, such as measurements and statistics.

Evidence-based practice – When clinicians (e.g., doctors, nurses) base their healthcare treatment decisions on the findings of current research, their clinical expertise, and the values/preferences of their patients.

Evidence-informed practice or program – A practice or program that is guided by theories and preliminary research. While there is some indication that these practices and programs produce positive outcomes, the evidence is too weak to refer to them as evidence-based. These are sometimes referred to as “promising” or “emerging” practices and programs.

Goals – General, non-measurable intentions or outcomes.

Incentives for participation – Factors that motivate an individual to take part in a program. Organizations sometimes provide incentives to encourage participants to begin and/or remain enrolled in a program. Common incentives include gift cards and program t-shirts.

Intervention – Organized efforts to promote health and prevent disease. This term is used because the efforts intervene, or come between, an individual and a negative health outcome in an attempt to prevent or delay the negative outcome. “Intervention” and “program” are often used interchangeably.

Interventionist – An individual who implements or carries out the components of a program.

Lay leaders – Individuals who do not have formal healthcare credentials who are trained to lead evidence-based programs.

Medicare – A publically-funded health insurance program for adults over age 65 and individuals with certain disabilities or health conditions.

Partnership – A cooperative relationship between two or more organizations that collaborate to achieve a common goal through the effective use of knowledge, personnel, and other resources.

Primary data – Original or new data being collected for a specific research goal.

Protocols - Predefined procedural methods. Examples include detailed program implementation procedures, required equipment, required data collection instruments with detailed instructions for administration, and recommended safety precautions.

Readiness – The degree to which an organization is prepared or ready for something.

Stakeholder – Any individual or group that has a stake or interest in a program.

Translation – The process of taking a program originally implemented in a controlled, laboratory-like setting and making it suitable for implementation in the community.

Table of Contents

Selecting a suitable evidence-based program

1

2

3

Implementing a suitable evidence-based program

4

5