Implement an Evidence Based Program

Section 5.3

In the context of program implementation, sustainability can be defined on the individual, organizational, or community levels.

  • Individual level — Maintenance of program benefits for individual participants after the program ends.
  • Organizational level — Maintaining program activities within an organization and making sure program goals and approaches adapt to changing needs over time. This is sometimes referred to as institutionalization.
  • Community level — Increasing the capacity of a community to develop and implement program activities.

All three levels of sustainability are important and each is particularly relevant depending on your circumstance. Individual level sustainability is important to examine during evaluation research. Data on individual level sustainability contributes to the evidence base of a program. An organization might choose to implement one program over another due to the individual level sustainability demonstrated in evaluation studies. Community level sustainability is especially important to consider if a community coalition participated in the selection or implementation process. If you are in a position to influence policy then you would also be interested in this level of sustainability.

Most organizations planning and implementing programs will be mainly concerned with organizational level sustainability. This level pertains to the abilities of individual organizations to maintain implementation of their programs in the face of changes in funding, resource availability, and audience characteristics.

Before implementing strategies to increase sustainability, organizations must determine if their program should be sustained. In general, programs that produce positive outcomes should be sustained. Sometimes programs will not produce positive outcomes or will produce few positive outcomes in proportion to the resources invested in them. In these latter situations, organizations must determine whether or not to sustain their programs.

If you believe a program failed to produce positive outcomes due to a distinct variable that will not be repeated (such as a community-wide disaster or significant staff turnover that occurred during implementation), then you may choose to sustain the program with the expectation that positive outcomes will pick up now that the variable has passed. If, on the other hand, you believe the program was unsuccessful due to it being a mismatch with your organization or the audience, then it would be unwise to continue investing in the program. If the program did produce positive outcomes but they were small in proportion to the resources invested into it, then consider searching for a program that targets a similar outcome. You may be able to identify and implement a similar program that is less costly, less time consuming, and/or requires fewer staff members.

Strategies to increase sustainability

Because settings and audiences vary so widely, there are no hard-and-fast strategies that can be recommended for application in all situations. Instead, various strategies should be applied if relevant to your organization, program, and audience. Be sure to pay close attention to fidelity when selecting strategies to increase sustainability so that you do not jeopardize program outcomes.

  • Identify alternative funding sources — Consider every possible source of future funding (see Obtaining Funding for ideas). Learn more about these sources, including what funding opportunities are available and what reporting requirements they have for funded organizations.
  • Involve key stakeholders — Determine which stakeholders are crucial to the ongoing implementation of your organization’s program. Do you need referrals from healthcare providers? Engage physician groups and nurses. Do you need access to city-owned park and recreation facilities? Engage staff at the parks and recreation department. Other important stakeholders may be members of local and state government, leaders of local community-based or faith-based organizations, senior center staff, hospital staff, or members of volunteer organizations.
  • Form partnerships — Partnering with other agencies allows you to share resources (staff, facilities, equipment, communication channels, etc.), which can decrease implementation costs considerably. Partnerships also give you access to a broader pool of potential participants, which means greater likelihood of maintaining enrollment.
  • Align program services with organizational goals — In general, organizational leadership will be more likely to reassign or find additional funding for a program if it is congruent with the organization’s mission.
  • Select affordable supplemental services — If you provide services to participants that are above and beyond those required for program fidelity, select services that are affordable. For example, it would be quite costly to provide nicotine replacement therapies to participants in a smoking cessation program. However, if your state has a program that provides these for free to participants enrolled in smoking cessation programs, then you could provide the service of assisting participants in applying to receive the therapies from the state.
  • Change staff composition — While program fidelity may require that certain implementation roles be filled by individuals with particular credentials or experience, there may be flexibility in who can fill some other positions. Consider using nonpaid staff (e.g., volunteers, interns) or lay leaders who are compensated with small stipends.

Glossary of Terms

Toolkit Glossary

scroll to see all

Adaptation– Making changes to an evidence-based program in order to make it more suitable for a particular organization and/or audience.

Baseline – A starting point. In evidence-based programming, the term “baseline” is usually used in the context of data collection, where baseline data is data collected before a program is implemented.

Credentials – A testimony of qualification, competence, or authority issued to an individual by a third party. Examples of credentials include academic diplomas, academic degrees (e.g., MSW, MPH, PhD), licenses (e.g., MD, RN, LCSW), certifications (e.g., CHES, CPR, first aid), and security clearances.

Evidence – Facts or testimony in support of a conclusion, statement, or belief. In some settings, individuals may refer to “levels of evidence” or “types of evidence.” These terms will have specific definitions unique to the setting in which they are used. When referring to evidence-based programs, the term “evidence” is generally used to describe the findings or results of program evaluation studies.

Evidence-based program – A program that has been thoroughly evaluated by researchers who determined it produces positive outcomes.

Fidelity – The extent to which a program is being implemented as its developers intended for it to be.

Implementation – Putting into action or carrying out a program.

Instrument – A measurement tool. Instruments can take many forms including biomedical equipment (e.g., glucometer, blood pressure monitor, weight scale), pencil and paper tests, questionnaires, and interviews. A thermometer is an instrument used to measure body temperature. Likewise, a survey is an instrument that can be used to measure anxiety.

Medicaid – A publically-funded health insurance program for individuals who have low incomes and fall into certain categories of eligibility.

Objectives – Specific, measurable steps that can be taken to achieve goals.

Peer review – When experts review a professional’s performance, research, or writings. Peer review is a way that qualified professionals self-regulate their professions. Performance, research, or writings that pass the peer review process have increased credibility or trustworthiness.

Program champion – An individual who advocates for a program.

Quality assurance- A collection of planned, systematic activities applied to ensure that the components of a program are being implemented well.

Secondary data – Previously collected data that is being used for a purpose other than that for which it was originally collected.

Theory of behavior change – An attempt to explain how and why people change their behaviors. Researchers typically generate theories of behavior change from research in the fields of psychology, education, health, and other social sciences. When developing evidence-based programs, researchers will select a theory or components from several theories to guide program development.

Audience – The individuals for whom you implement your program. Depending on your setting, these individuals may also be referred to as a target population, population of interest, or clientele.

Buy-in – Typically used in the business world, buy-in refers to a financial exchange. In the context of health programs, the buy-in of stakeholders (community members, organizational leaders, participants, etc.) is generally non-financial. It involves their acceptance of a concept, idea, or proposal.

Data – A collection of facts, such as measurements and statistics.

Evidence-based practice – When clinicians (e.g., doctors, nurses) base their healthcare treatment decisions on the findings of current research, their clinical expertise, and the values/preferences of their patients.

Evidence-informed practice or program – A practice or program that is guided by theories and preliminary research. While there is some indication that these practices and programs produce positive outcomes, the evidence is too weak to refer to them as evidence-based. These are sometimes referred to as “promising” or “emerging” practices and programs.

Goals – General, non-measurable intentions or outcomes.

Incentives for participation – Factors that motivate an individual to take part in a program. Organizations sometimes provide incentives to encourage participants to begin and/or remain enrolled in a program. Common incentives include gift cards and program t-shirts.

Intervention – Organized efforts to promote health and prevent disease. This term is used because the efforts intervene, or come between, an individual and a negative health outcome in an attempt to prevent or delay the negative outcome. “Intervention” and “program” are often used interchangeably.

Interventionist – An individual who implements or carries out the components of a program.

Lay leaders – Individuals who do not have formal healthcare credentials who are trained to lead evidence-based programs.

Medicare – A publically-funded health insurance program for adults over age 65 and individuals with certain disabilities or health conditions.

Partnership – A cooperative relationship between two or more organizations that collaborate to achieve a common goal through the effective use of knowledge, personnel, and other resources.

Primary data – Original or new data being collected for a specific research goal.

Protocols - Predefined procedural methods. Examples include detailed program implementation procedures, required equipment, required data collection instruments with detailed instructions for administration, and recommended safety precautions.

Readiness – The degree to which an organization is prepared or ready for something.

Stakeholder – Any individual or group that has a stake or interest in a program.

Translation – The process of taking a program originally implemented in a controlled, laboratory-like setting and making it suitable for implementation in the community.

Table of Contents