Select an Evidence Based Program

Section 2.1

Identifying an Important Health Issue

The first step in selecting an EBP is to identify the needs of your audience. Knowing the needs of your audience will enable you to select an intervention that is relevant and has the potential to make a positive impact. “Needs” can be defined simply as the difference between present conditions and more desirable conditions. Needs are generally classified into two categories: actual (or true) and perceived (or reported). Actual needs can be objectively identified and are substantiated or demonstrated by measurements, such as death or disease rates. Perceived needs are reported by audience members as they draw from their experiences, ideas, feelings, and instincts.

Actual and perceived needs are not mutually exclusive. Often times the two will overlap because audience members will perceive needs that are substantiated by objective data. It would take enormous amounts of time and energy to classify all of an audience’s needs as perceived, actual, or both. A more efficient approach is to identify as many needs as possible and then determine which of these can be addressed through your organization’s programming.

Needs can be identified through primary and secondary sources of data. These data types are explained in the subsequent sections. Though it is ideal to gather data from numerous sources, your organization may not have the resources to use all of the sources that are provided as examples. Do not let this discourage you from assessing your audience’s needs. Simply select the sources that are realistic given your organization’s resources. If at all possible, use at least one primary and one secondary source. For information on what to do with needs assessment data once it is collected, see the content on Prioritizing Needs.

Figure 3. Existing instruments available for use
Existing Instruments

Numerous organizations have developed needs assessment instruments that are available for use by other organizations. Many of these instruments can be used for free and without special permission. Be sure to carefully read the instructions and terms of use for any instrument you choose to use.

Example sources for instruments:

Sources of Primary Data

Primary data are data you collect yourself. Generally, primary data answer questions that are tailored to your audience and situation, which is a major advantage of this data type. A major disadvantage of primary data is the need to assure the validity and reliability of the tools and techniques used to collect the data. Validity is the degree to which a tool (e.g., survey) or technique (e.g., observation) measures what it is designed to measure. Reliability is the degree to which a tool or technique yields results that are consistent. While the abovementioned definitions are succinct, there are several types of validity and reliability. Experts with backgrounds in research methods and statistics are best able to determine which tools and techniques are sound. If you do not feel anyone at your organization is well suited to determine if tools and techniques are valid and reliable, you might benefit from existing instruments that can be adapted and used by your organization (see Figure 3). The advantages and disadvantages of various primary data collection methods are summarized in Table 2.

Single-step or cross-sectional surveys

In single-step or cross-sectional surveys, participants are asked to self-report answers to questions during a single contact. Surveys can be used to gather information about a variety of topics including subjective experiences (e.g., health concerns, health behaviors, feelings about a health program) and preferences (e.g., health programs of interest, the best location or time to offer a program). Though self-reported measures can evaluate a number of topics, they rely on participants’ abilities to accurately recall information and to answer honestly. Several survey approaches are discussed below.

  • Written questionnaires — Surveys in the form of a written questionnaire are frequently used to collect self-reported data. Written questionnaires can be distributed to participants in a number of ways, including mail. It is best to use simple and clear language on a questionnaire to avoid confusion. It is also beneficial to place questions in a logical order. Seniors often have trouble with their vision, so it is best to use a 14 point (or larger) font. To encourage completion, surveys should be concise and to-the-point. A mix of open-ended questions, which prompt narrative answers (e.g., What sports do you play?), and close-ended questions, which offer response choices (e.g., Please place a check mark by each sport you play: __Football __Basketball __ Soccer __Tennis), can be used. Remember that open-ended questions generally take longer to answer and require more effort than close-ended questions. Personal or sensitive questions (e.g., income, ethnicity, drug or alcohol use) should be limited to the extent possible and placed towards the end of the survey. Provide clear instructions, keep the survey pages uncluttered, use an easy-to-read font, and provide plenty of room for responses. Comprehensive information on questionnaire design can be found in Module 4 — Questionnaire Design of the Centers for Disease Control and Prevention’s Reproductive Health Epidemiology Series
    • Electronic questionnaires — Electronic questionnaires are written questionnaires that are in an electronic format. They may be distributed by email or linked to webpages. These questionnaires may be the most affordable and quickest to administer. Special attention must be paid to privacy when using electronic questionnaires. Additionally, some seniors may be uncomfortable with computer technology, so be sure to assess your audience to see if they are at ease with an electronic format.
  • Telephone interviews — Trained interviewers, along with a questionnaire that includes a script, are required to conduct telephone interviews. The interviewers must be careful to ask the questions in an objective manner, so they do not “lead” participants to answer in a given way. The organization conducting the interviews must have access to a directory of audience members’ phone numbers. The Wallace Foundation has created a useful guide for organizations interested in conducting telephone surveys.
  • Face-to-face interviews — Questionnaires can also be administered through face-to-face interviews. Trained interviewers who can build rapport with participants are needed for this approach. Like in telephone interviews, the interviewers must be careful to ask the questions in an objective manner, so participants are not led to answer in a particular way. This is generally the most expensive and most time consuming method of questionnaire administration. Additional information about face-to-face interviews is available in the previously mentioned questionnaire design document produced by the Centers for Disease Control and Prevention (see section on written questionnaires).

Delphi Technique

The Delphi technique is a form of multi-step survey where the administrators contact participants more than once. Generally, participants are contacted via mail or email with one or two broad questions. The responses to these questions are analyzed and a second questionnaire, with more specific questions, is returned to participants. Several iterations or cycles of this process (usually five or fewer) will occur until consensus is reached. The Delphi technique is often used with groups of experts on a given topic. During a needs assessment, key informants (individuals with the knowledge and ability to report the needs of the audience) serve as experts. Additional details on the Delphi method, including sample invitation letters, are available on a tip sheet developed by the University of Wisconsin Cooperative Extension.

Community Forums

Community forums, also termed town hall meetings, bring together the audience and needs assessors. The forums or meetings are often held in easily accessible buildings, such as libraries or schools, and are scheduled at convenient times. Once participants have gathered, a moderator will explain the meeting purpose and will ask participants to share their ideas and concerns. Generally, participants will stand up before the group or even stand before a centrally located microphone when sharing. The moderator should be skilled in maintaining order, keeping the meeting on track, and eliciting responses from participants who are reluctant to share. Recorders should take notes or audio record the session so responses can be analyzed after the meeting. Tips on planning a town hall meeting are available from the Centers for Disease Control and Prevention.

Focus Groups

A group of 8-12 individuals are guided through an open discussion during a focus group. Ideally, focus groups are comprised of similar individuals who do not know one another. Both the similarity among individuals and the fact that they will not see one another again lower inhibitions and encourage sharing. The moderator stimulates free-flowing conversation by providing approximately 10 open-ended questions as prompts. While free-flowing conversation is the goal, the moderator helps participants focus on the questions at hand in order for needs assessors to obtain the information they need. A 45-90 minute discussion is long enough to gather data without fatiguing participants. It is best for an audio recording to be made so the conversation can be transcribed and analyzed for themes. Always be sure to ask permission from participants before recording them. Also, it is beneficial for an observer to take notes during the focus group in case the recording fails. The Community Tool Box of the University of Kansas provides guidelines for planning and conducting focus groups:

Nominal Groups

Nominal groups are comprised of approximately five to ten participants. A facilitator will present the participants with a question. Each participant then documents his or her answer silently and independently. Once each participant has documented an answer, each shares his or her response in a round-robin fashion. The facilitator documents each answer in a way that all participants can see (e.g., chalk board, white board, flipchart). Once all of the responses have been documented, the participants discuss the responses for clarification and then rank-order the responses by importance. In some cases, participants vote privately to rank-order the responses. In other cases, participants will have a preliminary vote that is followed by discussion before conducting a final vote. A brief on nominal groups is available from the Centers for Disease Control and Prevention

Observation

Members of the audience and the environment surrounding them can be observed as part of a needs assessment. For example, needs assessors may stand on a street corner and observe drivers to determine how many are wearing seatbelts. Additional behaviors and environmental conditions that can be observed include the use of hard hats by workers in a construction site, cigarette smoking by employees during work breaks, or vegetable consumption by individuals in a cafeteria. A popular form of observation is a windshield survey or walking tour, during which assessors drive and/or walk through a designated area, viewing as much of it as possible. While moving through the area, they watch for various predetermined elements (e.g., public transportation availability, housing type and quality, existence and quality of parks). The assessors document notes about what they see, hear, and smell regarding the elements. The notes are later analyzed for themes. A brief guide to windshield surveys is available from The Community Tool Box of the University of Kansas.

Table 2 - Primary data collection methods
Method Advantages Disadvantages
Written questionnaire
  • Inexpensive
  • Can be completed relatively quickly
  • Minimal staff time required
  • Easy to administer
  • Can include a large number of participants
  • Can administer by mail to include geographically dispersed participants
  • No chance of interviewer asking “leading” questions
  • Sensitive questions can be asked
  • Tend to have low response rates
  • Participants may not follow instructions
  • Participants may misunderstand questions
  • Individuals with low reading levels or poor vision will be unable to participate
Method Advantages Disadvantages
Electronic questionnaire
  • Inexpensive
  • Can be completed relatively quickly
  • Minimal staff time required
  • Easy to administer
  • Can include a large number of participants
  • Can administer to geographically dispersed participants
  • No chance of interviewer asking “leading” questions
  • Sensitive questions can be asked
  • Participants may not follow instructions
  • Participants may misunderstand questions
  • Individuals with low reading levels or poor vision will be unable to participate
  • Individuals without internet access will be unable to participate
  • Special attention must be paid to privacy
Method Advantages Disadvantages
Telephone interview
  • Moderate cost
  • Can include a large number of participants
  • Can include geographically dispersed participants
  • Participants can clarify if they do not understand questions
  • Individuals with low reading levels or poor vision can participate
  • Time consuming
  • Trained interviewers must conduct interviews
  • Low response rates becoming increasingly common
  • Individuals without telephones will be unable to participate
  • Interviewers can ask “leading” questions
  • Participants may be uncomfortable answering sensitive questions for a live interviewer
Method Advantages Disadvantages
Face-to-face interview
  • High response rate
  • Participants can clarify if they do not understand questions
  • Interviewers can build rapport with individuals, encouraging participation from those who are otherwise hesitant to take part
  • Individuals with low reading levels or poor vision can participate
  • Expensive
  • Time consuming
  • Trained interviewers must conduct interviews
  • Interviewers can ask “leading” questions
  • Participants may be uncomfortable answering sensitive questions for a live interviewer
  • Can only include a small number of participants
  • Difficult to access geographically dispersed participants
Method Advantages Disadvantages
Delphi technique
  • Anonymity in responding encourages the expression of opinions that are free from the influence of other participants
  • Develops consensus
  • Can include geographically dispersed participants
  • Time consuming
  • Labor intensive
  • Anonymity may lead to a lack of accountability for the views expressed and encourage hasty decisions
  • Individuals with low reading levels will be unable to participate
Method Advantages Disadvantages
Community forum
  • Inexpensive
  • Can include a large number of participants
  • Participants can clarify if they do not understand questions
  • Individuals with low reading levels or poor vision can participate
  • Can identify the most interested individuals
  • Special interest groups may be overrepresented
  • Participants’ answers may be influenced by other group members
  • Could degenerate into a gripe session
Method Advantages Disadvantages
Focus group
  • Inexpensive
  • Participants can clarify if they do not understand questions
  • Individuals with low reading levels or poor vision can participate
  • Can easily explore topics about which little is known
  • Trained moderator must conduct the focus group
  • Participants’ answers may be influenced by other group members
  • Can only include a limited number of participants
  • Difficult to access geographically dispersed participants
Method Advantages Disadvantages
Nominal group
  • Inexpensive
  • Each group member has a chance to express his or her opinion
  • “Cross-pollination” of ideas may occur, generating a large number of responses
  • Allows the group to prioritize ideas
  • Can be time consuming
  • Participants’ answers may be influenced by other group members
  • Can only include a limited number of participants
  • Difficult to access geographically dispersed participants
  • Most suitable for exploring a single topic only
Method Advantages Disadvantages
Observation
  • Inexpensive
  • Can be more accurate than self-report
  • Can identify factors related to the behavior or environment under consideration
  • May be time consuming
  • Requires trained observers
  • Special attention must be paid to privacy issues
  • If participants know they are being observed, they may act differently than if they did not know they were being observed

Sources of secondary data

Secondary data are data that have already been collected by someone else. Secondary data are advantageous because they are often available for immediate use and are inexpensive, if not free, to use. However, because secondary data were collected previously, it is difficult to determine if the data are still applicable to the audience or if appropriate methodologies were used during data collection. These are significant disadvantages. Obtaining secondary data from reputable sources, several of which are identified below, can help you capitalize on the advantages of secondary data and guard against the disadvantages.

  • Government agencies
    • International — Abundant data on global health can be found at the World Health Organization (WHO) website and the websites of its regional offices (European, South-East Asia, Americas, etc.). The WHO is the directing and coordinating authority for health within the United Nations system. Data and statistics are available by health topic or by nation. Additionally, the WHO website contains a number of publications on evidence-based practices and programs.
    • National — Numerous governmental organizations provide useful information for program planners. Demographic data (e.g., median household income, percent of population over age 65) can be found on the U. S. Census Bureau website. The Centers for Disease Control and Prevention (CDC), a component of the U. S. Department of Health and Human Services, provides health data in a variety of formats (see Table 3) on its website.
    • State and local — Each state has a respective state-level health agency and local (e.g., regional, county, city) health agencies. Various terms are used to denote these agencies, including health departments, health services, and health districts. These agencies often conduct periodic needs assessments, which may contain important information regarding your audience. They may also provide information regarding funding opportunities for EBPs. In Texas, the Department of State Health Services is the state-level health agency. Useful information available on the Department of State Health Services website includes disease and death rates for the state, details on available health services, and contact information for local health departments.
  • Non-governmental agencies and organizations
    • Voluntary health organizations — Voluntary health agencies are nonprofit organizations comprised of individuals (lay and professional) who seek to improve and protect the health of a particular group of people or combat a particular disease, disability, or type of disease or disability. These agencies receive funding from voluntary contributions. Their activities often include education, advocacy, research funding, patient support groups, and other services to affected individuals and communities. As part of their educational efforts, voluntary health organizations will often provide data related to their respective disease or disability emphases. Well-known voluntary health organizations include the American Cancer Society and the American Heart Association. There are voluntary health organizations for almost all diseases and disabilities. Speaking with individuals with these conditions, contacting local healthcare specialists, and searching the internet can help you identify these organizations.
    • Foundations — Organizations established by endowment with provision for future maintenance are considered to be foundations. Several health-related foundations provide useful data for determining the needs of an audience. For example, the Robert Wood Johnson Foundation, the nation's largest philanthropy devoted solely to the public's health, partners with the University of Wisconsin Population Health Institute and Community Catalyst to maintain the County Health Rankings & Roadmaps program . The Rankings portion of the program provides information on death and disease rates, as well as data on health behaviors, healthcare access, and socioeconomic factors for each county in the U. S. The Roadmaps portion of the program provides grants to local coalitions and partnerships among the policy, business, education, public health, healthcare, and community sectors; grants to national organizations working to improve health; recognition of communities whose promising efforts have led to better health; and customized technical assistance on strategies to improve health. Other foundations that provide useful data include the Henry J. Kaiser Family Foundation, which focuses on healthcare, statistics, and the John A. Hartford Foundation, which focuses on improving healthcare for older Americans.
    • Academic institutions — Many universities and colleges have needs assessment data on the communities surrounding their institutions. For example, the Center for Community Health Development of the Texas A&M Health Science Center School of Rural Public Health, in conjunction with the Brazos Valley Health Partnership, generated the Brazos Valley Health Assessment, a comprehensive, valid, and reliable source of needs assessment information for the Brazos Valley region of Texas. Another example is The Lower Rio Grande Valley Community Health Assessment generated by The University of Texas Health Science Center at Houston School of Public Health. Contact local institutions to determine if they have any relevant needs assessment data. Colleges and universities can also be a source for interns, technical assistance, and funding for EBPs.
    • Healthcare organizations — Hospitals and other healthcare institutions often collect needs assessment data on the communities in their service areas. Though any healthcare organization may provide this sort of data, it is especially beneficial to check with non-profit hospitals, which are now required by provisions in the Patient Protection and Affordable Care Act to conduct community health needs assessments and then implement plans to meet identified needs. An example resource is the community health needs assessment conducted by Scott & White Healthcare, a non-profit hospital system in Central Texas. On this page, users can find information on demographics, community health indicators, health disparities, and promising practices.
Table 3 - Secondary data sources from the CDC
Source Type of Data Web Address
National Center for Health Statistics (NCHS) The NCHS collaborates with other agencies to collect health data from birth and death certificates, medical records, personal interviews, standardized physical examinations, and laboratory tests. After compilation, the NCHS makes data available on numerous topics including the leading causes of death specific to age, race, ethnic, and gender groups; injuries and disabilities and their impact on health status and functioning; and nutrition and physical activity. The data is presented in a number of formats, including charts and fact sheets. http://www.cdc.gov/nchs/index.htm
CDC WONDER CDC WONDER provides access to a wide array of public health information. Using fill-in-the-blank webpages, users can query data regarding topics including death rates, disease rates, and environmental health. The requested data are readily summarized and analyzed with dynamically calculated statistics, charts, and maps. http://wonder.cdc.gov/
Behavioral Risk Factor Surveillance System (BRFSS) The BRFSS is an ongoing, state-based system of health surveys that collect information on health risk behaviors, preventive health practices, and healthcare access primarily related to chronic disease and injury. It is conducted by telephone. Survey findings can be looked up by metropolitan and micropolitan statistical areas, allowing identification of local health needs and risks. http://www.cdc.gov/brfss/

Glossary of Terms

Toolkit Glossary

scroll to see all

Adaptation– Making changes to an evidence-based program in order to make it more suitable for a particular organization and/or audience.

Baseline – A starting point. In evidence-based programming, the term “baseline” is usually used in the context of data collection, where baseline data is data collected before a program is implemented.

Credentials – A testimony of qualification, competence, or authority issued to an individual by a third party. Examples of credentials include academic diplomas, academic degrees (e.g., MSW, MPH, PhD), licenses (e.g., MD, RN, LCSW), certifications (e.g., CHES, CPR, first aid), and security clearances.

Evidence – Facts or testimony in support of a conclusion, statement, or belief. In some settings, individuals may refer to “levels of evidence” or “types of evidence.” These terms will have specific definitions unique to the setting in which they are used. When referring to evidence-based programs, the term “evidence” is generally used to describe the findings or results of program evaluation studies.

Evidence-based program – A program that has been thoroughly evaluated by researchers who determined it produces positive outcomes.

Fidelity – The extent to which a program is being implemented as its developers intended for it to be.

Implementation – Putting into action or carrying out a program.

Instrument – A measurement tool. Instruments can take many forms including biomedical equipment (e.g., glucometer, blood pressure monitor, weight scale), pencil and paper tests, questionnaires, and interviews. A thermometer is an instrument used to measure body temperature. Likewise, a survey is an instrument that can be used to measure anxiety.

Medicaid – A publically-funded health insurance program for individuals who have low incomes and fall into certain categories of eligibility.

Objectives – Specific, measurable steps that can be taken to achieve goals.

Peer review – When experts review a professional’s performance, research, or writings. Peer review is a way that qualified professionals self-regulate their professions. Performance, research, or writings that pass the peer review process have increased credibility or trustworthiness.

Program champion – An individual who advocates for a program.

Quality assurance- A collection of planned, systematic activities applied to ensure that the components of a program are being implemented well.

Secondary data – Previously collected data that is being used for a purpose other than that for which it was originally collected.

Theory of behavior change – An attempt to explain how and why people change their behaviors. Researchers typically generate theories of behavior change from research in the fields of psychology, education, health, and other social sciences. When developing evidence-based programs, researchers will select a theory or components from several theories to guide program development.

Audience – The individuals for whom you implement your program. Depending on your setting, these individuals may also be referred to as a target population, population of interest, or clientele.

Buy-in – Typically used in the business world, buy-in refers to a financial exchange. In the context of health programs, the buy-in of stakeholders (community members, organizational leaders, participants, etc.) is generally non-financial. It involves their acceptance of a concept, idea, or proposal.

Data – A collection of facts, such as measurements and statistics.

Evidence-based practice – When clinicians (e.g., doctors, nurses) base their healthcare treatment decisions on the findings of current research, their clinical expertise, and the values/preferences of their patients.

Evidence-informed practice or program – A practice or program that is guided by theories and preliminary research. While there is some indication that these practices and programs produce positive outcomes, the evidence is too weak to refer to them as evidence-based. These are sometimes referred to as “promising” or “emerging” practices and programs.

Goals – General, non-measurable intentions or outcomes.

Incentives for participation – Factors that motivate an individual to take part in a program. Organizations sometimes provide incentives to encourage participants to begin and/or remain enrolled in a program. Common incentives include gift cards and program t-shirts.

Intervention – Organized efforts to promote health and prevent disease. This term is used because the efforts intervene, or come between, an individual and a negative health outcome in an attempt to prevent or delay the negative outcome. “Intervention” and “program” are often used interchangeably.

Interventionist – An individual who implements or carries out the components of a program.

Lay leaders – Individuals who do not have formal healthcare credentials who are trained to lead evidence-based programs.

Medicare – A publically-funded health insurance program for adults over age 65 and individuals with certain disabilities or health conditions.

Partnership – A cooperative relationship between two or more organizations that collaborate to achieve a common goal through the effective use of knowledge, personnel, and other resources.

Primary data – Original or new data being collected for a specific research goal.

Protocols - Predefined procedural methods. Examples include detailed program implementation procedures, required equipment, required data collection instruments with detailed instructions for administration, and recommended safety precautions.

Readiness – The degree to which an organization is prepared or ready for something.

Stakeholder – Any individual or group that has a stake or interest in a program.

Translation – The process of taking a program originally implemented in a controlled, laboratory-like setting and making it suitable for implementation in the community.

Table of Contents