´óÏóÊÓƵ

Contents

On Page 1 of 2:
5.A. Analyze CAHPS Survey Results
5.B. Analyze Other Sources of Information for Related Information

Page 2 of 2:
5.C. Evaluate the Process of Care Delivery
5.D. Gather Input From Stakeholders
References

---

5.C. Evaluate the Process of Care Delivery

If it is not clear why you are doing well on some CAHPS survey measures and not so well on others, you may need more detailed information to help you identify actions that can improve patient experience in specific areas. To get that information, you need to go beyond the survey results to do some additional analyses targeted at one or more specific topics addressed by the survey items or composites. The purpose of these analyses is to "drill down" to find very specific, underlying performance problems that are actionable—i.e., that you can change through quality improvement activities.

Consider a clinical practice whose score for the Access composite "Getting timely appointments, care, and information" is lower than average. An initial analysis of this practice's survey scores may find that a key driver of the composite score was a low score on this survey question: "When you made an appointment for a check-up or routine care, how often did you get an appointment as soon as you thought you needed?"

Why might patients be having trouble getting a timely appointment for check-ups or routine care? Various operational issues in the practice could contribute to this problem:

• The physicians may not be available sufficient hours to handle all the patients served by the practice.
• Problems in scheduling appointments may have a seasonal pattern related to when physicians take vacations or are otherwise not available.
• Routine appointments may be bumped frequently by last-minute emergency visits.
• Limitations of office hours may make it difficult to find visit times that are convenient for patients.
• The staff working on the appointment calendar may not be interacting well with patients to identify their needs and priorities.

This section uses this example to explore several tools and techniques you can use to examine the underlying causes of performance problems revealed by survey results. Although some of these approaches were developed for use in industrial settings, they apply equally well to health care.

• Root cause analysis
• Process mapping
• Process observation (including shadowing)
• Walkthroughs
• Small-scale surveys

5.C.1. Root Cause Analysis

Root cause analysis, also called "5 Whys," is a method for identifying the root causes of a problem and determining the relationship among different root causes. Repeatedly asking the question "Why" peels away the layers of issues to uncover the fundamental source of a problem. You may find that you will need to ask "why" fewer or more times than five to reach a conclusion. This tool, which does not involve a statistical hypothesis or analysis, is most useful when problems involve human factors or interactions.

Use the following steps to complete a root cause analysis:

Step 1: Write down the specific problem. Articulating the issue in writing helps you formalize the problem and describe it completely. It also helps everyone on the improvement team focus on the same problem.

Example of a problem: A medical practice has received low CAHPS scores for the item on getting an appointment scheduled as soon as patients would like. It also is receiving a large number of complaints from patients on this issue.

Step 2: Ask why the problem happens and write the answer down below the problem.

Why? (#1): There are not sufficient times available on the calendar for scheduling the number of patients calling in a timely manner.

Step 3: If the answer you just provided does not identify the root cause of the problem that you wrote down in step 1, ask why again and write that answer down.

Why? (#2): The practice only has office hours 4 days a week and is not open on Saturdays.

Step 4: Loop back to step 3 until the team is in agreement that the problem's root cause is identified.

Why? (#3): The physicians in the practice are not willing to work on Saturdays, and many of them are not always available to see patients for all of the weekday hours.

5.C.2. Process Mapping

To figure out how to improve a process, it helps to map it. A process map is a picture or flow chart showing the steps involved in transforming the inputs into the outputs of the process. For example, the practice in the example above would list each step involved in scheduling appointments for routine care. The chart seen in Figure 5-7 shows a simple process map for an appointment process in a medical practice. It includes:

• The process steps (best described using nouns [blue boxes]),
• The activities between the steps (best described using verbs [white boxes]), and
• For each activity, the inputs and outputs involved (arrows).¹

Figure 5-7. Process map of a physician visit appointment

You can choose from a variety of formats for preparing your process map. But within any given process map, use consistent symbols for each type of process component, such as process steps, activities, and decision steps. This will support clear communication among participants as you develop and work with the process map to guide improvement decisions. Learn more about .

Process mapping can address two aspects of process improvement:

• Developing an initial understanding of how things are done currently. It is critical to start by depicting the process the way it really works, not the way you think it should work.
• Examining and testing alternative changes to improve the process.

For best results, this method needs to be accurate and fast; it should also involve a high degree of staff ownership as well as input from patients or enrollees who can provide their perspective on what really happens.

5.C.2.a. Steps in Developing a Process Map

• Start with the big picture. Draw a macro-level process first, after which you may want to develop other diagrams with increased levels of detail. For example, you could develop a more detailed process map of the "Call from patient to schedule a visit" to understand the steps a patient goes through with your phone system to make an appointment.
• Observe the current process. Walk through the current process, observing it in actual operation. (Read about walkthroughs and shadowing below.)
• Record the process steps you observed. Document the steps as they actually occur. Start by writing the steps separately on index cards or sticky notes.
• Arrange the sequence of steps. Lay out the cards or sticky notes exactly as you observed the steps. Using cards lets you rearrange the steps without erasing and redrawing and prevents you from discarding ideas simply because it is too much work to redraw the diagram.
• Draw the final process map. Depict the process exactly as you observed, recorded, and arranged the sequence of steps.

5.C.2.b. Common Weaknesses of Process Maps

Take steps to avoid and correct for these common pitfalls that can interfere with your interpretation and full understanding of the process.

• Those working on the map may have drawn it for the process as they envision it, not as it really is.
• People may be reluctant to depict the obviously illogical parts of the process for fear they will be asked to explain why things have been working that way.
• Rework loops are either not seen or not documented because people assume rework is small and inevitable.
• The people drawing the map do not really know how the process works.

 

 

5.C.3. Process Observation

Process observation is a way of confirming exactly what is happening during any particular process. It allows you to gather useful information about almost any process, activity, or human behaviors that you can use to refine your process map as well as to help uncover issues that are compromising the effectiveness of the process.

Often you will not be able to observe all relevant activities by people, location, or over time, so you can observe only a sample of activities. If you sample, consider how important it is to have a probability sample, which would allow you to generalize to the entire process.

 

5.C.3.a. Methods of Observation

To choose an observation method, start by answering these questions:

• What do you want to learn from the observation?
• What will the users/stakeholders view as credible and useful information?

You may use either structured or unstructured observation methods, depending on the type of information you want to collect.

• Structured observation looks for certain things that have already been identified and can be tracked in a preset guide, checklist, or rating scales. This method generates quantitative data from frequency counts, rankings, and ratings.
• Unstructured observation looks at what is happening in a process or activity without confining the observer to preset items. The observed activities are recorded during the observation period, which produces qualitative data.

5.C.3.b. Observation Tools

Several types of tools are available to record observation data. Choose your recording methods—alone or in combination—based on your observation design.

• Observation guides. These printed forms provide space for recording observations, which allows for the consistent collection of information across observers or sites. The more detailed you make the guide, the easier it will be to tally results, but the less flexibility it will provide for recording findings.
• Recording sheets or checklists. These forms are used to record observation in either yes/no or rating scale formats. They are used when observations are looking for specific items or activities that are easily identified.
• Field notes. This tool is the least structured way to record observations. When the observer sees or hears something of import, he or she records it in a narrative, descriptive style, typically in a notebook. Observations should be accompanied by the date, location, and relevant contextual information.
• Pictures or videos. The observer can also record pictures or videos, which can be analyzed later and used to illustrate points in a report.

 

 

5.C.4. Small-Scale Surveys

A small-scale survey can be used to drill down on the experience behind CAHPS scores or to survey staff about barriers they encounter when trying to schedule patients. You can conduct a small-scale survey with a convenience sample of as few as 10 individuals and usually no more than 100; examples of a convenience sample include:

• All patients who visit a specific clinic on a given day.
• All patients who report a problem scheduling appointments.
• Staff who participated in a specific training exercise.

These kinds of surveys are useful in that they provide information that you can act on or help you to understand what kinds of experiences may be driving your CAHPS scores. For example, one large health system took advantage of its marketing department's online opinion panel to survey 1,000 clinic patients about what "helpfulness" meant to them and what office staff could do to be more helpful.² However, it is important to recognize that the results of small-scale surveys are not generalizable to your patient population because they are not based on a scientific sample. That is, they reflect only the experience of the patients you surveyed, who are not representative of your total patient population.

Return to Contents

---

5.D. Gather Input From Stakeholders

Your analysis of performance issues can benefit from good information on the views, experiences, needs, and motivations of the various stakeholders who are involved in or affected by the processes you're addressing. To help identify and examine the causes of your performance problem, consider contacting the relevant stakeholders to find out what they know, how they feel about issues, and their ideas for improvement. Different stakeholders have unique perspectives that you need to consider together to understand the full dynamics involved in delivering and receiving health care and how those dynamics influence patients' experiences with care.

5.D.1. Overview of the Process of Gathering Stakeholder Input

Imagine that an initial analysis of the practice with poor performance on the access composite found that a key driver of the composite score was a low score on this CAHPS question: "When you made an appointment for a check-up or routine care, how often did you get an appointment as soon as you needed?" What can you learn from stakeholders about the problems with timely appointments for care and how to fix those problems?

Step 1: Working as a team, identify the groups that are key stakeholders for the CAHPS performance issue you're addressing. Stakeholders can include patients and their family members, physicians, nurses, other clinical personnel, clerical staff, managers of the health care organization, and staff of other involved organizations. You should include groups who are involved in the process (such as nurses) as well as others who are affected by it (such as patients), since both would be affected by any changes you make during quality improvement work. For example, for a problem related to the appointment process, stakeholders may include:

• The physicians in the practice
• The patients who are getting appointments for care
• The office staff who handle the appointment process
• Nursing staff who initiate the office visit with patients
• The office manager who supervises the practice operation

People on the "front line" of care typically have the best understanding of what works well and what doesn't because they live with it every day. However, front-line caregivers sometimes become so accustomed to working in a "broken" system that they accept some problems as inevitable ("just the way it is") when the problems can—and should—be fixed.

Step 2: Develop a list of the topics you want to discuss with the stakeholder groups to learn:

• How the process works
• What they think is wrong with it
• How they think it needs to be improved

Step 3: Use qualitative data collection methods to gather information from people in each of your stakeholder groups. (Read about these methods below.) The exact methods you choose to use will depend on which types of stakeholders you will be talking with, and whether you want to have group discussions or talk separately with individuals.

Step 4: Summarize your findings. With feedback from all your stakeholder groups on each of the topics, you can compare responses to find similarities and differences in views and concerns across the groups.

Step 5: Use the information from the stakeholders to refine your process map and your list of possible issues affecting performance. You can also use this information to help guide strategies and actions for improving performance on the CAHPS measures.

5.D.2. Techniques for Gathering Feedback From Stakeholders

Techniques you can use to gather information from stakeholders on their experiences and views of performance problems include:

• Focus groups
• Semi-structured interviews
• Walkthroughs
• Patient and family advisory councils
• Patient Partners on improvement teams

5.D.2.a. Focus Groups

A focus group is a moderator-led discussion among staff and/or patients that is designed to collect more precise information about a specific problem and new ideas for improvement strategies. This approach allows for in-depth exploration of the drivers of dissatisfaction and can provide excellent ideas for reengineering services.

In addition, videotapes of focus groups can be very effective at changing the attitudes and beliefs of staff members because the participants' stories often bring to life the emotional impact of excellent service as well as service failures.

When conducting a focus group, the moderator uses a written topic guide to ensure that the group addresses all key topics in the discussion; another person usually serves as a note taker. The moderator typically uses various techniques during the discussion so that everyone in the group has a chance to speak and discussion among group members takes place. Examples of these techniques include going around the table to ask each person to give their views on a topic being discussed and specifically asking people who have not said much for their opinions.

5.D.2.b. Semi-Structured Interviews

In contrast to focus groups, interviews allow you to collect a great deal of rich, detailed information on the experience of an individual. They also offer greater flexibility in terms of the order in which topics are discussed. Interviews are also useful when you want to:

• Collect information that is not influenced by the opinions of others in a group discussion.
• Collect information from staff that is not influenced by the presence of supervisors or managers.

Semi-structured interviews are conducted one-on-one or in groups of no more than three people. The interviewer typically uses a topic guide and is accompanied by a note taker.

5.D.2.c. Walkthroughs

A walkthrough recreates for clinicians and staff the emotional and physical experiences of being a patient or family member. It is an easy way to give members of your organization the patient's perspective and the fastest way to identify system, flow, and attitude problems. Walkthroughs provide a different perspective and bring to light rules and procedures that may have outlived their usefulness.

How a Walkthrough Works

During a walkthrough, one staff member plays the role of the patient and another accompanies him or her as the family member. They go through a clinic, service, or procedure exactly as a patient and family do. They do everything patients and families are asked to do and they abide by the same rules. They do this openly, not as a mystery patient, and throughout the process ask staff members a series of questions to encourage reflection on the processes or systems of care and to identify improvement opportunities.

The staff conducting the walkthrough take notes to document what they see and how they feel during the process. They then share these notes with the leadership of the organization and quality improvement teams to help develop improvement plans. For many who do this, it is the first time they have ever entered their clinics, procedure rooms, or labs as the patient and family do. Clinicians are routinely surprised about how easy it is to hear staff comments about patients from public areas and waiting rooms. Walkthroughs usually turn up many problems with flow, signage, and wasteful procedures and policies that can be fixed almost immediately.

A walkthrough is similar to shadowing (discussed in Process Observation), where a staff member asks permission to accompany a patient through the visit and take notes on the patient's experience. Since shadowing does not require taking a slot away from a real patient, it can be useful in settings where visits are at a premium.

Tips on Conducting a Walkthrough

• Let the staff know in advance that you will be doing this walkthrough. As a result of this warning, they will probably be on their best behavior. However, experience suggests that it is far better to have them part of the process than to go behind their backs. Ask them not to give you special treatment.
• Go through the experience just as the patient and family member would. Call in advance, if the patient would have to. Get dropped off or find a place to park. Try to act as if you have never been there before. Follow the signs. Tell the clerk that you are simulating a patient's experience and that you want to go through whatever a normal patient would have to do (e.g., the check-in process). Actually fill out the forms if there are ones to fill out. Find out how long a patient would typically wait and sit in the waiting room for that amount of time. Wait your turn. Do the same in the examining room. If a patient would undress, you should undress. If a patient does a peak flow meter, you should too. Ask each health care provider to treat you as if you were a real patient. If you are doing a walkthrough of the cardiac catheterization service, hold the sandbags on your leg the required amount of time.
• As you go through the process, try to put yourself in the patient's (or family member's) position. Look around as they might. What are they thinking? How do they feel at this moment?
• At each step, ask the staff to tell you what changes (other than hiring new staff) would make the experience better for the patient and what would make it better for the staff. Write down their ideas as well as your own, and also write down your feelings. As you do the walkthrough, think about how you would answer the following questions and ask the staff you interact with to answer them when you can:
  • What made you mad today?
  • What took too long?
  • What caused complaints today?
  • What cost too much?
  • What was wasted?
  • What was too complicated?
  • What involved too many people or too many steps?
  • What did you have to do that was just plain silly?
• Finally, between the two of you (patient and family member), make a list of any issues you identified and any improvements that could be made. Keep track of the things that can be fixed the next day versus problems that will take longer to remedy.

5.D.2.d. Patient and Family Advisory Councils

You can obtain feedback as well as improvement ideas from patients and families through strategies that engage their participation on an ongoing basis. A Patient and Family Advisory Council is one of the most effective strategies for involving families and patients in the design of care and ensuring that those on the receiving end of health care have a voice in the organization's decision-making process.

A patient and family advisory council can help overcome a common problem that most organizations face when they begin to develop patient-and family-centered processes: They do not have the direct experience of illness or the health care system. Consequently, health care professionals often approach the design process from their own perspective, not the patients' or families'. Improvement committees with the best of intentions may disagree about who understands the needs of the family and patient best. But family members and patients rarely understand professional turf boundaries. Their suggestions are usually inexpensive, straightforward, and easy to implement because they are not bound by the usual rules and sensitivities.

Council responsibilities may include input into or involvement in:

• Program development, implementation, and evaluation;
• Planning for major renovation or the design of a new building or services;
• Staff selection and training;
• Marketing the plan's or practice’s services;
• Participation in staff orientation and in-service training programs; and
• Design of new materials or tools that support the doctor-patient relationship.

While councils can play many roles, they do not function as boards, nor do they have fiduciary responsibility for the organization.

Tips for Starting a Patient and Family Advisory Council

• Recruitment: You can start with members that are recommended by staff. Look for people who:
  • Can listen and respect different opinions.
  • Are supportive of the institution's mission.
  • Are constructive with their input. Staff members will frequently describe good council members as people who know how to provide "constructive critiques."
  • Are comfortable speaking to groups and in front of professionals.
• Size: Depending on the size of the organization, most councils have between 12 and 30 patient or family members and 3 or 4 members from the staff of the organization.
• Time commitment: The council members are usually asked to commit to one 2- to 3-hour meeting a month, usually over dinner, and participation on one committee. Most councils start off with one-year terms for all members to allow for graceful departures in case a member is not well suited for the council.

 

5.D.2.e. Patient Partners on Improvement Teams

You can take the strategy of engaging patients in the process of care design and improvement one step further by embedding patients as active partners working together with clinicians and staff on quality improvement teams. This approach, referred to as Patient Partners, recognizes that true patient-centered transformation of care cannot be achieved without enlisting the active involvement of patients in the redesign process. Including patient partners as members of practice improvement teams brings the patient voice and perspective directly into the hard and sometimes messy work of process redesign, and can be an enlightening and rewarding experience for patients, clinicians, and staff alike.

 

Tips for Integrating Patient Partners on Improvement Teams

• Determine how often patients will attend improvement team meetings. Some practices have meetings twice a month and integrate patients into one of the meetings. This leaves one meeting to discuss business-related issues that the practice may not be ready to share with patients. However, this approach may also create discontinuity between meetings and make it difficult for patients to follow unless meeting agenda topics do not cross between meetings, which may be difficult to achieve.
• Select two or three patients that can commit to attend the QI team meetings regularly and can provide "constructive criticism" and input to the team. Practices implementing this approach typically ask patients to make at least a 1-year commitment to being a Patient Partner.
• Create an environment where the patients are encouraged to participate and share positive and negative thoughts and experiences.
• Provide some background and training in quality improvement (QI) for Patient Partners. While Patient Partners are experts at representing the patients' perspective of the practice, they may not be familiar with QI processes, interpreting standard QI data reports, and commonly used acronyms.
• In order to make the meeting time most productive, provide some advance preparation to the Patient Partners. Many practices that have integrated Patient Partners have received support from community collaborative organizations, such as special training sessions to help them prepare for their new roles.
• Give the Patient Partners the same kinds of tasks and activities that staff members would do. For example, Patient Partners can be valuable in doing walkthroughs and conducting interviews with other patients. Similarly, give Patient Partners the ability to add issues to the team's agenda. They may identify issues from the patient perspective that staff do not recognize as problems.

Return to Contents

---

References

• Cousins M. Follow the Map; 2003. Available at: http://saferpak.com/process_mapping_art2.htm. Accessed August 2010.
• ´óÏóÊÓƵ. How Two Provider Groups Are Using the CAHPS^® Clinician & Group Survey for Quality Improvement. Available at /cahps/quality-improvement/reports-and-case-studies/cgcahps-webcast-brief-2014.pdf. Accessed July 21, 2015.

Return to Contents