´óÏóÊÓƵ

Purpose of This Tool

Monitoring the RED lets you know whether each component of RED is being successfully implemented and whether your hospital is achieving the expected outcomes. This information can be used to:

• Identify and address challenges and barriers in the implementation process.
• Hold staff accountable for performance goals.
• Justify further investments in spreading the RED.

Monitoring helps staff learn from missteps. In addition, monitoring provides opportunities for adapting the implementation process to local conditions. This tool discusses implementation and outcome measures and the way to use such data for continuous quality improvement in the area of care transitions.

Getting Started

Form a monitoring team to develop a monitoring plan before the RED implementation begins. You may assign this responsibility to the interdisciplinary implementation team that is tasked with improving care transitions. As described in Tool 2, "How To Begin RED Implementation at Your Hospital," the implementation team should include representatives from such disciplines as nursing, medicine, case management, and information technology.

At times, personnel such as statisticians or health economists may be needed to ensure success. Your monitoring plan will describe what measures will be collected, how frequently they will be collected, who will collect them, and how the data will be used.

Selecting and Specifying Measures

This tool offers a menu of many different possible measures you can use to monitor the RED. You will want to choose a manageable number to track. Your measurement set does not have to be static. You may want to phase in certain measures as your implementation of the RED evolves.

You need to select a combination of implementation and outcome measures that are:

• Meaningful (understandable and important for accountability and/or quality improvement).
• Credible (based on reliable data).
• Feasible (can be produced without undue burden).
• Timely (provide information in time for corrective action and decisionmaking).

Almost all the measures in this tool are rates and are expressed as percentages. That means that both the numerator and the denominator have to be adequately specified. For many of the measures, you can use the target population as the denominator. Using this approach means you measure the implementation and impact of the RED on everyone you intended to give the RED, whether or not they in fact got the RED.

You could also use the population that is receiving any RED component as your denominator. This will allow you to measure the implementation and impact of the RED on those patients who actually receive the RED (or part of it). You may, however, find it difficult to calculate some measures using this denominator. For example, your hospital may be unable to compute outcome measures (e.g., readmission to the hospital) for only those patients who received the RED as opposed to the entire target population.

For some of the measures suggested in this tool, the appropriate denominator is actually a subset of patients who receive the RED. For example, you need to look only at patients for whom postdischarge tests have been ordered when measuring the percentage of patients for whom postdischarge tests have been scheduled.

Various organizations have suggested discharge-related measures (e.g., National Quality Forum, American College of Cardiology Hospital to Home Program, Joint American Board of Internal Medicine, American College of Physicians, and Society of Hospital Medicine Care Transitions Program). "Discharge Measures Used by Other Organizations," at the end of this tool, has additional information about the measures and provides addresses for the organizations' Web sites. The rest of this section of this tool will cover possible implementation and outcome measures for the RED.

Implementation Measures

To have the desired impact, the RED must be properly implemented. It is therefore important to measure the extent of RED implementation. Implementation measures can be used to identify where the implementation process can be improved. Take, for example, RED component #2, ensuring that all patients are discharged with an appointment for posthospital followup. Monitoring the percentage of patients who leave the hospital with an appointment with a primary care provider gives an opportunity to see whether most patients are leaving with appointments, and to take corrective action if they are not.

Is the RED Being Delivered to Target Patients?

Before measuring how well components of the RED are being implemented, you will need to know the proportion of patients targeted to receive the RED who are actually getting it. This is true whether you are implementing the RED for your entire hospital population or for a subset of your hospital's patients (e.g., patients with a specific condition or in a specific unit). (Go to Tool 2, "How To Begin the Re-Engineered Discharge Implementation at Your Hospital," for a discussion of the options for selecting a RED target population.)

Two measures to consider are:

• Percentage of the target population receiving any RED component.
• Percentage of the target population receiving all RED components currently being implemented.

The first measure tells you the proportion of targeted patients the RED is reaching at all, while the second measure tells you how comprehensively the RED is being implemented.

Is the Correct Information Being Collected?

To deliver the RED properly, the hospital (often the discharge educator, known as the DE) needs to collect information from the patient and sometimes the patient's caregiver. The following measures give an indication of whether the information needed for a re-engineered discharge is being collected.

• Percentage who were asked about language preference for oral communication, phone calls, and written materials.
• Percentage who were asked about English proficiency and need for interpreter services (of those whose language preference for oral communication is not English).
• Percentage who were asked about English proficiency and need for translation services (of those whose language preference for written materials is not English).
• Percentage who were asked about the best time for appointments.
• Percentage with whom the ability to keep appointments was discussed.
• Percentage who were asked about interest in treatment for addiction (of patients diagnosed as having addiction to tobacco, alcohol, or other substances).
• Percentage who were asked about traditional healers, treatments, and dietary supplements.
• Percentage who report that hospital staff asked whether they would have the help they needed when they left the hospital (of those who complete that survey question).
• Percentage who report that nurses always or usually listened carefully (of those who completed that survey question).

Is Evidence-Based Care Being Delivered?

Several RED components are designed to ensure that the care being delivered is evidence based. The following measures examine whether both clinical treatment and the discharge are following standards and recommendations of national organizations:

• Percentage for which medication reconciliation was completed.
• Percentage for which the discharge plan has been reconciled with national guidelines.
• Percentage for which a discharge summary is delivered to the clinicians accepting care of the patient within 24 hours of discharge.
• Average time between discharge and delivery of discharge summary to the primary care provider. (Note: this measure is not a rate and therefore does not have a denominator.)

Is Appropriate Followup Care Being Arranged?

An important part of the RED is making arrangements for posthospital care. The following are some measures of whether the hospital is making those arrangements:

• Percentage with an appointment with clinicians accepting care of the patient.
• Percentage with appointments for tests and labs (of those for whom tests and labs were ordered).
• Percentage with delivery dates for durable medical equipment (of patients needing new equipment).
• Percentage with appointments for postdischarge services (e.g., visiting nurse services) (of patients for whom postdischarge services have been ordered).
• Percentage who were provided with information for addiction treatment (of patients diagnosed as having addiction to tobacco, alcohol, or other substances).

Are Patients Being Prepared for Discharge?

Measures of the teaching and educational components of the RED include:

• Percentage who received qualified interpreters for all encounters with a DE or whose DE was assessed as proficient in patient's preferred language for oral communication (of those without English proficiency).
• Percentage who got education about all diagnoses.
• Percentage who were instructed on how to take medicines (of patients prescribed medicine).
• Percentage who report that hospital staff explained the purpose of a medicine in a way that was easy to understand (of those who completed that survey question).
• Percentage who report that hospital staff explained in a way that was easy to understand how much to take of each medicine and when to take it (of those who completed that survey question).
• Percentage with plan to obtain medicines (of patients prescribed medicine).
• Percentage who got instruction on nutrition and exercise and activity limitations.
• Percentage whose ability to make scheduled appointments was confirmed.
• Percentage who were told what to do if problems arise.
• Percentage who report that written information about what symptoms or health problems to look out for after discharge was easy to understand (of those who completed that survey question).
• Percentage whose understanding of information and instructions was confirmed.
• Percentage who report that nurses explained things in a way that was easy to understand (of those who completed that survey question).
• Percentage who report that all questions were answered satisfactorily by the DE (of those who completed that survey question).
• Percentage who were told how they would get pending test results (of patients with pending test results).
• Percentage who received the AHCP.
• Percentage who received AHCP in English and preferred language for written materials (of patients whose preferred language for written materials is not English).

Are Patients Receiving Postdischarge Care?

• Percentage who received postdischarge followup phone call.
• Percentage who received postdischarge followup phone call in preferred language for phone communication or with interpreter (of those whose preferred language for oral communication is not English).
• Percentage who received postdischarge followup phone call within 3 days.
• Average time between discharge and postdischarge followup phone call. (Note: this measure is not a rate and therefore does not have a denominator.)
• Percentage who strongly agree or agree that all questions about medical care were answered during postdischarge followup phone call (of those who completed that survey question).

Selecting Implementation Measures

The preceding sections contain a broad array of measures that can help you assess how successfully you have implemented the RED. One approach to selecting from among them is illustrated in Table 1. Specifically, you may wish to select one or two measures that will enable you to assess implementation of each individual component of the RED. The table also indicates where to collect and record the relevant data.

Table 1. Implementation Measures by RED Component

Outcome Measures

Once the RED has been fully implemented, it can be expected to have an impact on a number of outcomes. These outcomes include:

• Reduced hospital reutilization (i.e., patients returning to the hospital shortly after discharge).
• Improved connections with primary care and other providers.
• Increased knowledge for self-management.
• Increased patient satisfaction.

In addition to monitoring for expected results of implementing the RED, you need to monitor for unintended consequences. For example, in the clinical trial of the RED, the average time of discharge was monitored. When it was discovered that discharges were occurring 30 minutes later in the day after the RED was introduced, RED implementers were able to modify RED processes to complete discharges at the same time of day as before RED implementation. Length of stay is another measure you may want to monitor to catch unintended consequences.

In addition to generating outcome measures for patients targeted to receive or receiving the RED, you may want to generate measures for a comparison group. A comparison group might be patients in a similar hospital or comparable patients in your hospital who are not receiving the RED. Finding differences between RED and non-RED patients will add to your confidence that the RED was in fact responsible for changes in outcome measures observed over time.

Hospital Reutilization Measures

Most hospitals focus on the use of hospital services in the 30 days following discharge. While you could choose to evaluate the RED on whether it averts hospital reutilizations related to the original primary diagnosis, the comprehensiveness of the RED means it should be able to have an impact on 30-day all-cause hospital reutilization.

Furthermore, the Centers for Medicare & Medicaid Services (CMS) uses disease-specific risk-adjusted 30-day all-cause readmission rates when gauging excess hospital readmission. CMS currently reduces payments to hospitals with excess readmissions for three conditions: acute myocardial infarction, heart failure, and pneumonia. There are plans to expand beyond this "starter set" in the next few years.

The rates that you calculate will differ from CMS's rates because they will not be risk adjusted and will not include readmissions to other hospitals. (Select for How Centers for Medicare & Medicaid Services (CMS) Measures the "30-Day All-Cause Rehospitalization Rate" on the Hospital Compare Web Site). If you want to try to capture your patients' postdischarge use of other hospitals, you can use patient surveys. These reports, however, rely on patients' recall of what hospital services they used in the 30 days after leaving the hospital.

Common measures of 30-day all-cause hospital reutilization are percentages of patients with:

• All-cause readmissions (admission >24 hours) within 30 days of discharge.
• All-cause observations (admission <24 hours) within 30 days of discharge.
• All-cause emergency department visits within 30 days of discharge.
• All-cause urgent care visits within 30 days of discharge.

These rates can be reported separately or in combination.

Consider examining reutilization rates by subsets to identify important opportunities for improvement. For example, if a specific nursing home was shown to have high rates of what appear to be avoidable events, your hospital could work with the nursing home to determine the source of this problem and take corrective action. The report ¹ suggests that hospitals could examine readmissions data in the following ways:

• Rates for different conditions: To the extent feasible, examine readmission rates by diagnosis and significant comorbidities, and look for correlation with the patient's severity.
• Rates by practitioner: Examine the rates by physician, physician group, and service to determine if the patterns of readmissions are appropriate or if any type of practitioner or groups/services are associated with an unexpected readmission rate or trend for certain diagnostic groups.
• Rates by readmission source: Examine the rates by readmission source (e.g., home, nursing home) to determine the places from which patients are most often being readmitted.
• Rates at different times: Examine readmissions within a given time period, such as 7, 30, 60, and 90 days. Examining a shorter timeframe may bring to light issues more directly related to hospital care or flaws in the process of transitioning the patient to the ambulatory setting. Examining the longer timeframe may reveal issues with followup care and patients' understanding of self-care or the hospital's ability to arrange posthospital care.
• Rates by sociodemographics: Examine readmissions by race, ethnicity, neighborhood (ZIP Code), and language preference to identify disparities and the adequacy of language and cultural services for patients throughout the transition process.
• Rates by insurance: Examine readmissions by insurer type to ensure the appropriate use of benefits and identify the ways patients may be guided to optimize their benefits.

If outliers are identified, you may want to conduct a root cause analysis to figure out why they are experiencing high rates. (Go to the section below.)

Connections With Outpatient Providers

Research shows that patients who return to the hospital often have not seen a clinician since they left the hospital.² One of the most important components of the RED is that all patients go home with a followup appointment made in such a way that there is a good chance they will keep it.

As described in Tool 3, "How To Deliver the Re-Engineered Discharge," the DE works with the patient to identify a date and time that the patient can attend a posthospital followup appointment. To monitor how successful your hospital has been in arranging an appointment that the patient can keep, you can measure:

• Percentage who completed an appointment with their medical provider (of those who completed that survey question).

Knowledge for Self-Management

An important objective of the RED is to teach patients how to take care of themselves when they get home. The postdischarge followup phone call provides an opportunity to monitor whether teaching done in the hospital has improved the patients' knowledge of how to self-manage their conditions. Possible measures include:

• Percentage who correctly report during postdischarge followup phone call the reason for their hospital visit (of those who completed a postdischarge followup phone call).
• Percentage who correctly report during postdischarge followup phone call the symptoms to watch out for or things to do for their condition (of those who completed a postdischarge followup phone call).
• Percentage who correctly report during postdischarge followup phone call how to take their medicines (of those who completed a postdischarge followup phone call and had prescribed medicines).

Patient Satisfaction

The RED is designed to improve communication, which in the RED trial was shown to improve overall patient satisfaction. Measures of patient satisfaction with hospitals include:

• Percentage who rate hospital a 9 or 10 on a 1 to 10 scale (of those who completed that survey question).
• Percentage who would probably or definitely recommend your hospital to friends and family (of those who completed that survey question).
• Percentage who report nurses always or usually treated them with courtesy and respect (of those who completed that survey question).
• Percentage who report doctors always or usually treated them with courtesy and respect (of those who completed that survey question).

Collecting Data

Your monitoring plan will specify who will collect data and how data for each measure will be collected. This section of the tool discusses the various sources of data you can use to calculate measures. Expect to modify your data collection to generate the measures that you choose.

RED Workbooks and Contact Sheets

Much of the data for monitoring RED implementation can be collected and recorded in the DE's Workbook. You can find a copy of the Workbook in Tool 3, "How To Deliver the Re-Engineered Discharge." If your hospital is not using the RED Workstation, you will have to manually calculate the measures using a spreadsheet. This can be done by conducting a chart review for at least 10 percent of RED patients, defined as either patients receiving any RED component or as the entire target population that was supposed to receive the RED, whether or not they received any RED components.

Electronic Health Records and the RED Workstation

Some data needed for calculating measures will be available from your hospital's EHRs (if you have EHRs). For example, if your hospital routinely collects language preference data at admission, the measure of the percentage of patients asked about language preferences could be generated by using EHRs.

If your hospital is entering data from the Workbook into a RED Workstation, the Workstation can be programmed to generate many of the implementation measures automatically. The RED Workstation can also be linked to your EHR system so that it can pull data needed for measure calculation.

Patient Surveys

Some measures, such as the patient satisfaction measures, require gathering data from patients after discharge. The patient survey should be conducted shortly after discharge (e.g., within 6 weeks). You can add questions to patient surveys your hospital already conducts to assess patient experiences and satisfaction with care if you implement the RED with your hospital's entire patient population. If, however, you implement the RED with a subset of patients, these anonymous surveys will not allow you to distinguish between RED patients and others. Therefore, you will need to field a separate patient survey to monitor the RED.

You can administer the surveys by mailing a survey to the patient after discharge or by administering the survey by phone. Mail surveys are less expensive but typically have low response rates. You may therefore choose to conduct telephone followup with patients who do not respond to a mail survey.

This tool includes mail and phone versions of a survey that will assist you in collecting data from patients. These surveys do not capture all the data needed to calculate all the measures listed in this tool, but you can individualize the survey for your hospital to reflect your priorities and goals. A number of the survey items were developed for , the hospital survey of patients' experience of care, and have been validated as part of the °ä´¡±á±Ê³§Â® development process. (°ä´¡±á±Ê³§Â® is the Consumer Assessment of Healthcare Providers and Systems, and HCAHPS is the hospital version.)

Measurement Timing and Frequency

When and how often you want to generate RED measures will depend on the measure, the amount of progress you have made in implementing the RED, and the level of effort it takes to generate the measure.

For outcome measures, it is useful to get a baseline measurement, that is, to calculate the measure before RED implementation begins. Ideally, you will calculate outcome measures for an extended time (e.g., the year before implementation) rather than only at one point in time (e.g., the month before implementation). You may then be able to identify seasonal variations or trends over time that will help you interpret your measures after RED implementation.

Subsequent calculations of outcome measures will depend on how quickly you expect the measure to respond to the RED. Set reasonable expectations for improvement. It will take a while before the RED is fully and properly implemented, so there will be a lag in measurable results. For example, you should see an impact on patient satisfaction among those who receive the RED almost immediately, but the impact on the overall hospital patient satisfaction scores depends on the percentage of the hospital's patients that receives the RED.

Consider producing outcome measures that can be generated using electronic data on a monthly basis and track changes over time as well as differences from baseline. You may want to produce outcome measures that require patient surveys or spreadsheet calculations less frequently. You can, however, reduce the burden of producing those measures by using only a sample, rather than the universe, of RED patients.

Implementation measures can be calculated as soon as 30 patients have received the RED. At first, you will need to generate these very frequently (e.g., every other week). If electronic data cannot be used, you can generate these measures using a small sample of patients. The idea, however, is to generate these measures quickly so they can be used for continuous quality improvement.

As the RED matures, you may want to reduce the frequency of measurement. For example, once it has been determined that the RED components have been fully integrated into standard operating procedures, you may want to generate the measures that monitor those components infrequently or even drop them from the measurement set. Once a steady state has been reached, you may want to harmonize the intervals for RED outcome measurement with other key quality measures your hospital monitors.

Other Means of Monitoring the RED

Qualitative methods can be useful to monitor RED implementation and outcomes and develop strategies for quality improvement.

Root Cause Analysis

Root cause analysis is the study of when things go wrong to identify ways bad outcomes can be prevented. The goal is to identify underlying trouble that increases the likelihood of problems while resisting the urge to focus on mistakes by individuals.^3-5

You start by identifying patients who have experienced a bad outcome, such as an avoidable readmission. Using a systematic approach, such as conducting chart reviews and structured interviews with patients, DEs, and other providers, you will uncover the underlying failures in the care process. Ideally, you will select cases from several different clinical units and include patients with varying diagnoses.

After you conduct a series of root cause analyses, you are likely to understand where processes are breaking down. If you conduct such analyses at least monthly during RED implementation, the findings will provide valuable feedback about times when the RED process is not working and advice about how to adapt the RED processes for your hospital. These monthly discussions can also generate enthusiasm for an organizational culture that emphasizes the importance of improving transitions of care.

Discharge Educator Help Line Logs

The logs that DEs keep of patients who call the help line can help identify systematic problems. The RED Workstation can identify patterns in reports of:

• Postdischarge unanticipated problems.
• Postdischarge areas of confusion or uncertainty.
• Need for additional social support services.

Direct Observation

Many measures of RED implementation rely on staff self-report in the Workbooks and contact sheets. To augment these reports, direct observation by staff overseeing the delivery of the RED can give insight into implementation glitches.

Taking Action

Monitoring the RED improves outcomes only if staff review the results and take action. Forums for reviewing RED monitoring data include:

• Weekly meetings of DE and other members of the frontline RED clinical team.
• Monthly meetings of the RED implementation team.
• Reviews of other key quality indicators (e.g., hospital board meetings, quality committees).
• Senior management resource allocation meetings.

When areas for improvement are identified, rapid-cycle, continuous quality improvement methods can be implemented to improve care delivered by individual providers, units, and systems. Once a process that needs improvement is identified, a team representing various stakeholders is gathered to understand the process and learn what can be done. Action to prevent future failures involves reaching consensus for what changes are needed, setting goals, transforming processes and educating staff, and measuring results. If necessary, the implementation plan may be revised based on the results so that improvement is ongoing.

Summary

Developing a monitoring plan as suggested in this tool will help you identify what is going well and what needs to be improved. Monitoring the RED implementation measures lets you know whether you are successfully implementing the components of the RED, and monitoring the outcome measures tells if you are achieving the expected results. Collecting data should be limited to only that information that will help you determine if you are achieving the goals that you set for your hospital in the area of transitions in care.