Roadmap for Implementing Pediatric Quality Measures for Performance Improvement

This Roadmap serves as a guide for stakeholders planning to implement pediatric quality measures for performance improvement. It is intended for use by states, health plans, hospitals, and other providers embarking on a quality improvement project. The Roadmap provides overall guidance on key steps to consider when launching an improvement effort—from defining the project’s goal(s), to forming a team, to understanding context for quality improvement, through implementation and evaluation.

The Roadmap was developed as part of the Pediatric Quality Measures Program (PQMP), sponsored by the ��Ƶ (AHRQ) and the Centers for Medicare & Medicaid Services (CMS). The PQMP includes two phases: (1) development of new evidence-based pediatric quality measures, and (2) dissemination and implementation of the measures, with a focus on the feasibility and usability of the measures in real world settings. Using an implementation science approach, the second phase of the PQMP project seeks to understand the contextual factors—barriers or facilitators—that may impact measure uptake and quality improvement. Understanding how context plays a role in implementation can better inform the adoption of pediatric quality measures, with the ultimate goal of driving improvements in care for children.

Using the Roadmap below, follow the steps to explore the measure implementation process, including understanding contextual factors that may help or impede the project. Many of the steps include links to existing external resources. For specific information on implementing selected measures from the PQMP, refer to the PQMP Implementation Toolkits (Step 4).

Implementation Roadmap

Step 1: Defining Your Improvement Goal
Step 2: Forming an Implementation Team
Step 3: Understanding and Optimizing Your Context
Step 4: Implementing an Improvement Project
Step 5: Evaluating Implementation

Step 1: Defining Your Improvement Goal—Setting an Improvement Aim and Establishing Measures

A first step in any quality improvement effort is identifying areas for improvement based on an initial assessment and available data. Then, you must set a clear improvement goal that is specific and measurable, and based on an applicable and appropriate quality measure. Measures enable you to assess whether the changes being made are leading to improvement. Although the broader evidence base for pediatric care has historically been more limited than for adults, the PQMP has set out to increase the available measures for assessing pediatric care. For information on setting an aim and selecting pediatric measures for quality improvement, see the below links to relevant resources. An illustrative example of a PQMP measure and associated quality improvement goal is can be found below.

Example: PQMP NCINQ II Team

Resources

Setting an Improvement Aim

Establishing Measures

PQMP Resources

Illustrative Example from PQMP Toolkit (NCINQ II Team)

NCINQ (the National Collaborative for Innovation in Quality Measurement) developed a set of quality measures to assess the safe and judicious use of antipsychotic medications in children and adolescents.

Pediatric Quality Measure

Use of First-Line Psychosocial Care for Children and Adolescents on Antipsychotics.

Improvement Goal

Increase the rate of children and adolescents receiving at least one psychosocial intervention prior to starting on antipsychotic medication by 25%.

Considerations for Selecting Appropriate Improvement Goals

To identify benchmarks that could be used to set improvement goals and to subsequently monitor performance for participating health plans, the team:

Reviewed the Healthcare Effectiveness Data and Information Set (HEDIS) national performance benchmarks as well as the state benchmarks.
Conducted an “improvement analysis” using 2015 and 2016 HEDIS reporting data to calculate the amount of improvement at the plan level seen year to year. “High improving” plans were defined as those that exhibited improvement rates in the top 25th percentile.
Worked individually with the participating plans to compare baseline performance rates to the national and state benchmarks, as well as the findings from the “improvement analysis.”
Coached plans to track the amount of improvement by comparing current performance rates with their previous year performance (i.e., internal benchmarking).

Step 2: Forming an Implementation Team

Based on your improvement goal, the next step is building a team that will carry out the project. An effective team will include a range of staff with expertise related to the area identified for pediatric quality improvement, as well as relevant stakeholders including those who would be using the given measure(s). The team may include operational leaders, clinicians, quality measurement and data experts, informatics and IT experts, administrative staff, case managers, health educators, payers, patients, children, youth, parents, families and other caregivers, and community representatives. For information on forming your team, see the below links to external resources. An illustrative example of a team from one of the PQMP projects is below.

Example: PQMP CEPQM Team

Resources

References

Illustrative Example from PQMP Team (CEPQM Team)

CEPQM (the Center of Excellence for Pediatric Quality Measurement) piloted a discharge confidence assessment tool, a one-question survey administered within 24 hours of a child’s planned discharge, that asks parents about their confidence to continue care at home.

Forming the Implementation Team

The nursing teams led the implementation on each unit and were responsible for asking patient-families the question.

Keys to success: A culture of improving care delivery extended from leadership to the staff involved in this intervention. The grantee was able to leverage an existing nursing committee that had implemented several successful interventions. Additionally, nurse managers on each of the three pilot units had strong buy-in and influence among their teams, and their enthusiasm was effective in encouraging their teams’ participation.

Team Member	Role
Nurse Leader	Chaired the nursing committee and lead the Nursing Committee that spear-heads unit-based initiatives to improve care provided.
Nursing Committee	Established committee that provided a structure for discussing the initiative, piloting the QI intervention, and reviewing results.
Unit Nurse Managers	Local champions on the units that worked with the Center of Excellence (CoE) team to roll out the intervention.
Physician Champion	Contributed to the CoE to develop the intervention and QI strategy. Joined the Nursing Committee to provide input on intervention roll out.
CoE Team: QI Physician Lead Project Manager Research Assistants	Provided support to the QI teams on each unit to analyze data and work with units to increase adoption through Plan-Do-Study-Act (PDSA) cycles.
Parent Partner	Member of the CoE team. Provided input on intervention and findings.

Step 3: Understanding and Optimizing Your Context

Context plays an important role in the implementation process and can influence the degree of success achieved in a quality improvement project. Having an awareness and understanding of the contextual factors that may be at play, before initiating a project, allows you to plan for them by addressing any barriers or leveraging any facilitators. In order to better understand your context, see the link below to a self-study exercise on contextual factors. This exercise is based on the Model for Understanding Success in Quality (MUSIQ), which is a conceptual model for understanding and optimizing contextual factors affecting quality improvement efforts. Additional links to other relevant external resources are also included below, including other frameworks that can be used to assess contextual factors. An illustrative example of a contextual factor from one of the PQMP projects is also provided below.

Example: PQMP IMPLEMENT Team

Self-study Exercise on Contextual Factors Using the MUSIQ Framework

This exercise, developed by Cincinnati Children's Hospital Medical Center, allows you to score a variety of factors related to your QI team, organizational support, and the external environment to identify areas in need of attention before QI efforts are started: Excel calculator for MUSIQ¹.

Additional Resources

(PDF).
, including an to help guide choice of implementation strategies for mitigating identified barriers.

References

^1.MUSIQ Excel Calculator, from at Cincinnati Children’s Hospital Medical Center, is licensed under .

Illustrative Example from a PQMP Contextual Factor Analysis (IMPLEMENT Team)

IMPLEMENT (IMPLEMENT for Child Health initiative) tested the usability of the PQMP asthma emergency department measure by conducting quality improvement initiatives in San Francisco, California and in Burlington, Vermont.

Background on the Project

Pediatric Quality Measure:

Rate of emergency department visit use for children managed for persistent asthma.

Main System and Process Changes:

Electronic Health Record (EHR) improvements for child asthma-related process measures: Participating practices focused mostly on improvements related to implementing decision support features and improving the sites’ documentation capabilities to help capture and track the process measures. Strategies included: reconfigured visit report pull from EHR, smart forms, dot/smart phrases, asthma patient identification.

Contextual Factors Impacting Implementation

EHR Modification Costs:

The ability to modify or customize an EHR requires time and resources. For practices in large organizations, there may be many competing priorities. Unless the particular measure is a priority for an organization, changes to the EHR may require a practice to “wait their turn in line”. Having leadership buy-in to invest in the EHR implementation was key to being able to expedite the work.

Enhanced Workflow:

Sustained uptake of EHR tools in clinical care delivery depends in part on the design of the EHR tool and how much it facilitates making the right thing to do the easy thing to do. Providers indicated that using the smart form was helpful in effectively managing patients with asthma as it provided a more efficient way of documenting asthma care. However, initial utilization was not sustained, likely due to a lack of automated reminders, which were then planned for the next phase of EHR modifications.

Step 4: Implementing an Improvement Project

Once you are ready for implementation, there are a number of resources that can help with the process of deciding what changes you will make and how you will integrate those changes into your organization’s processes. This includes selecting changes; testing them, for example, through Plan-Do-Study-Act (PDSA) cycles; and then implementing them. For information on implementing specific pediatric quality measures that were created and tested as a part of the PQMP, see the link below to a set of PQMP implementation toolkits. Additional links to other more general resources on measure implementation are also included below. An illustrative example of the core toolkit elements from one of the PQMP projects is also provided below.

PQMP Implementation Toolkits

Safe and Judicious Use of Antipsychotic Medications in Children and Adolescents.
Pediatric Asthma Emergency Department Use.
CAHPS Child Hospital Survey (Child HCAHPS).
Quality of Pediatric Hospital-to-Home Transitions.
Transcranial Doppler Ultrasonography (TCD) Screening Among Children with Sickle Cell Anemia.

Example: PQMP IMPLEMENT Team

Resources

Institute for Healthcare Improvement Resource: .
Institute for Healthcare Improvement Resources: ; ; and .

References

Illustrative Example from PQMP Toolkit (Q-METRIC Team)

QMETRIC (Quality Measurement, Evaluation, Testing, Review, and Implementation Consortium) sought to implement sickle cell measures at the state, health plan, and provider levels.

Core Elements Contained in the Toolkit

Overview:

Intended End Users.
Measure Implementation Feasibility at State, Health Plan and Health System Levels.
Methods.
Results.
Conclusions.

About the Measure:

Measure Specifications.
Measure Reporting.

Key Driver Diagram:

Lessons Learned in Using the Transcranial Doppler Ultrasonography Measure at Multiple Levels.

Quality Improvement Strategies:

Strategic Road Maps.
Multi-Stakeholder Approach to Sickle Cell Anemia Quality Improvement.

Improvement Data:

Michigan and New York Medicaid Program.
Collaborating Medicaid Health Plans.
Collaborating Health Systems/Sickle Cell Clinics.

Other Resources.

Step 5: Evaluating Implementation

A key tool for assessing performance in quality improvement is a statistical process control chart, which allows you to visually plot your performance data over time in a graph or chart to see and better understand any variation that may be occurring. For information on how to assess your quality improvement efforts using statistical process control tools, see the links below to external resources. An illustrative example of a statistical process control chart used in the PQMP is also described and provided below.

Additionally, to learn more about how or why an intervention did or did not result in improvements, a process or summative evaluation may be conducted. A process evaluation allows you to monitor the implementation process throughout the course of a QI initiative to assess how implementation is proceeding and if changes are being implemented as intended. A summative evaluation is planned throughout the project and focused on assessing if the planned changes were successful in changing outcomes, based on data collected during the program. Links to general external references on evaluation are included below.

Example: PQMP P-HIP Team

Resources

Institute for Healthcare Improvement Resource: Videos on Statistical Process Control Charts ( and ).
(PDF).

References

.
.
(PDF).
.
.
.

Illustrative Example from PQMP Toolkit (P-HIP Team)

P-HIP (the Pediatric Hospital Care Improvement Project) sought to improve documentation and reporting on hospital-to-home transitions through timely and feasible data collection, uploading, and measure scoring.

Statistical Process Control Chart

The Pediatric Hospital Care Improvement Project (P-HIP) Collaborative collected Hospital-to-Home Transitions data from eight participating hospitals for nearly 4.5 years in order to analyze results over time. The analysis includes 24 months of data before the interventions were implemented to allow researchers to establish a baseline for comparison purposes.

The statistical process control charts that were used are available for site-specific results for each of the eight participating hospitals at this link and combined results for the overall collaborative are shown below and at this link.

SHARE: