´óÏóÊÓƵ

Background

In 2000, the U.S. Office of the Inspector General recommended developing a standardized experience of care survey of patients with end-stage renal disease (ESRD) in order to provide valid comparative information to the public. MedPac's Report to Congress in 2003 also reiterated the need to evaluate ESRD patient satisfaction. In response to these recommendations and to ensure quality of care for dialysis patients, the Centers for Medicare & Medicaid Services (CMS) decided to work with the ´óÏóÊÓƵ (AHRQ) to develop a survey that could be used both for internal quality improvement at the facility level and for public reporting to patients and caregivers. CMS has been reporting comparative clinical information at the facility level since January of 2001 on its .

The development of this CAHPS survey involved several steps, including a review of existing surveys, a scientific assessment of potential items, public input, cognitive testing, and field testing, which includes psychometric analyses of the performance of items and their domains. All of these steps contributed to the drafting of the questionnaire and subsequent revisions.

Feasibility Report

CMS recognized that the development of a patient experience survey for ESRD might be significantly more challenging and less straightforward than surveys for other facilities and populations. For the first time, development of a CAHPS survey was focused on a specific chronic disease population. Moreover, ESRD is a particularly complex condition: patients require either dialysis or a kidney transplant to survive, and typically have comorbid conditions such as diabetes and hypertension, high levels of cognitive impairment, and low levels of literacy. Consequently, CMS asked that ´óÏóÊÓƵprovide guidance in a formal report concerning the proper balance between a survey aimed primarily at internal quality improvement to aid facilities and one that would focus on public reporting to consumers.

In October 2003, the CAHPS Consortium submitted a Feasibility Report to CMS. This report was based on a review of the literature, as well as interviews and focus groups with

• Patients and their families.
• Nephrologists and other health care professionals.
• ESRD Network executives and facility staff.

The Report recommended that CMS develop a standardized survey specifically for in-center hemodialysis patients to help assess their experiences with care. Surveys for patients receiving other dialysis modalities, such as peritoneal dialysis, could be developed in the future.

If you would like a copy of the Feasibility Report, please contact the CAHPS Help Line at 1-800-492-9261 or cahps1@westat.com.

Meetings With Technical Experts

CMS, AHRQ, and the CAHPS grantees met several times with a Technical Expert Panel (TEP) representing a variety of stakeholders, including facilities, ESRD Networks, providers, researchers, patients, and patient advocates. Meetings took place in June and October of 2003 and March and September of 2004. At these meetings, the TEP discussed various issues related to assessing ESRD patients' experiences with care and provided feedback on different versions of the instrument.

In addition, the team presented information about the survey development and testing process to a special ESRD Stakeholders Meeting convened by CMS in Baltimore, Maryland, on March 25, 2004. At that meeting, Network executives, facility administrators and other facility staff, and nephrologists, among others, provided feedback and suggestions to the CAHPS team on the public comment version of the survey and field test protocol.

Federal Register Notices

In anticipation of developing a survey, ´óÏóÊÓƵposted a formal call for measures in the Federal Register on August 25, 2003, and received public comments through October 24, 2003. The call resulted in the submission of eleven surveys, which were supplemented by two renal-specific surveys identified in the original literature review. The team evaluated each survey received in light of the scientific evidence that supported it.

CMS published additional Federal Register notices on January 30, 2004, requesting comments on the draft instrument and on July 23, 2004, requesting comments on the administration options for the pilot test.

Cognitive Testing

In early spring of 2004, the instrument underwent an initial round of cognitive testing with a total of 30 dialysis patients. The team used the cognitive testing findings along with the public comments ´óÏóÊÓƵreceived as the basis for revisions to the draft.

A second round of cognitive testing was conducted in May and June of 2004. The survey was further revised, then translated into Spanish, and prepared for a third round of cognitive testing with Spanish-speaking dialysis patients, completed in September 2004.

Pilot Testing

The CAHPS grantees conducted a pilot test during the winter of 2004-2005 with a representative sample of 32 facilities that varied with respect to region, size of facility, rural/urban location, hospital affiliation, and tax status. A sample size of 3,143 patients was used. The final number of respondents was 1,454, with an overall response rate of 46%: 56% responded by telephone, while 44% responded by mail.

The purpose of the pilot test was to examine the reliability and validity of the draft survey questions, identify those questions most robust for public reporting and those most appropriate for internal quality improvement, and evaluate different data collection strategies. Specifically, researchers involved in the pilot did the following:

• Evaluated the psychometric properties of the questions with the aim of reducing the number of items in the final survey.
• Tested two modes of administration (telephone; mail with telephone follow-up).
• Examined the need for assistance in completing the survey.
• Tracked the involvement of facilities in promoting the survey.
• Evaluated the use of Spanish interpreters.

The field test showed that most items on the survey performed well in the field; based on psychometric criteria, only four items were recommended for deletion. The final instrument reflects these recommendations.

Endorsement by NQF

The National Quality Forum first endorsed the CAHPS In-Center Hemodialysis Survey as a measure in November 2007. The survey’s endorsement was renewed in January 2015. 

Learn More About the Development of the CAHPS In-Center Hemodialysis Survey

In presentations at the 10th National CAHPS User Group Meeting, CAHPS grantees and staff from the Centers for Medicare & Medicaid Services (CMS) discussed the rationale, development, and testing of the CAHPS In-Center Hemodialysis Survey.

• (PDF, 81.84 KB) Barbara Crawley, M.Sc., Social Science Research Analyst, CMS
• (PDF, 78.71 KB) Barbara Crawley, M.Sc., Social Science Research Analyst, CMS
• (PDF, 434.94 KB) Patricia Gallagher, Ph.D., Senior Research Fellow, Center for Survey Research, University of Massachusetts – Boston
• (PDF, 78.85) Roger Levine, Ph.D., American Institutes for Research
•  (PDF, 84.93 KB) Beverly Weidmer Ocampo, M.A., Survey Director, RAND Corporation