Best Hospitals: Methodology

The Leapfrog Group is a nonprofit organization that aims to help consumers make informed decisions about their health and access safe, high-value care. For over two decades, it has collected and published data on patient safety and quality of care from voluntary surveys and other sources. More than 2,200 hospitals completed its flagship Leapfrog Hospital Survey in 2021.

To compile its new list of the top hospitals in the country in partnership with Money, the Leapfrog Group used data from the 39 performance measures in the seven different categories in its 2021 hospital survey, which participants were required to submit by August 31, 2021. Those categories are inpatient care management, adult and pediatric complex surgeries, pediatric care, medication safety, maternity care, outpatient procedures and infections.

To qualify for the list, eligible hospitals were required to carry an A letter grade on the Leapfrog Hospital Safety Grade assessment for fall 2021. Hospitals were also required to meet a qualitative assessment of excellence, and were disqualified if they exceeded national mortality rates for heart attacks, heart failure, pneumonia, chronic obstructive pulmonary disease or coronary bypass surgery or if they earned a C or worse on the 2022 spring Safety Grade assessment.

Leapfrog applied additional criteria related to safe medication ordering practices (to prevent medication errors), intensive care unit staffing, the response to “never events” (medical errors that are so serious they should never happen to patients, like operating on the wrong body part or a transfusion of the wrong type of blood), and performance related to certain high-risk surgeries including heart surgeries, bariatric surgery, hip replacements and lung cancer surgeries.

The group also evaluated criteria specific to pediatric hospitals, including safety and management procedures related to keeping radiation exposure during CT scans at safe levels for children and assessments related to the overall experience of care for children and their families. Some criteria for safe medication ordering, high risk procedures as well as the qualitative assessment of excellence were not applied to pediatric hospitals.

The 148 hospitals that made the cut are divided into categories: general hospitals, children’s hospitals, rural hospitals, large teaching hospitals (more than 500 staffed beds), and small teaching hospitals (fewer than 500 staffed beds).

For more information on Leapfrog’s standards, visit Leapfrog’s hospital survey measure page.

For the complete methodology, read below.

For the Top Hospital list, the following criteria were applied (criteria applies to all hospital types unless indicated otherwise):

I. Safe Medication Ordering: A hospital must achieve Leapfrog’s standard for Computerized Physician Order Entry (CPOE).

Note: For pediatric hospitals, the CPOE Evaluation Tool does not apply.

Doctors and other licensed prescribers should order medications through a computer that alerts them to serious ordering errors such as a wrong dose or a patient allergy to reduce and prevent adverse drug events. Hospitals report on both the utilization of computerized physician order entry (CPOE) and the effectiveness of that system in alerting prescribers to ordering errors. For more information, see: https://ratings.leapfroggroup.org/measure/hospital/safe-medication-ordering.

Leapfrog’s standard for Computerized Physician Order Entry (CPOE) measures the extent to which a hospital has adopted CPOE, and whether decision-support tools in the CPOE system are working effectively. On the 2021 Leapfrog Hospital Survey, hospitals had the option to report on implementation of CPOE only or to report on CPOE implementation status and participate in the Adult Inpatient Test via the CPOE Evaluation Tool. To achieve this standard based on implementation only, physicians must enter at least 85% of medication orders through a CPOE system. To achieve the standard based on implementation and score on the CPOE Evaluation Tool, physicians must enter at least 75% of medication orders through a CPOE system and demonstrate via an online, timed evaluation, that their inpatient CPOE system can alert physicians to at least 60% of common, serious prescribing errors.

CPOE systems can reduce the number of ADEs by up to 88%¹, preventing three million serious medication errors in the U.S. each year².

II. Specially Trained Doctors Care for Critical Care Patients: A hospital must achieve Leapfrog’s standard for ICU Physician Staffing (IPS).

NOTE: Rural hospitals need to earn either Achieved the Standard or Considerable Achievement to meet the criterion.

Hospitals are asked whether physicians who are board-certified in critical care medicine (known as “intensivists”) care for adult and pediatric critical care patients, the extent of their coverage of that care, and whether they are responsible for admission and discharge decisions. A critical care unit or Intensive Care Unit (ICU) is a special part of the hospital that provides care for extremely ill patients. For more information, see: https://ratings.leapfroggroup.org/measure/hospital/critical-care.

Hospitals achieving Leapfrog’s standard for ICU Physician Staffing must operate adult and/or pediatric ICUs that are managed or co-managed by intensivists who: a) Are present during daytime hours and provide clinical care exclusively in the ICU OR are present via telemedicine 24/7, with some on-site intensivist presence, and, b) When not present on site or via telemedicine, return pages at least 95% of the time (i) within five minutes and (ii) arrange for a certified physician or physician extender to reach ICU patients within five minutes. Hospitals that do not have an ICU are not assessed on this standard in the evaluation of Top Hospitals.

Mortality rates are significantly lower in hospitals with ICUs managed exclusively by board-certified intensivists (physicians trained in critical care medicine). Research has shown that there is a 30% reduction in overall hospital mortality and a 40% reduction in ICU mortality in ICUs where intensivists manage or co-manage all patients³.

III. Responding to Never Events: A hospital must fully comply with the elements of Leapfrog’s Never Events policy.

Although rare, some medical errors are so serious that experts agree they should never happen to a patient. These kinds of errors include leaving an object inside a patient’s body after surgery or operating on the wrong body part. Hospitals report on their adoption of nine (9) actions that should occur following a never event. For more information, see: https://ratings.leapfroggroup.org/measure/hospital/responding-never-events.

Leapfrog utilizes the National Quality Forum's list of serious reportable events in asking hospitals to adopt a Never Events policy. Leapfrog's Never Events policy asks hospitals to commit to nine basic acts if a Never Event does occur: apologize to the patient and family, waive all costs related to the event and follow-up care, report the event to an external agency, conduct a root-cause analysis of how and why the event occurred, make a copy of this policy available to patients, interview patients/families to inform root cause analysis, inform patient/families of actions taken by hospital to prevent similar Never Events in the future, have a protocol to provide support for caregivers involved in Never Events, and perform an annual review to ensure compliance with Leapfrog’s Never Events Policy for each Never Event that occurred. Hospitals that achieve Leapfrog’s standard have all nine elements of the policy in place and are demonstrating their commitment to treating patients, purchasers, and payers with respect when a Never Event occurs.

IV. Experience of Children and Their Parents: A hospital must achieve, or make considerable progress towards achieving Leapfrog’s standard for Patient Experience, based on responses to the CAHPS Child Hospital Survey.

Note: This criterion only applies to pediatric hospitals.

Hospitals that care for children should survey those children and their families on important issues like communication with doctors and nurses, communication about medicines, and whether they were told how to report problems they observed. Hospitals report on the results of a nationally standardized patient experience survey called the CAHPS Child Hospital Survey. For more information, see: https://ratings.leapfroggroup.org/measure/hospital/experience-children-and-their-parents.

Leapfrog assesses children’s hospitals based on a subset of the domains included on the CAHPS Child Hospital Survey: 1) Communication with Parent – Communication about your child’s medicines, 2) Communication with Parent – Keeping you informed about your child’s care, 3) Communication with Child – How well nurses communicate with your child, 4) Communication with Child – How well doctors communicate with your child, and 5) Attention to Safety and Comfort – Preventing mistakes and helping you report concerns. Hospitals that achieve Leapfrog’s standard are in the top percentile of hospitals, based on points achieved through aggregate scoring of individual domains’ Top Box Scores. Quartiles were determined using hospital performance reported in 2021 Leapfrog Hospital Surveys submitted by August 31, 2021. Hospitals that had fewer than 100 returned CAHPS Child Hospital Surveys, had too few pediatric inpatient admissions (n < 500) to administer the CAHPS Child Hospital Survey, or had fewer than 100 non-NICU pediatric inpatient admissions during the reporting period are not assessed on this standard in the evaluation of Top Hospitals.

V. Radiation Dose for Abdomen/Pelvis and Head Scans: A hospital must achieve, or make considerable progress towards achieving, Leapfrog’s standards for Head and Abdomen/Pelvis Pediatric Computed Tomography (CT) Radiation Doses.

Note: This criterion only applies to pediatric hospitals.

Because radiation used for Computed Tomography (CT) is far higher than conventional radiographs (x-rays), it is important for hospitals to monitor their dosage when scanning children given their smaller size and lower body weights. Hospitals report their doses for routine abdomen/pelvis and head CT scans. For more information, see: https://ratings.leapfroggroup.org/measure/hospital/pediatric-care.

Leapfrog assesses hospitals on their performance for head scans and abdomen/pelvis scans separately by comparing the median radiation dose length product (DLP) for each anatomic region and age stratum to two benchmarks. The first benchmark is the Median Benchmark, which is the median of the median doses reported across all Leapfrog-reporting hospitals from 2021 Leapfrog Hospital Surveys submitted by August 31, 2021. The second benchmark is the 75th Percentile Benchmark, which is the median of the 75th percentile doses reported across all Leapfrog-reporting hospitals from 2021 Leapfrog Hospital Surveys submitted by August 31, 2021.

Hospitals that achieve Leapfrog’s standard on each type of body scan are in the top percentile of hospitals, based on points achieved through aggregate scoring of individual age range strata. Hospitals that do not perform CT scans on pediatric patients or have fewer than 10 CT scans for all age ranges for a type of body scan are not assessed on one or both standards in the evaluation of Top Hospitals.

Given that doses of radiation used for Computed Tomography (CT) are far higher than conventional radiographs (x-rays), it is important for hospitals to review the dosage exposure for their patients, especially pediatric patients given their smaller size and lower body weights. The goal of the measure is to provide a framework where facilities can easily assess their doses, compare them to benchmarks, and take corrective action to lower their doses if they exceed threshold values.

VI. A hospital must report on all applicable measures and achieve Leapfrog’s standards on at least 40% (large teaching hospitals), 50% (pediatric hospitals, rural hospitals) or 60% (general hospitals, small teaching hospitals) of applicable measures.

The Leapfrog Hospital Survey uses 39 national performance measures to evaluate individual facilities in seven domains: inpatient care management, inpatient surgeries, pediatric care, medication safety, maternity care, outpatient procedures, and infections. The measures included on the Survey are predicated on the latest science and are selected with guidance from scientific advisors at the Armstrong Institute for Patient Safety as well as Leapfrog's volunteer Expert Panels. Hospitals are evaluated only on the services or procedures performed in their facility. Reporting on all measures and achieving Leapfrog’s standards on the measures demonstrates a strong commitment to transparency on safety and quality.

VII. Hospitals eligible for a Leapfrog Hospital Safety Grade must receive an A on the letter grades publicly reported at the time of the Top Hospital public announcement.

Note: Excludes hospitals that earned a C or worse in the most recent spring 2022 Safety Grade. This criterion does not apply to pediatric hospitals.

The Leapfrog Hospital Safety Grade assesses how safe hospitals are for patients. Each A, B, C, D or F score comes from expert analysis of publicly available data consumers can use to protect their families from harm or death during a hospital stay. Some hospitals are exempted from receiving a Safety Grade, including specialty hospitals like children’s or surgical, and Critical Access Hospitals. Hospitals that are eligible for a grade, which includes general acute-care hospitals, must receive an A on the current Leapfrog Hospital Safety Grade (fall 2021) in order to qualify for Top Hospitals.

VIII. Hospitals must satisfy the Top Hospital Selection Committee that in general the hospital embodies the highest standards of excellence worthy of the Leapfrog Top Hospital designation.

Note: This criterion does not apply to pediatric hospitals.

Hospitals that satisfy the quantitative criteria outlined above must also meet the Committee’s qualitative requirements for overall excellence, which includes a review of data from the Centers for Medicare & Medicaid Services (CMS) and other publicly available information pertaining to the hospital. Among those requirements: hospitals that perform worse than the national rate on CMS’ mortality measures for heart attack, heart failure, pneumonia, COPD, or CABG are excluded from receiving a Top Hospital award.

IX. A hospital must achieve Leapfrog’s standards for high-risk surgeries and procedures

(Hospitals must “achieve the standard” for more than 50% of the surgeries or procedures that apply)

Note: This criterion does not apply to pediatric hospitals.

Three decades of research have consistently demonstrated that patients that have their high-risk surgery at a hospital and by a surgeon that have more experience with the procedure have better outcomes, including lower mortality rates, lower complication rates, and a shorter length of stay than for patients who have their surgery done at a hospital or by a surgeon with less experience. Based on the research by Dartmouth-Hitchcock Medical Center, Michigan Medicine, and Johns Hopkins Medicine, as well as guidance from Leapfrog’s national expert panel, Leapfrog has identified eleven high-risk procedures for which there is a strong volume-outcome relationship. For more information, see: https://ratings.leapfroggroup.org/measure/hospital/complex-adult-and-pediatric-surgery.

Research indicates that a patient’s risk of dying is reduced by approximately two to four times, depending on the high-risk procedure, if care is obtained in hospitals that meet Leapfrog’s standards. Hospitals fulfilling Leapfrog’s standards for high-risk surgeries and procedures (carotid endarterectomy, mitral valve repair and replacement, open aortic procedures, lung resection for cancer, esophageal resection for cancer, pancreatic resection for cancer, rectal cancer surgery, bariatric surgery for weight loss, total hip replacement surgery, total knee replacement surgery, and Norwood procedure) have met Leapfrog’s robust volume, processes of care, and outcomes standards.

¹ Bates D, Teich J, Lee J, et al. The impact of computerized physician order entry on medication error prevention. JAMIA. 1999;6:313-321.
² Classen D, Pestotnik S, Evans R, Lloyd J, Burke J. Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277:301-306.
³ Pronovost PJ, Young T, Dorman T, Robinson K, Angus DC. Association between ICU physician staffing and outcomes: a systematic review. Crit Care Med. 1999; 27:A43.