Due to the numerous public hospital quality rating systems available, offering conflicting results, and that there is no gold standard for how a rating system should be constructed or perform and no objective way to compare the rating systems, a group of medical experts evaluated the strengths and weaknesses of four major public hospital quality rating systems (CMS Hospital Compare, Healthgrades Top Hospitals, U.S. News & World Report Best Hospitals, Leapfrog Hospital Safety Grade and Leapfrog Top Hospitals) based on their experience as physician scientists with methodological expertise in health care quality measurement. This evaluation was published in NEJM Catalyst.
No rating system received an A or an F. The highest grade received was a B by U.S. News & World Report. The Centers for Medicare and Medicaid Services’ (CMS) Star Ratings received a C. The lowest grades were for Leapfrog, C-, and Healthgrades, D+. Each rating system had unique weaknesses that led to potential misclassification of hospital performance, ranging from inclusion of flawed measures, use of proprietary data that are not validated, and methodological decisions.
The analysis found there were several issues that limited all rating systems examined: limited data and measures, lack of robust data audits, composite measure development, measuring diverse hospital types together, and lack of formal peer review of their methods. In this Rating the Raters initiative, the authors found that the current hospital quality rating systems should be used cautiously as they likely often misclassify hospital performance and mislead. These results can offer guidance to stakeholders attempting to select a rating system for identifying top-performing hospitals.
- Rating the Raters: An Evaluation of Publicly Reported Hospital Quality Rating Systems. NEJM Catalyst. August 14 2019