Health care quality measurements are widely used to assess and improve patient care. But a new Kaiser Permanente study, published in Population Health Management, suggests the data used by a national group to measure an individual hospital’s quality may lead to inaccurate results.
The research team started the study after Kaiser Permanente Northern California (KPNC) surgical quality and safety committees reviewed a routine report that showed their hospitals had a higher-than-average rate of patients who developed heart problems after surgery.
The study included 157,075 adults age 65 and over who had elective, urgent, or emergency surgery not due to a heart problem between January 2017 and December 2019 at 20 Kaiser Permanente Northern California (KPNC) hospitals. Of these, 137,722 (88%) had elective surgery and 19,353 had urgent surgery (within 48 hours) or emergency surgery (within 24 hours). Through an electronic medical record review, the researchers identified every patient who had a heart attack (blood clot blocking an artery to the heart), cardiac arrest (heart stoppage), or similar heart-related problem after surgery.
The analyses showed that after accounting for patient characteristics such as age, gender, smoking history, body-mass index, and chronic conditions, the risk of a post-surgical heart problem was about the same at each of the 20 hospitals. Next, the researchers averaged odds ratios for post-surgical heart risk from the 3 annual reports they had received from the National Surgical Quality Improvement Project (NSQIP) from 2017 to 2019. They found that the odds ratios the researchers calculated did not differ significantly from one hospital to the next. In addition, the calculated odds ratios differed significantly — and were lower than — what from NSQIP reported. The researchers said this suggests that risk adjustment performed by NSQIP may be inadequate.
The Department of Veterans Affairs (VA) developed NSQIP in 1994 to evaluate causes of surgical deaths in VA hospitals, in an effort to improve patient outcomes, and expanded it to other hospitals in the mid-2000s. NSQIP remains the only national system to evaluate and report a hospital’s surgical quality and the data it collects is used to identify opportunities for improvement, develop training programs, and set benchmarks for hospitals nationwide. More than 700 hospitals are currently participating in NSQIP. (Hospital participation is voluntary.) NSQIP reports on multiple aspects of hospital care. The study focused on heart attacks and other heart problems that develop after surgery because, although not common, they are a leading cause of illness and death after surgery.
- Limitations to Health Care Quality Measurement: Assessing Hospital Variation in Risk of Cardiac Events After Noncardiac Surgery. Population Health Management. 12 Sep 2022.