Hospital death rates not the answer

Wednesday, November 21, 2012


Hospital death rates are not always a good measure of hospital performance, according to a new study in the Australian Health Review.

This review is reported in the most recent edition of the Australian Health Review, the peer reviewed journal of the Australian Healthcare and Hospitals Association.

“Increased reporting of hospitals’ performance is a popular measure, welcomed as a part of the current Government’s reform agenda by consumers. However, some of the indicators used to demonstrate hospitals’ performance may not provide an accurate measure of the safety or quality of health care within hospitals,” said study leader Dr Anna Barker, Senior Research Fellow at the Centre of Research Excellence, in Patient Safety based at Monash University.

“Our research has looked at the use of mortality rates for conditions with overall low mortality as an indicator of hospital performance. We found that there are a number of statistical limitations, challenges and biases inherent in these indicators and argue that they should be used with extreme caution.

“One of the major reasons for this is the difficulty in differentiating a normal variation in mortality rates from an abnormally high one. Statistically, the smaller the data set, or the rarer the event, the more difficult it is to determine if there is a genuine increase in the number of deaths when compared to other similar hospitals.

“There may also be problems with the quality of underlying administrative data which make it difficult to use this information to compare hospitals’ performance. The differences in coding between hospitals is likely to affect the quality of the data, along with difficulties involved with adjusting for hospital size and monitoring small hospitals. 

“When this data falsely identifies a hospital as under-performing, time spent by hospital staff responding to this is time that is unavailable for other valuable work that may have a more direct impact on the quality of patient care. This process can also cause considerable inefficiency and damage to morale.

“Improving the quality of mortality data collected could be solved through the increased use of clinical quality registries that are epidemiologically sound and clinically relevant.  However, there are better methods to detect a poorly performing hospital than using mortality data include: properly structured morbidity and mortality meetings, independent audits, evidence-based bundles and checklists and more sophisticated statistical analysis.

“To achieve maximum safety it is necessary, in addition to using these methods, to understand the characteristics of hospitals as complex systems that exhibit safe emergent behaviour, e.g. using the science of complex systems and its tools. Genuine safety cannot be achieved simply be studying ‘unsafety’.

“We believe that public reporting of timely accurate information by hospitals is essential for transparency and accountability. However, this information needs to provide consumers with useful indicators of hospitals’ performance. Mortality rates for conditions with overall low mortality should be used with caution, if at all,” Dr Barker said.

For more information and comment:  Dr Anna Barker, 0408 033 287