Channel 4's 'UK/US hospital mortality' story is based on Jarman's sandy foundations

Last night I sat down to watch the Channel 4 news and was deeply upset by what was then to follow.  The 'news' exclusive on the 'increased mortality' in UK hospitals versus those in the US was presented as if the data to prove this theory was robust, there was no discussion of the huge flaws in the methods used and the panel discussion was completely one sided.  Channel 4 gave no neutral academics a chance to speak and they gave no one the chance to defend the NHS.  It was trial and execution by a biased one man band, it was shoddy journalism at its worst, it was very disappointing and very unlike Channel 4's normally excellent coverage.  I shall be factual and careful with what I shall say next, as it appears some cannot listen to criticism of their pet methodologies without resorting to threats of GMC referral, not the sign of a robust argument I would say.

The story claimed that patients was 45% more likely to die in the UK than the US, when admitted with acute illness such as pneumonia and septicaemia.  This was based on 'research' done by Professor Brian Jarman, he of Dr Foster fame and a big supporter of the HSMR tool (Hospital Standardised Mortality Ratio).  It must be noted that Dr Foster are rather close to the Department of Health, with the latter being obliged to promote the business interests of Dr Foster 'intelligence'.  It is worth reading about the rather cosy relationship involving Jarman, Dr Foster and the government, because it puts Jarman's potential motives into the open, conflicts of interest are often key in understanding such matters.

 Essentially the Channel 4 story was based upon several assumptions that look rather naive, flawed and ignorant to anyone that has a basic grasp of scientific evidence and statistics.  Firstly the UK mortality data is based upon HES (Hospital Episode Statistics) data, which is notoriously inaccurate and unreliable.  For example in the recent past HES data showed there were 17,000 pregnant men in the UK, a truly unbelievable statistic.  There is also abundant evidence showing that HSMRs themselves are very poor tools, even for comparing hospitals within the same country, let alone different continents.  HSMRs are crude tools and many academics feel their use should be abandoned entirely.

"Nonetheless, HSMRs continue to pose a grave public challenge to hospitals, whilst the unsatisfactory nature of the HSMR remains a largely unacknowledged and unchallenged private affair."

The above quote shows exactly what the problem is, here we have a flawed and scientifically dubious measure being used when it should not be used, as a political bandwagon and some vested interests are running out of control.  Brian Jarman's baby is the HSMR and he is a powerful man with a large sphere of influence.  It is notable that even the Keogh review was rather critical of the use of HSMRs and the way in which they had been inappropriately use to create divisive anti-NHS propaganda.  There are so many things that can change the HSMR, and many are absolutely nothing to do with the actual quality of care provided.

In simple terms if you put poor quality in, then you get dubious data out and will reach dodgy flawed conclusions.  Firstly HES data, on which the UK mortality rates are based, is poor and this means that mortality cannot be adequately adjusted for confounding factors.  The data is so inaccurate that information about illness type, severity of illness and co-morbidities cannot be adequately adjusted for in the statistical modelling.  The way data is coded differently in the US and UK is also likely to have a massive effect on the results.  Generally HES data is poor and patients are under-coded, ie their illness and co-morbidities are not coded to be as bad as they actually are.  However the exact opposite is true in US coding, they have a vast army of bureaucrats and have had a marketised system for decades, meaning that over-coding is rife, ie hospitals exaggerate the illness and co-morbidities of patients in order to increase their revenues.  There have also been huge issues with the accuracy of the US data.  There is also a huge volume of evidence showing that over-charging as a result of over-coding has been rife in the US, estimates put the cost of over-coding in the multi billion dollar ball park.

Overall this is a case study in dodgy data and dodgy journalism leading to what appears a hugely erroneous conclusion.  Brian Jarman has been reckless and irresponsible in my opinion, he has tried to justify his leaking of unpublished data but his squirming doesn't cut the mustard for me.  In honesty Brian Jarman simply doesn't know why the HSMRs are so different in the UK and US, interestingly the UK outperformed the US on fractured hip mortality, a fact sadly ignored by the news and not mentioned by Jarman in his interview.  This is also rather salient as hip fractures are one of the few things coded accurately in the UK, as it has been attached to a specific tariff for years, again yet more evidence that coding is responsible for the differences in HSMRs and not care quality.  Overall the likely explanation for the differences is the huge discrepancies in terms of coding practice between the US and the UK, in the US they tend to over-code and in the UK we tend to under-code. The huge effects of coding and admission practices on HSMRs have been well made in the BMJ in the past.  There is also the fact that many US patients are discharged from acute hospitals to die elsewhere, and Brian Jarman has admitted this hasn't been taken into account in his calculations.

Brian Jarman should not be leaping to such conclusions and he should be publishing in a reputable journal before recklessly leaking such propaganda to the media.  I wonder if Brian Jarman has done any work at all to investigate the differences between coding in the US and the UK, or did he simply say what he was nudged towards by the Department of Health?  I also wonder if Brian Jarman can claim his data is adequately corrected for confounders when it is based on unreliable and inaccurate HES data?  Personally speaking I have lost all respect for Brian Jarman and am now highly suspicious of the cosy nepotistic little relationship involving the government, Dr Foster and Brian Jarman.  It is just extremely sad that Victoria McDonald was played and used in such a game.  This whole episode has shown very clearly that one thing the NHS needs is good independent statistics and high quality data, and Dr Foster is definitely not going to be part of a progressive solution.