Thursday, 12 September 2013

Channel 4's 'UK/US hospital mortality' story is based on Jarman's sandy foundations

Last night I sat down to watch the Channel 4 news and was deeply upset by what was then to follow.  The 'news' exclusive on the 'increased mortality' in UK hospitals versus those in the US was presented as if the data to prove this theory was robust, there was no discussion of the huge flaws in the methods used and the panel discussion was completely one sided.  Channel 4 gave no neutral academics a chance to speak and they gave no one the chance to defend the NHS.  It was trial and execution by a biased one man band, it was shoddy journalism at its worst, it was very disappointing and very unlike Channel 4's normally excellent coverage.  I shall be factual and careful with what I shall say next, as it appears some cannot listen to criticism of their pet methodologies without resorting to threats of GMC referral, not the sign of a robust argument I would say.

The story claimed that patients was 45% more likely to die in the UK than the US, when admitted with acute illness such as pneumonia and septicaemia.  This was based on 'research' done by Professor Brian Jarman, he of Dr Foster fame and a big supporter of the HSMR tool (Hospital Standardised Mortality Ratio).  It must be noted that Dr Foster are rather close to the Department of Health, with the latter being obliged to promote the business interests of Dr Foster 'intelligence'.  It is worth reading about the rather cosy relationship involving Jarman, Dr Foster and the government, because it puts Jarman's potential motives into the open, conflicts of interest are often key in understanding such matters.

 Essentially the Channel 4 story was based upon several assumptions that look rather naive, flawed and ignorant to anyone that has a basic grasp of scientific evidence and statistics.  Firstly the UK mortality data is based upon HES (Hospital Episode Statistics) data, which is notoriously inaccurate and unreliable.  For example in the recent past HES data showed there were 17,000 pregnant men in the UK, a truly unbelievable statistic.  There is also abundant evidence showing that HSMRs themselves are very poor tools, even for comparing hospitals within the same country, let alone different continents.  HSMRs are crude tools and many academics feel their use should be abandoned entirely.

"Nonetheless, HSMRs continue to pose a grave public challenge to hospitals, whilst the unsatisfactory nature of the HSMR remains a largely unacknowledged and unchallenged private affair."

The above quote shows exactly what the problem is, here we have a flawed and scientifically dubious measure being used when it should not be used, as a political bandwagon and some vested interests are running out of control.  Brian Jarman's baby is the HSMR and he is a powerful man with a large sphere of influence.  It is notable that even the Keogh review was rather critical of the use of HSMRs and the way in which they had been inappropriately use to create divisive anti-NHS propaganda.  There are so many things that can change the HSMR, and many are absolutely nothing to do with the actual quality of care provided.

In simple terms if you put poor quality in, then you get dubious data out and will reach dodgy flawed conclusions.  Firstly HES data, on which the UK mortality rates are based, is poor and this means that mortality cannot be adequately adjusted for confounding factors.  The data is so inaccurate that information about illness type, severity of illness and co-morbidities cannot be adequately adjusted for in the statistical modelling.  The way data is coded differently in the US and UK is also likely to have a massive effect on the results.  Generally HES data is poor and patients are under-coded, ie their illness and co-morbidities are not coded to be as bad as they actually are.  However the exact opposite is true in US coding, they have a vast army of bureaucrats and have had a marketised system for decades, meaning that over-coding is rife, ie hospitals exaggerate the illness and co-morbidities of patients in order to increase their revenues.  There have also been huge issues with the accuracy of the US data.  There is also a huge volume of evidence showing that over-charging as a result of over-coding has been rife in the US, estimates put the cost of over-coding in the multi billion dollar ball park.

Overall this is a case study in dodgy data and dodgy journalism leading to what appears a hugely erroneous conclusion.  Brian Jarman has been reckless and irresponsible in my opinion, he has tried to justify his leaking of unpublished data but his squirming doesn't cut the mustard for me.  In honesty Brian Jarman simply doesn't know why the HSMRs are so different in the UK and US, interestingly the UK outperformed the US on fractured hip mortality, a fact sadly ignored by the news and not mentioned by Jarman in his interview.  This is also rather salient as hip fractures are one of the few things coded accurately in the UK, as it has been attached to a specific tariff for years, again yet more evidence that coding is responsible for the differences in HSMRs and not care quality.  Overall the likely explanation for the differences is the huge discrepancies in terms of coding practice between the US and the UK, in the US they tend to over-code and in the UK we tend to under-code. The huge effects of coding and admission practices on HSMRs have been well made in the BMJ in the past.  There is also the fact that many US patients are discharged from acute hospitals to die elsewhere, and Brian Jarman has admitted this hasn't been taken into account in his calculations.

Brian Jarman should not be leaping to such conclusions and he should be publishing in a reputable journal before recklessly leaking such propaganda to the media.  I wonder if Brian Jarman has done any work at all to investigate the differences between coding in the US and the UK, or did he simply say what he was nudged towards by the Department of Health?  I also wonder if Brian Jarman can claim his data is adequately corrected for confounders when it is based on unreliable and inaccurate HES data?  Personally speaking I have lost all respect for Brian Jarman and am now highly suspicious of the cosy nepotistic little relationship involving the government, Dr Foster and Brian Jarman.  It is just extremely sad that Victoria McDonald was played and used in such a game.  This whole episode has shown very clearly that one thing the NHS needs is good independent statistics and high quality data, and Dr Foster is definitely not going to be part of a progressive solution.


Anonymous said...
This comment has been removed by a blog administrator.
Dilettante Voice said...

Were there really 17, 000 "pregnant men" or were they really, as I suspect, male infants requiring obstetric care at or shortly after the time of birth?

Garth Marenghi said...
This comment has been removed by the author.
Anonymous said...

comment edited

"Jarman is a useful idiot. The DH hasn’t changed from the New Labour days. They use bogus think academics & sham think tanks to pedal the message that the NHS is bad. This guy is no different to Zack Cooper, the LSE academic who used comparative data (between London & provincial units) to support the ludicrous contention that competition improves quality.

People like @******** (a struck off doctor) will bang on about Jarman being eminent and engage in pseudointellectual critiques of the methodology. Its not worth wasting your breath. Nobody is interested in an academic debate. People remember the headlines, not the protracted correspondence in the letters pages of dusty old journals.

Jarmans’ latest is part of an orchestrated campaign to discredit the NHS. During the passage of the HSC Bill, the media turned a blind eye & deaf ear. In the last few weeks, I’ve noticed far more “NHS negligence” stories in the media. I’m talking about BBC radio news stories that would never have been broadcast a year or so back. Why?

Health correspondents merely cut and paste press DH spin doctor releases. It is far easier not to run stories critical of Coalition health policy because these correspondents will lose their access to DH press releases, which save them having to do old-fashioned journalism. The result is that the public only get one side of the story: that the NHS is bad and needs reform.

We are seeing this with the attack on GPs and consultants. The implication that consultants don’t work nights gets repeated until people believe it. This has nothing to do with new contracts to improve care, but everything to do with new contracts to deprofessionalise senior doctors, creating an army of shift-workers, with no SPA time and struck on low pay. FTs will then introduce short-term contracts and then they’ll outsource services. Current changes in TUPE mean that employment/pension protection will disappear.

Jarman is just part of the Coaltions' media strategy.