MTAS scandal - Patterson's smoking gun

Below is the evidence that shows MTAS was used a research project for Professor Fiona Patterson to help validate her ideas for selection of doctors into higher training. This is a scandal.

The lives of thousands of junior doctors have been turned upside down by an inadequately researched and unvalidated selection system. I am shocked that the Royal Colleges let this happen. In fact there is evidence that the colleges “assumed” it would work form this document form the RCPCH here.

“Though not entirely reassured by this we, like most Colleges, assumed the process would nevertheless run better than has since transpired”

This is bordering on negligence. I cannot believe that the Royal Colleges could allow this to happen and just assume that it would turn out ok. We are talking about the lives and careers of thousands of doctors and they just let it happen without a serious fight and without the need to consult their members and fellows!

Please see these notes from the RCS Regional Representatives Meeting in June 2006. I refer you to pages 4 and 5 (but there is also some interesting stuff later in the document about cutting training numbers and DGH reconfigurations).

This document provides the evidence that the basis for MTAS was incompletely researched and unvalidated particularly for surgical training. It also confirms that this years MTAS was going to be used to validate the methods! Here are the crucial paragraphs:

"The background was to design a selection system that was defensible, reliable, valid, cost-effective and feasible as well as to develop and validate current criteria"

How are her selection methods valid and what is the evidence? Nothing solid on Pubmed.

"In terms of progress, the stakeholder consultations (stage 1) and job analysis and literature review (stage 2) have taken place. The current phase is the third which is the design of selection tools and scoring frames. The other stages are;
• Conduct pilots and evaluate outcomes
• Train selectors
• Implementation"

This meeting was held in June last year. MTAS opened in January !
No publications of her work on this have been seen in peer reviewed journals.

"A strong recommendation from the pilots was that it is extremely important to get standardisation of assessors/selectors"
But what is the evidence that a rushed half day training session achieves standardisation of assessors/selectors? Again, Pubmed no help.

"Although a literature review of 25 years worth of research was relevant, the challenge in designing a model for surgery was that none of this research had been validated for surgery and currently in the process of doing that"
She admits the lack of evidence herself.

"Early implementation is expected to take place next spring with the intention to do early validation work"
The smoking gun! She admits that the validation work will be done during in the Spring of 2007 i.e the actual MTAS process itself! This is scandalous.

"The next steps would be the piloting of the selection tools and validating them as that is the missing link"

"There are a number of challenges to the project including timeframes and engaging stakeholders, pitching assessment at the right level, calibrating of assessment and selectors as well as feasibility in the short term. Along with the challenges are opportunities"

Too right! Of course - the opportunities for Professor Patterson to churn out lots of research are incredible. The amount of data that she has amassed from 34,000 doctors is phenomenal.

It is clear that MTAS has been used as a research project for the future selection of doctors into specialist training. This is an absolute scandal. The Royal Colleges have allowed 34,000 junior doctors be guinea pigs for a baseless and unvalidated selection process.

We have ended up with a complete crisis because the colleges “assumed” it would be all ok. This has coincided with situation where 33,000 doctors are applying for 18,500 training posts, which is clearly a situation when you want the most robust selection method possible in order to ensure the best got the best chance of getting a training post. What we ended up with was a bloody pie in the sky research project with 34,500 guinea pig doctors. This is an absolute scandal. When are the Colleges, Deaneries, CMOs and DH going to put their hands up and admit that this was a terrible mistake and they are all to blame?

Don’t hold your breath….