Most (Flu Vaccination) Studies in Prestigious Medical Journals Have Conclusions That Don't Hold Up (BMJ)

You are a Doctor. You rely on your medical journal to guide you on what line to take on many important matters, for example, whether to tell your patients that it's good for them to have an annual flu jab. Your medical journal states that such and such a study proves the case. So you go with that, and repeat it to your patients, if asked. After reading the following BMJ article maybe you'll want to think deeply about the (biased) "evidence" you have probably unwittingly relied on in the past, and what steps you and your profession should in future take to avoid the present undue influence from your ultimate paymasters, the pharmaceutical industry?

----------------

Given the crucial role that prestigious medical journals play in educating doctors and scientists across the globe another research study - published in the BMJ - looking at 274 comparative studies (on flu vaccination) - found that only 18% of the studies were deemed to actually prove what the articles claimed to be their findings.

That means that over 80% of them were judged not to have "made the case" outlined in their conclusions.

"70% of the studies reported conclusions favourable to the vaccines, but only 18% showed complete concordance between data reported and study conclusions. Over half (56%) of studies were at high risk of bias, with only 4% being at low risk".

With an estimated 7,287 items relevant to primary care published in these prestigious Medical Journals every month (studies, letters, and editorials) and the fact that few healthcare workers have the time - or the skills - to carry out in depth critical appraisals of any of these published articles (1), most doctors in the field rely on a brief (two minute) scan of material (2) to bring them up-to-date with 'best practice' and how to treat their patients (3). 

In most cases this means browsing the article's 'abstract' or its 'conclusions'.

And generally, the more prestigious the journal, the more trustworthy the findings have always been assumed to be. But given the proven unreliability of these published studies and in previous reviews (4) this is a major cause for concern for all healthcare workers.

Perhaps the most worrying effects of these often re-hashed published Research Studies in these prestigious journals is the fact they are the ones quoted and re-quoted in other scientific journals and at conference after conference, to "prove" the effectiveness of the medications, or, as in this case the treatments (flu vaccinations), being studied.

The authors of the study put it succinctly by stating (in relation to flu vaccination): "Studies partly or completely sponsored by industry, however, were published in more prestigious journals and are probably cited more, although their methodological quality and size were similar. Some of these findings might help to explain the continuation of a near global policy, despite growing doubts as to its scientific basis".

One of the key findings was that what decides whether scientific papers made it into prestigious medical journals wasn't related to the quality of the study, but more to who paid for it - and those papers funded by major companies, despite the methodology or conclusions, were more likely to be published.

Tom Jefferson, lead author said, "The study shows that one of the levers for accessing prestigious journals is the financial size of your sponsor. Pharmaceutical sponsors order many reprints of studies supporting their products, often with in-house translations into many languages. They will also purchase publicity space on the journal. Many publishers openly advertise these services on their website."

References:

1.       Alper BS, Hand JA, Elliott SG, Kinkade S, Hauan MJ, Onion DK, et al. How much effort is needed to keep up with the literature relevant for primary care? J Med Libr Assoc 2004;92:429-37.[ISI][Medline]

2.       Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319:358-61.[Abstract/Free Full Text]

3.       Saint S, Christakis DA, Saha S, Elmore JG, Welsh DE, Baker P, et al. Journal reading habits of internists. J Gen Intern Med 2000;15:881-4.[CrossRef][ISI][Medline]

4.        Yank V, Rennie D, Bero LA. Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ 2007;335:1202-[Abstract/Free Full Text]

BMJ 2009;338:b354

http://www.bmj.com/cgi/content/abstract/338/feb12_2/b354