Read all about it!

I've mentioned the "Behind the Headlines" service before. It replaced the "Hitting the Headlines" service that the National Library for Health used to produce. Although still available through the National Library for Health it's actually produced as part of NHS Choices - the patient information site.

As with Hitting the Headlines this takes stories that are in the news looks at the science behind it. It presents the information in a really useful way. Take a recent example - reports in various papers that chlorinated water causes birth defects. Scary stuff - but it is it true?

The report on this topic is broken down into sections. There's a brief explanation an summary of what the news stories were about, what the stories were based on, and a brief appraisal of that original source. This section alone is useful, but there is more.

There are details of who wrote the paper, the researchers' affiliations, and the publication it appeared in. There is a longer section looking at the study and explaining what sort of study it was an how it was carried out. This is rounded off with an explanation of the researchers' findings and conclusions.

The next section is what the National Knowledge Service thinks about the paper - basically a critical appraisal of the original research. It looks at how the study was carried out, considers the statistical significance of figures in the study, and highlights gaps in the information presented.

The whole piece concludes with links to the original newspaper stories and also to the actual paper that sparked off the whole debate.

I think this is an excellent resource. It's a useful example of how we should look at evidence and appraise it, but also useful for sitting down with patients and explaining to them why they don't need to worry about the latest doom and gloom health headline.

In fact, I like these so much I have a nice RSS feed to bring Behind the Headlines to my Google Reader and from there I often flip them into the current awareness service.

image (c) creative commons attributed

Presumption of innocence

I once heard a joke about a sheriff who stopped a woman in a car and cautioned her for illegally fishing, because her husband had left his fishing gear in the boot of the car and she was near a river. She was adamant she wasn't intending to fish, but the sheriff replied - "you've got the tackle - how do I know you're not going to use it?" In that case, parried the women, I'm accusing you of rape. The sheriff is gobsmacked - rape?! But he hasn't even touched her!! "I know," replies the women, "but you've got all the tackle - how do I know you're not going to use it?"

In English law we have the concept of the presumption of innocence - more commonly known by the phrase "innocent until proven guilty". The idea is that it's much harder to prove innocence than to prove guilt. If you are guilty the prosecution should be able to find witnesses to your crime, traces of your DNA or fingerprints at the crime scene. If you were home alone last night, no witnesses, no alibi, can you positively prove beyond doubt you were there and not up to no good elsewhere?

Sometimes guilt is easily proved. You were seen by witnesses, you left fingerprints, there are recorded phone conversations or CCTV images. Similarly cause and effect can be easy to demonstrate. Yesterday I walked into the open drawer of my desk (ouch) and today I have a big bruise on my shin. Cause and effect.

It's a little harder with health issues. It took years to link smoking and lung cancer. But it's not possible to say that if you smoke you will die of lung cancer - or that if you don't smoke you wont. Roy Castle, who never smoked, died from lung cancer. Conversely we all know elderly relatives who smoked 40 a day for 60 years or more and died peacefully in their beds of old age.

At the moment scientists keep looking at mobile phone use. So far there is no positive link between phone use and cancer - but is that because there is no link to find? Or because we haven't yet proved the link? Or haven't waited long enough for the cancers to start appearing?

What about aluminium and breast cancer? Various eminent scientists have said that there is no link, although they admit that more research needs to be done. Just because the police haven't yet had time to investigate a crime does it mean that crime hasn't been committed?

The debate that still rumbles on is MMR. MMR stands accused of causing autism. There is plenty of evidence - eyewitness accounts from hundreds of parents who had happy, normal children who were changed into troublesome and troubled children about the time of their MMR vaccine. The judge says that this evidence is circumstantial. Certainly MMR was hanging around looking suspicious - perhaps wearing a hoody and smoking - but that isn't proof that it committed the crime.

Since then their have been endless papers trying to prove MMR's innocence beyond doubt. Today MMR is in the papers again with yet another study failing to prove a link between MMR and autism. Is there no link? Or are scientists asking the wrong questions, looking at the wrong children, or the wrong numbers? (The Guardian calls this latest study "huge" but it only covers 250 children).

The key issues here are risk and evidence.

What is the actual risk of a child developing autism? Of an unvaccinated child catching measles? Of a child dying from measles? How can we understand relative risks? Read Reckoning with Risk, Risk by Lupton and Risk Matters in Healthcare.

And evidence - what is evidence? Are all health interventions guilty until proven innocent? Is it wise to assume every new idea will kill us until we can positively prove otherwise? Is evidence only evidence when it tells us what we want to hear? What value does the experience of ordinary people have in the hierarchy of evidence? If one scientist publishes a paper that says magnets don't relieve arthritis pain, but I can find 50 friends who say they do - who is right? What is good evidence? There are many books in the library on evidence, statistics, reading and understanding papers.

What can any of us do? Be vigilant. Be critical of the evidence. Use our brains.

(c) creative commons attributed

The proof of the pudding

The Lancet today reports that a "collaborative reanalysis" of epidemiological studies "shows beyond doubt that ovarian cancer can be prevented by the long term use of different generations of oral contraceptives." (Full text free with your Athens password!) In simple terms this means that if you take an oral contraceptive pill you are less likely to get ovarian cancer.

The point that interests me here is the way in which this conclusion was reached - not by doing any trials or studies, but by reading existing studies. The authors looked at epidemiological studies that included over 100 women. They defined what was meant by "ovarian cancer" and "oral contraceptive use". They looked at the figures they had and used them to come to wider conclusions than could be reached from any one study alone. Their findings are expressed using confidence intervals and there is an awful lot of statistics in the paper.

In health research there are levels of evidence. These range from (at the bottom) "expert opinion" to "systematic reviews". A systematic review is the gold standard of evidence. Like the Lancet paper it gathers together a number of studies - Randomised Controlled Trials - selects those that fall within defined limits, and using a "meta analysis" of the data attempts to draw wider conclusions than can be drawn from one trial alone.

If this is confusing, don't despair. There is plenty of help available, starting with a veritable mountain of books. One of the best for a straightforward accessible read, covering all the types of evidence and how to look for them and assess them, is Trisha Greenhalgh's How to Read a Paper, now in its third edition.

There are very specific books: Chalmer's Systematic Reviews, Systematic Reviews in Healthcare by Glasziou, and Systematic Reviews to Support Evidence-Based Medicine from the Royal Society of Medicine Press.

There are general books on understanding research: Studying a Study and Testing a Test, Reading Research, Bandolier's Little Book of Making Sense of the Medical Evidence.

There are endless titles on reading and understanding statistics, and evidence-based practice. Your local NHS library is, of course, the best place to start looking.

Online, CASP has tools to help you appraise a systematic review and randomised controlled trials. The Centre for Evidence-Based Medicine has tools to help you understand the maths and the jargon.

If you want to find systematic reviews the place to look is the Cochrane Library.

As to whether the lowered risk of ovarian cancer outweighs the increased risk of breast, cervical or thyroid cancer...in this post, at least, I'm not even going there.

(c) creative commons attributed

Eliminating the impossible

Counterknowledge. No - not a new word for arithmetic, or the things you need to know to be a successful shop assistant. Counterknowledge, according to Damian Thompson, is "misinformation packaged to look like fact." Apparently we are a gullible bunch and can be persuaded that all sorts of things are true, and it can be blamed on muddled gathering of evidence.

Thompson tells us that counterknowledge is behind conspiracy theories from Dan Brown to Diana, but also exists in health. Everything from belief in the usefulness of fad diets to a fear that MMR causes autism might be defined as counterknowledge.

Half truths work best when based partly on truth - there has to be a grain of credibility to start with. The best urban legends are always stories that happened to someone that you have some connection with. The story teller will also assure you it must be true because it happened to their next door neighbour's best friend's cousin. Even hoax emails tend to be prefaced with "my friend at BigCorporationBank sent this to me" or are apparently originated by a bigwig at Microsoft, Hotmail or elsewhere. We feel better disposed to trust something that comes from a source that is known to us.

Sometimes we believe because we want to hope, especially when we are sick. We want to believe that Aloe Vera, omega 3, coenzymes and carrot juice cure cancer if the alternative is to believe that we are going to die.

Health professionals are not immune to being hoaxed. We must all think as we read and question what we hear. I've mentioned CASP before, which helps you ask questions around evidence you find. The other important thing is to take care where you collect your evidence from. The National Library for Health, Intute, PatientUK, reputable charities are all a better start than the open web for good quality information - clean, clear knowledge. Oh - and did I mention that libraries can look for quality information for you and your patients?

(c) creative commons attributed

Well I never!

The media have been buzzing this week with news of a study published in the Lancet looking at HRT and cancer. The Scotsman read the article. It was, they said, a “shocking” report showing women on HRT were at “substantially increased risk” of cancer. The figure they quote is a 20% risk increase. Shocking indeed.

A doctor writing in the Telegraph was so shocked that she believes HRT should be banned. It’s a lifestyle drug, she says used by “feisty modern women” hoping to hang on to their sex lives and she wouldn’t recommended it at all, so dangerous is it.

The BBC also read about a 20% increase. They kindly explained that “This translates to one extra case of ovarian cancer for every 2,500 women taking HRT and one extra death from ovarian cancer in every 3,300 users.”

Reuters, it seemed, had read a different report. The study they read was one “suggesting – but not yet proving – that HRT causes breast and ovarian cancer”.

A professor writing in the Telegraph, alongside the sceptical doctor, says risks and benefits should be balanced. The risk is small and some women are “severely disabled” by menopause symptoms.

The headline in the Lancet itself says that 1000 deaths amongst the study cohort since 1991 can be put down to HRT. Just to put that into perspective that works out as one extra death a week over the 16 years of the study.

Is it worth the risk? The BBC tells us that the symptoms of menopause may include aches and pains, insomnia, hot flushes, dry skin, tiredness, bladder problems, depression, panic attacks, poor memory. Oh and there’s osteoporosis too.

So – what’s the moral of the story? Always read with an open mind and never be dazzled by statistics. Look at resources such as CASP to help you read papers critically, and read books like “How to Read Paper” or the recent Guardian Bad Science article on manipulation of statistics.

You can also look at the Hitting the Headlines facility on the National Library for Health. This regular looks at the stories in the headlines and looks at the facts behind them.

Read with your brain fully engaged, and never believe everything you read in the papers!

Bad Science?

The Guardian newspaper has a regular column on “bad science”. On 24th February that column looked at a review in a medical journal of a book on autism aimed at lay people.

Why was this “bad science”? Well, the article argued that the book’s author is an “activist” who has been “highly criticised” and who has his own agenda that is not much supported by the body of medical knowledge and opinion. The review of the book, however, was “flattering” and made no mention of the author’s history.

Quite rightly the Guardian doesn’t propose censorship. After all – where would we be now in terms of scientific knowledge if it wasn’t for those over the centuries who have dissented from mainstream opinion? The Guardian argues that some background on the author would have been helpful in enabling people to make a thoughtful and balanced reading.

A journal perhaps has some duty of care towards its readers in offering some sort of background information. But it is up to each and every one of us to read with our brains switched on, to read thoughtfully, carefully and critically.

There are various resources to help you do this. CASP (Critical Appraisal Skills Programme) has tools for appraising different types of health research. For appraising websites there are various things to look out for including the HoN (Health on the Net) logo. It’s also worth checking dates showing when material was last updated and any “about” page that might show who the author is and what particular bias they might bring to information on the site. On a lighter there is a brief animated checklist, the Quality Information Checklist . It is aimed at children but covers the basics.

The other issue that this Guardian piece raises is that of newspapers picking up on stories and giving them sensationalist, and possibly misleading, headlines. It’s best always to read beyond the headline. Most health news in papers can be tracked down to a recently published scientific paper. Don’t accept the interpretation of the Sun, the Mail – or even the Times or the Telegraph. Go straight to the source and make you own mind up. Another useful source for assessing news stories is the Hitting the Headlines section on the National Library for Health website. It looks at recent health news stories and assesses the evidence behind the headline.

For further reading on all of this, why not drop into the library? We have Trisha Greenhalgh's very readable work on the subject, How to Read a Paper, now in its third edition, and many others. We can also offer more advice and guidance. All free, as ever.