Peer review

Today's post is interactive. Someone came in today with this very question, and it seemed to be one of wider interest.

When writing an essay you need to ensure your cited references are (a) recent and (b) from peer reviewed journals.

The up to date bit is easy enough. Any paper that you photocopy or print will have a date on it somewhere.

But what about the peer reviewed bit of it? How do you know if something is peer reviewed? Come to that - what IS peer review? It always makes me think of an elderly, dusty old party squinting myopically at a pile of papers...

Peer review, also known as refereeing, basically means asking people in the same field of expertise to have a look at any paper submitted for publication. These reviewers or referees check the paper for problems or weaknesses and may make suggestions for improvement.

The problem with this is that it's time consuming. If you've discovered something new and important it must be frustrating to have the paper announcing your findings held up for months while various people look at it. Some journals will actually show the data a paper was submitted, and the date it was accepted for publication (both dates can be some time before publication).

Sense About Science has a nice guide to peer review, written for the layperson in ordinary language. Peer review is supposed to improve the quality of articles, but doesn't guarantee that papers are free from errors. Not everyone thinks peer review is a good idea. And what about evidence that peer review works? Because asking if a paper is peer reviewed is just a part of the process of assessing the quality of evidence.

How do you know which journals are peer reviewed? When I was asked this today my first reaction was that you don't. Certainly an individual paper wont stat on it that it's peer reviewed. For that information you need the paper journals (remember those?) or the journal's website. Somewhere under information for authors, perhaps, or in a statement about the journal's purpose and readership) it will state that the journal is peer reviewed, that papers are refereed or give instructions for referees.

My second reaction was that surely there is a list somewhere. But there isn't. Now, wouldn't that be a nice project for someone...

(c) image copyright creative commons

One in a million

More on relative risk and quality of information. Phil Bradley's blog alerted me to an item that looks at the incidence of inaccurate data on breast cancer web sites. Apparently 5% of sites looked at contained inaccuracies. The conclusion is that patients should be sceptical of things they read online.

I'd agree that a spot of scepticism is always useful, but it seems to me that the result of this study is that a whopping 95% of websites looked at had no inaccuracies. I find that really heartening - it restores my faith in the quality of online information. Having said that I am, of course, suffering from a common problem here. I haven't read the paper in Cancer that reports the study. I haven't even read the abstract. I've just read the Reuters report on the paper. Did the original paper say how many inaccuracies there were on each of the offending sites? Or how big those inaccuaries were? Or how important? How did they select the sites they looked at? The Reuters report doesn't say.

On a similar theme I've today posted a link on the current awareness page to a paper in JNCI looking at women's perception of risk of recurrence of Ductal Carcinoma In Situ. Women with DCIS are understandably anxious and that leads them to believe that the chances of it recurring are far higher than is actually the case. It's just as well that there is no link between having an anxious - or any other type - of personality, and cancer.

For accurate statistical data on breast and other cancers, Cancer Research UK is a good starting point.

(c) creative commons attributed

Please be advised that your evening repast may be found situated within the family pet

This blog has been rather quiet of late. I've been away on leave, and although I've had access to the internet I had other things to do and didn't post a single thing.

Since I've been back I've been reading patient leaflets. I proof read them. A lot of people who write leaflets apparently don't have a spell check on their PCs. I check the evidence behind any statements made. I check that the material hasn't been plagiarised (i.e. blatantly copied from elsewhere without permission or proper attribution). I ask tricky questions about the copyright of the various pictures, diagrams and other embellishments that appear. And I despair over the quality of English used.

Why is it that although we manage to use simple words and sentence structures face to face, we get hit by some horrible wordiness every time we put finger to keyboard? If a patient asks where the canteen is you say, "oh - the canteen's on the second floor. Follow the signs to Cafe Blue." If we want to put a paragraph in a leaflet about the canteen we write, "canteen facilities may be found situated upon the second floor." If a patient rings up and asks about bringing their life savings and all their jewellery with them when they are admitted we say, "it's best to leave all your valuables at home. I'm afraid we don't have anywhere really safe to keep them here". If we write that down it transmutes into something awful on the lines of "Please be advised that it is not recommended to bring valuables..." (and I can't even face typing the rest of it.)

For those of you who are Essex Rivers staff there are guidelines on the intranet about patient leaflets. There are plenty of guides on writing patient information. There are plenty of guides on writing simple and straightforward English. My simple tip would be to read - out loud - everything you write. And then see if you can actually imagine saying that to a patient or anyone.

I sometimes feel we have an obsession with writing our own leaflets. Why reinvent the wheel? (Or - in NHS speak - "Do once and share.") Find someone else's evidence-based, up-to-date, well-written leaflet. Then you have two options. Contact them. Ask them to allow you to use their text. Get their written permission. Then make the leaflet your own - add your own logo, your own phone number and so on. Please, please do not try to "improve" the leaflet by adding extra words that really aren't needed.

Option two? Well, why not be really radical and not write a leaflet at all? That's right. Don't write one. Buy in copies of the perfect leaflet you found elsewhere, show your patients how to find it on the internet, or give them a phone number to call for their own copy.

You'll find good quality information for patients in the National Library for Health, at Patient UK and through NHS Choices. For cancer information the major cancer charities produce excellent information and the Great Ormond Street factsheets are useful for childhood illnesses. The American National Institutes for Health are also a good source of straightforward patient information, but these may need to be tweaked to have UK versions of drug names and so on added.

(c) creative commons attributed

Presumption of innocence

I once heard a joke about a sheriff who stopped a woman in a car and cautioned her for illegally fishing, because her husband had left his fishing gear in the boot of the car and she was near a river. She was adamant she wasn't intending to fish, but the sheriff replied - "you've got the tackle - how do I know you're not going to use it?" In that case, parried the women, I'm accusing you of rape. The sheriff is gobsmacked - rape?! But he hasn't even touched her!! "I know," replies the women, "but you've got all the tackle - how do I know you're not going to use it?"

In English law we have the concept of the presumption of innocence - more commonly known by the phrase "innocent until proven guilty". The idea is that it's much harder to prove innocence than to prove guilt. If you are guilty the prosecution should be able to find witnesses to your crime, traces of your DNA or fingerprints at the crime scene. If you were home alone last night, no witnesses, no alibi, can you positively prove beyond doubt you were there and not up to no good elsewhere?

Sometimes guilt is easily proved. You were seen by witnesses, you left fingerprints, there are recorded phone conversations or CCTV images. Similarly cause and effect can be easy to demonstrate. Yesterday I walked into the open drawer of my desk (ouch) and today I have a big bruise on my shin. Cause and effect.

It's a little harder with health issues. It took years to link smoking and lung cancer. But it's not possible to say that if you smoke you will die of lung cancer - or that if you don't smoke you wont. Roy Castle, who never smoked, died from lung cancer. Conversely we all know elderly relatives who smoked 40 a day for 60 years or more and died peacefully in their beds of old age.

At the moment scientists keep looking at mobile phone use. So far there is no positive link between phone use and cancer - but is that because there is no link to find? Or because we haven't yet proved the link? Or haven't waited long enough for the cancers to start appearing?

What about aluminium and breast cancer? Various eminent scientists have said that there is no link, although they admit that more research needs to be done. Just because the police haven't yet had time to investigate a crime does it mean that crime hasn't been committed?

The debate that still rumbles on is MMR. MMR stands accused of causing autism. There is plenty of evidence - eyewitness accounts from hundreds of parents who had happy, normal children who were changed into troublesome and troubled children about the time of their MMR vaccine. The judge says that this evidence is circumstantial. Certainly MMR was hanging around looking suspicious - perhaps wearing a hoody and smoking - but that isn't proof that it committed the crime.

Since then their have been endless papers trying to prove MMR's innocence beyond doubt. Today MMR is in the papers again with yet another study failing to prove a link between MMR and autism. Is there no link? Or are scientists asking the wrong questions, looking at the wrong children, or the wrong numbers? (The Guardian calls this latest study "huge" but it only covers 250 children).

The key issues here are risk and evidence.

What is the actual risk of a child developing autism? Of an unvaccinated child catching measles? Of a child dying from measles? How can we understand relative risks? Read Reckoning with Risk, Risk by Lupton and Risk Matters in Healthcare.

And evidence - what is evidence? Are all health interventions guilty until proven innocent? Is it wise to assume every new idea will kill us until we can positively prove otherwise? Is evidence only evidence when it tells us what we want to hear? What value does the experience of ordinary people have in the hierarchy of evidence? If one scientist publishes a paper that says magnets don't relieve arthritis pain, but I can find 50 friends who say they do - who is right? What is good evidence? There are many books in the library on evidence, statistics, reading and understanding papers.

What can any of us do? Be vigilant. Be critical of the evidence. Use our brains.

(c) creative commons attributed

The proof of the pudding

The Lancet today reports that a "collaborative reanalysis" of epidemiological studies "shows beyond doubt that ovarian cancer can be prevented by the long term use of different generations of oral contraceptives." (Full text free with your Athens password!) In simple terms this means that if you take an oral contraceptive pill you are less likely to get ovarian cancer.

The point that interests me here is the way in which this conclusion was reached - not by doing any trials or studies, but by reading existing studies. The authors looked at epidemiological studies that included over 100 women. They defined what was meant by "ovarian cancer" and "oral contraceptive use". They looked at the figures they had and used them to come to wider conclusions than could be reached from any one study alone. Their findings are expressed using confidence intervals and there is an awful lot of statistics in the paper.

In health research there are levels of evidence. These range from (at the bottom) "expert opinion" to "systematic reviews". A systematic review is the gold standard of evidence. Like the Lancet paper it gathers together a number of studies - Randomised Controlled Trials - selects those that fall within defined limits, and using a "meta analysis" of the data attempts to draw wider conclusions than can be drawn from one trial alone.

If this is confusing, don't despair. There is plenty of help available, starting with a veritable mountain of books. One of the best for a straightforward accessible read, covering all the types of evidence and how to look for them and assess them, is Trisha Greenhalgh's How to Read a Paper, now in its third edition.

There are very specific books: Chalmer's Systematic Reviews, Systematic Reviews in Healthcare by Glasziou, and Systematic Reviews to Support Evidence-Based Medicine from the Royal Society of Medicine Press.

There are general books on understanding research: Studying a Study and Testing a Test, Reading Research, Bandolier's Little Book of Making Sense of the Medical Evidence.

There are endless titles on reading and understanding statistics, and evidence-based practice. Your local NHS library is, of course, the best place to start looking.

Online, CASP has tools to help you appraise a systematic review and randomised controlled trials. The Centre for Evidence-Based Medicine has tools to help you understand the maths and the jargon.

If you want to find systematic reviews the place to look is the Cochrane Library.

As to whether the lowered risk of ovarian cancer outweighs the increased risk of breast, cervical or thyroid cancer...in this post, at least, I'm not even going there.

(c) creative commons attributed

Eliminating the impossible

Counterknowledge. No - not a new word for arithmetic, or the things you need to know to be a successful shop assistant. Counterknowledge, according to Damian Thompson, is "misinformation packaged to look like fact." Apparently we are a gullible bunch and can be persuaded that all sorts of things are true, and it can be blamed on muddled gathering of evidence.

Thompson tells us that counterknowledge is behind conspiracy theories from Dan Brown to Diana, but also exists in health. Everything from belief in the usefulness of fad diets to a fear that MMR causes autism might be defined as counterknowledge.

Half truths work best when based partly on truth - there has to be a grain of credibility to start with. The best urban legends are always stories that happened to someone that you have some connection with. The story teller will also assure you it must be true because it happened to their next door neighbour's best friend's cousin. Even hoax emails tend to be prefaced with "my friend at BigCorporationBank sent this to me" or are apparently originated by a bigwig at Microsoft, Hotmail or elsewhere. We feel better disposed to trust something that comes from a source that is known to us.

Sometimes we believe because we want to hope, especially when we are sick. We want to believe that Aloe Vera, omega 3, coenzymes and carrot juice cure cancer if the alternative is to believe that we are going to die.

Health professionals are not immune to being hoaxed. We must all think as we read and question what we hear. I've mentioned CASP before, which helps you ask questions around evidence you find. The other important thing is to take care where you collect your evidence from. The National Library for Health, Intute, PatientUK, reputable charities are all a better start than the open web for good quality information - clean, clear knowledge. Oh - and did I mention that libraries can look for quality information for you and your patients?

(c) creative commons attributed

C'mon, c'mon. I need an answer!

Towards the closing minutes of each episode of University Challenge Jeremy Paxman ups the ante by putting panels under pressure to answer as quickly as possible. Sometimes this pushes a team to greatness. Sometimes it results in spectacularly wrong answers, or a set of three straight "we don't know"s.

On the Today programme this morning the decision making of NICE was discussed. Apparently they make very good decisions but take forever over them - two years compared with four months for similar decisions to be made in Scotland. NICE, MPs say, should make decisions faster - we need an answer! (The Today programme website is not helpful. It gives no details of this discussion. Luckily the story is now on the BBC news website.)

When you look at how NICE make decisions it's no wonder it takes time - the process is lengthy and involves a wide range of organisations, as this example on how clinical guidelines are developed shows.

NICE does more than churn out guidelines. Each piece of guidance is accompanied by tables of evidence, search strategies, excluded studies, background information and patient leaflets. The patient leaflets carry Plain English Campaign crystal marks and are very good.

You can search for NICE guidance on their website or through the National Library for Health . The NLH search covers guidance from overseas, too.

NICE aren't the only ones issuing guidance. In Scotland there is the Scottish Intercollegiate Guidelines Network. Decisions about what drugs can be used are made by the Medicines and Healthcare Products Regulatory Agency. In the USA this job is done by the FDA - Food and Drug Administration.

MHRA focuses on benefits and risks, NICE tends more towards benefits and costs. Neither type of decision should be made in a hurry. When drugs are used or tested too early the results can be devastating - perhaps not immediately, but in the long term. Last summer there were concerns that diabetes drug rosiglitazone could cause heart problems. Other research considered a possible link between cancer and statins - the drugs that lower cholesterol.

Rush into a decision too soon and people could get hurt. Dither over a decision and people could die while you're at it. Either way Jeremy Paxman wont be there to say "too late! I'll tell you."

(c) creative commons attributed image 1, image 2, image 3

Mother and baby doing well

We're counting the days until Christmas. For many it's a chance to have a few days off work, get lots of presents, and over indulge. It is also about the Christmas story and a woman who went on a long donkey ride and gave birth in a barn.

Mary's baby arrived safely and Mary herself suffered no ill effects. Many women aren't so lucky. Globally half a million women die every year during pregnancy and childbirth.

Even in the UK having a baby isn't without its risk. Between 2003 and 2005 almost 300 women died during pregnancy or birth. This isn't a figure we hear often. In fact, racking my brains for a story to illustrate this with the last I could recall was the Laura Touche case - she died at the Portland private hospital in London following a Caesarean section. Most of the news stories I did find were about babies dying, rather than their mothers.

In Mrs Touche's case there was an investigation into claims of negligence. Earlier this year Rowan Pelling, writing in the Independent about her Channel 4 documentary claimed that staff shortages were putting mothers at risk.

CEMACH - the Confidential Enquiry into Maternal and Child Health, has today published a report looking into the reasons why women die. They discovered that many deaths were related to obesity. According to an item on the PatientUK website the previous report into maternal deaths found that psychiatric illness was the largest cause of death.

CEMACH is one of many bodies that keeps a watchful eye on health in the UK. NCEPOD looks at patient outcome and deaths and has just published "Trauma: who cares?" looking at emergency care in the UK.

The Healthcare Commission keeps a beady eye on all aspects of health care, including rating individual NHS Trusts. It has published its own survey on the quality of maternity care.

The National Library for Health has a specialist library devoted to women's health. For the public the BBC has a range of information on pregnancy, birth and becoming a parent. It doesn't mention the risk of dying.

Interestingly, although CRUSE, BUPA and the Royal College of Psychiatrists, among others, provide information on coping with bereavement, I've not been able to find any support particular for men who have lost their wives or partners in childbirth.

(c) creative commons - image 1, image 2, image 3

Seal of approval

If you visit the library website you’ll see that we are “proud to be the first Helicon Accredited Library in the NHS in Essex”. It looks impressive - but what does it mean?

Like everything else in this modern age NHS libraries are subject to quality controls. Libraries in the NHS currently use the Helicon (Health Libraries & Information Confederation) accreditation scheme, developed in 1998 and updated in 2005. Accreditation allows the Trusts that host libraries and the SHAs that fund us to see what they are getting for their money.

Helicon is a type of peer review. A typical assessing team will consist of a senior librarian from the area, a librarian from another region, and a third person who might be from clinical governance or HR or another part of the NHS outside libraries. Some teams will also have a “shadow” who is there to see the assessing process in action, either because their own library is due to be accredited, or because they are hoping to become an assessor themselves in future.

The assessment looks at a wide range of aspects of the library service, including age and subject coverage of book stock, equality of access for all staff groups, availability of IT resources and training, the qualifications and skills of library staff and the library budget. The assessors look at what they can see in the library, talk to key people in the organisation (directors of HR, training managers, chief executives), and look over the portfolio of evidence. This will include copies of library strategies, printouts of websites, minutes of meetings, copies of leaflets, training materials, budget statements, and so on.

An assessing visit is something of a snapshot and doesn’t take into account how much the service may have changed and grown (although three yearly revisits will give some indication of this). Nor are there extra points for those libraries that have done well despite being homed in an organisation that might be less supportive than others. And, as is often the way, there is a fair amount of subjectivity in the system. However, it’s all that exists while we wait for the release of a National Service Framework for libraries.

For various reasons the East of England have been slow to take up Helicon. This library was in the first wave of services to be assessed, and since then many others have been through the process. This library is due for reaccreditation in early 2008. Sadly, some recent changes could mean that we’ll struggle to reach even the basic grade, which would make us the first library in Essex to lose accredited status.

Bad Science?

The Guardian newspaper has a regular column on “bad science”. On 24th February that column looked at a review in a medical journal of a book on autism aimed at lay people.

Why was this “bad science”? Well, the article argued that the book’s author is an “activist” who has been “highly criticised” and who has his own agenda that is not much supported by the body of medical knowledge and opinion. The review of the book, however, was “flattering” and made no mention of the author’s history.

Quite rightly the Guardian doesn’t propose censorship. After all – where would we be now in terms of scientific knowledge if it wasn’t for those over the centuries who have dissented from mainstream opinion? The Guardian argues that some background on the author would have been helpful in enabling people to make a thoughtful and balanced reading.

A journal perhaps has some duty of care towards its readers in offering some sort of background information. But it is up to each and every one of us to read with our brains switched on, to read thoughtfully, carefully and critically.

There are various resources to help you do this. CASP (Critical Appraisal Skills Programme) has tools for appraising different types of health research. For appraising websites there are various things to look out for including the HoN (Health on the Net) logo. It’s also worth checking dates showing when material was last updated and any “about” page that might show who the author is and what particular bias they might bring to information on the site. On a lighter there is a brief animated checklist, the Quality Information Checklist . It is aimed at children but covers the basics.

The other issue that this Guardian piece raises is that of newspapers picking up on stories and giving them sensationalist, and possibly misleading, headlines. It’s best always to read beyond the headline. Most health news in papers can be tracked down to a recently published scientific paper. Don’t accept the interpretation of the Sun, the Mail – or even the Times or the Telegraph. Go straight to the source and make you own mind up. Another useful source for assessing news stories is the Hitting the Headlines section on the National Library for Health website. It looks at recent health news stories and assesses the evidence behind the headline.

For further reading on all of this, why not drop into the library? We have Trisha Greenhalgh's very readable work on the subject, How to Read a Paper, now in its third edition, and many others. We can also offer more advice and guidance. All free, as ever.