Category Archives: Search Strategies

When search examples go bad

You know the feeling when you are looking for a search example and up comes an article advocating a Google search? Irritation! Annoyance! Eye rolls! Big sighs!

Years ago I performed a search demonstration about EpiPens and accidental overdoses. One article that came up was titled: Accidental Epipen injection into a digit – the value of a Google search. Arrggghh! So what happened here? A child accidentally injected himself with his mother’s EpiPen and was rushed to the emergency dept at a UK hospital. Experts were consulted but didn’t know what was the best course of action. Clinicians searched PubMed but didn’t find anything: “a literature search was carried out on PubMed using the following key words: ‘pediatric’, ‘digital artery’, ‘epinephrine injection’. No citations were found.” So they turned to Google and found one article that was very pertinent.

But guess what? The article they found is in PubMed!!!!! And the hospital has a well resourced medical library. So what is going on here? Is it ignorance of search methods? Is it a rapid need for info leading to knowledge of search methods going out the window?   I wonder if this hospital has access to the Chasing the Sun service – a service set up between hospitals in the UK and Australia in order to provide 24hr librarian assistance (and this service has been used in out of hours emergencies).

What can we learn from this?

Advertisements

The death of a research subject

In 2001, a clinical trial participant at Johns Hopkins (JH) died as a result of inadequately researched safety information. Ellen Roche was a 24 year old lab assistant at the JH Asthma and Allergy Center and thought to enrol in the study in altruistic intent (as most clinical trial participants do). Immediately following Roche’s death, research funding was suspended and JH was left scrambling to address the controversy. Medical librarians in the US went into overdrive over the incident. It appeared that the lead researcher did a rudimentary search of the literature. He searched PubMed (Medline records from 1966 onwards were available in 2001 and in order to discover older records (hexamenthonium was used extensively in the 1950s) you would have to have searched print indexes), Google, Yahoo!, LookSmart and GoTo.com. Librarians and researchers said that the literature search was lazy and foolhardy. Not only were the researchers at fault, but the JH Institutional Review Board (IRB) was at fault also for not providing proper oversight. Librarians talked about literature search standards and what should constitute a reasonable search. JH have now included literature search standards in the IRB application form and many other IRBs across the US have too.

JH has well resourced libraries – why didn’t the researcher contact them to ask for assistance? Was this a failing of the library as well? The Welch Medical Library began liaison services in 2000 for a few departments (Asthma and Allergy and the IRB were not included). What does this mean for medical librarians now? Perhaps librarians could try and work closer with their research offices in providing some sort of literature search service – approving search strategies and advising researchers where they can be improved. It could be spun as part of the quality control process of clinical trial management. There also needs to be better reporting of adverse effects.  Apparently in a hexamenthonium trial in the ’70s, adverse effects were not included. Adverse effects reporting is still a problem today and also a problem for librarians in searching for adverse effects literature due to inadequate indexing amongst others.

When I first read reports about the death of Ellen Roche and the swirl of commentary about it in the library e-list, it touched me, as it did others. As health professionals, we have a duty to get involved in research activities in order to prevent another incident like this happening again.

Further reading:

The hexamethonium asthma study and the death of a normal volunteer in research

Johns Hopkins’ Tragedy: Could Librarians Have Prevented a Death?

 

HTAi15 IRG Advanced Searching Workshop: Understanding and Using Text Analysis Tools

IRG Text Analysis 15The final session of the IRG Advanced Searching Workshop was divided up into 4 parts and was rather intensive. Well, the entire workshop was intense! A lot was covered. Julie Glanville from York Health Economics Consortium spoke briefly about text analysis tools (which was covered in more detail at the HLA event earlier this year) and Carol Lefebrve talked about the 2002- project to test the Cochrane Highly Sensitive Search Strategy for RCTs (developed in 1993-4) to determine whether it still performed well with as little terms as possible. Since the development of the Medline filter, a few changes had occurred – the addition of the Controlled Clinical Trial publication type in MeSH (quasi-randomised trials), better trial reporting due to CONSORT guidelines, and the development of text analysis tools. It was through testing with a gold standard set of RCTs and a set of non-trial records using WordStat that the best identifier term was Controlled Clinical Trial PT. But due to indexing changes (RCTs and CCT PT double-indexing cessation) reassessment led to the best identifiers (those terms locating most RCTs) was  RCT MeSH OR CCT PT. This was one of the issues with the filter that Carol mentioned (always be aware of indexing policies and changes!) and the other was the non-text terms to catch non-indexed records. Siw Waffenschmidt and Elke Hausner from the Institute for Quality and Efficiency in Healthcare (IQWIG) discussed generating search strategy development test sets for data analysis and guidance for identifying subject terms and text-words. The guidance is from the recently published EUnetHTA Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness (still in 2nd draft stage). Hausner spoke about the work she and other IS researchers did in comparing the conceptual and objective approach to search strategy development, one which is elaborated in this journal article: Development of search strategies for systematic reviews: validation showed the noninferiority of the objective approach.  Basically, the research showed that a conceptual  strategy developed by Cochrane with more synonyms was not superior to a objective search strategy on the same topic developed by IQWIG. However, the objective approach is not faster than the conceptual. Time saved is not the issue here though, it is the quality of the search. IQWIG demonstrated with their projects that the conceptual approach can produce high quality strategies that are stable over time and more suited to complex questions.  Take home points: text analysis tools are here to stay! It will take time to learn this approach but the plus side is that it produces strategies of equal quality to those developed using the conceptual approach as well as data to demonstrate strategy development and decision-making.