Adventures in screening systematic reviews

I’m screening full text systematic reviews for an umbrella review and I have seen some atrocious examples. If you follow me on xTwitter, you will have come across some of my rants (I also say hooray when authors get things right). In this post, I want to review some of the common errors I’ve come across in order to help others who are on the review writing train.

1. Searching in just one database
2. Listing databases used
3. The search strategy
4. PRISMA
5. Not consulting a librarian

Searching in just one database. A few reviews stated that only one database was searched. One had the audacity to state that no duplicates were found. If only one database is used, this makes it a literature review, NOT a systematic review.

Listing databases used. A common issue is listing platforms and publishers as databases. Another is not specifying what database is being used when a collection of databases is being used. A review I looked at today listed Cochrane and Elsevier in the list of databases. Cochrane is an international organisation and Elsevier is the world’s largest STEM publisher. When you want to indicate that you used databases in the Cochrane Library, specify whether it was the Cochrane Database of Systematic Reviews (CDSR) or CENTRAL. It is important to list what platform was used to search. This is because databases are available on a variety of different platforms and informing you which platform was used increases reproducibility. A few others have listed Web of Science (WoS) as a database. WoS is a platform that contains many databases, including Medline. One review stated they searched PubMed, Medline and WoS (depending on their institution’s subscription, they probably searched Medline). They searched Medline 3 times!

The search strategy. Medline alone contains over 36 million citations. When your citations from all your database searching doesn’t even reach 500 (or is even less), something is seriously wrong with your search strategy. Other issues include very badly crafted search strategies (and I have seen some gawd awful ones, let me tell you). When someone is doing a critical appraisal of a systematic review, one of the first questions is about the search strategy. Is it robust enough to make it worthwhile continuing the appraisal? Many reviews have fallen at the first hurdle. Another strange issue is the number of people running the search strategies. One review stated all four authors ran the search strategy in all databases. Why? They would all get the same results (hopefully)!! Only one person has to run the searches and download all citations.

A mantra to keep in mind: the search strategy is the foundation of the systematic review. A bad foundation undermines the review.

PRISMA. PRISMA is a reporting guideline as per its full title: Preferred Reporting Items for Systematic Reviews and Meta-analysis. It is NOT a handbook or guideline for writing/conducting the review. When referring to PRISMA, please state that it was reported using/according to PRISMA.

Not consulting a librarian. All of these problems could have been avoided if the authors included a librarian on the team. If your institution or organisation has librarians on staff, please consult them! They can be on the sidelines reviewing your strategy and providing advice, right up to full authorship with screening and commenting on draft versions before journal submission. Do you want your review to pass the first test in critical appraisal? Get a librarian on board!!

Catherine goes to prison

Last Friday, I went to visit a Forensicare researcher at the Melbourne Assessment Prison (formerly the Melbourne Remand Centre). Our library services a large range of groups who work in a variety of locations. Forensicare staff work in high security sites as well as in the community. This researcher worked in a prison and I thought it would be an interesting excursion to visit her (OK, I was very curious – I’ve never visited a prison before) and sometimes, showing a person how to search databases etc works better if they are in a familiar environment. It was raining when I left work to get on the tram and by the time I was at LaTrobe St, it was pouring heavily. My trousers were almost soaking! The researcher met me at reception and I signed in. Then I passed over my passport and staff ID for ID purposes to the guard and was then given a blue bracelet and visitor lanyard.. I had to put all my stuff in a locker – not even allowed to bring a pen in, but allowed paper. I had my sheaf of library bumpf and how-tos – that was allowed. Then we passed through a body scanner and then another scanner (this was for items being brought in ). Then to get to the staff meeting room with a PC, we had to go through 3 or 4 heavy doors with eye scanners. The researcher had to lift the flap and look into the scanner. At one guard point, it was looking in the eye scanner for her and for me, holding up my wrist to the window to show the guard I had been processed by the guards at entry. Past the first heavy door was the prisoner visitor communication booths. It was quite small and cramped looking. The building looks quite large from the outside but inside, from the areas I saw anyway, it was quite small. I guess the walls are very thick. I was warned it would take a long time getting through (it took about 30mins) and before I even got to this point, I had to provide the researcher with some personal details in order to be approved.

How can I end this post? If you have clients in multiple locations, it is worthwhile to do site visits. Not every client is able to physically visit the library for instruction sessions or research advice. Not only is it an excursion for you, it is also an opportunity to see what their work environment is like – what is technologically possible and what work-arounds could work. Next up is a visit to the Royal Park campus.

451? Its library ethics!

I was invited to join in this movement by Sarah Houghton this week. 451 has a US focus, but librarians of all types in Australia can engage in 451 activities on January 20th 2017 in their own way – be it individually, as a group or as an organisation. These activities are:

1. libraries should co-operate with people working to resist restriction of freedom of expression and access to ideas
2. a person’s right to use a library should not be denied due to cultural heritage, views or age
3. no one should interfere with a persons right to free speech, freedom of assembly or association

There is a worrying trend of ultra-conservatism mixed with the rise of fake news and racism and it is easy to feel disheartened. However, there are positive steps being taken.  IFLA recently announced the creation of a new network for librarians interested in Freedom of Access to Information and Freedom of Expression (FAIFE). Article 19 in the Universal Bill of Human Rights is Intellectual Freedom, which all information industry workers should and must work for. In early 2015, ALIA introduced FAIR – Freedom of Access to Information and Resources, but its main focus is on access and not expression or privacy.

Contrary to popular belief, Australia does not have a freedom of speech clause in the Australian Constitution, freedom of political communication is implied only. However, the Constitution does ensure the right of Australians to a trial by jury in criminal cases. This (these?) seems to be the only human rights enshrined by the Constitution. Australia is signatory to International Covenant on Civil and Political Rights which gives people the right to assembly and association, but these can be restricted under certain conditions. In some cases, these conditions are being circumvented.

What can you do? There are a number of things I can think of:

1. Help people to distinguish fake news from fact
2. Assist people in how to protect themselves online – not only from scammers but also from organisations harvesting their details and from intrusive spying
3. Have an inclusive public library collection
4. Make library spaces safe neutral zones
5. Protest against government mass metadata collection and broadening of spy agencies powers
6. Support individuals in their Freedom of Information requests
7. Condemn censorship

Let’s make Jan 20 2017 a day to celebrate and support intellectual freedom. Let’s make this day an everyday.