Adventures in screening systematic reviews

I’m screening full text systematic reviews for an umbrella review and I have seen some atrocious examples. If you follow me on xTwitter, you will have come across some of my rants (I also say hooray when authors get things right). In this post, I want to review some of the common errors I’ve come across in order to help others who are on the review writing train.

1. Searching in just one database
2. Listing databases used
3. The search strategy
4. PRISMA
5. Not consulting a librarian

Searching in just one database. A few reviews stated that only one database was searched. One had the audacity to state that no duplicates were found. If only one database is used, this makes it a literature review, NOT a systematic review.

Listing databases used. A common issue is listing platforms and publishers as databases. Another is not specifying what database is being used when a collection of databases is being used. A review I looked at today listed Cochrane and Elsevier in the list of databases. Cochrane is an international organisation and Elsevier is the world’s largest STEM publisher. When you want to indicate that you used databases in the Cochrane Library, specify whether it was the Cochrane Database of Systematic Reviews (CDSR) or CENTRAL. It is important to list what platform was used to search. This is because databases are available on a variety of different platforms and informing you which platform was used increases reproducibility. A few others have listed Web of Science (WoS) as a database. WoS is a platform that contains many databases, including Medline. One review stated they searched PubMed, Medline and WoS (depending on their institution’s subscription, they probably searched Medline). They searched Medline 3 times!

The search strategy. Medline alone contains over 36 million citations. When your citations from all your database searching doesn’t even reach 500 (or is even less), something is seriously wrong with your search strategy. Other issues include very badly crafted search strategies (and I have seen some gawd awful ones, let me tell you). When someone is doing a critical appraisal of a systematic review, one of the first questions is about the search strategy. Is it robust enough to make it worthwhile continuing the appraisal? Many reviews have fallen at the first hurdle. Another strange issue is the number of people running the search strategies. One review stated all four authors ran the search strategy in all databases. Why? They would all get the same results (hopefully)!! Only one person has to run the searches and download all citations.

A mantra to keep in mind: the search strategy is the foundation of the systematic review. A bad foundation undermines the review.

PRISMA. PRISMA is a reporting guideline as per its full title: Preferred Reporting Items for Systematic Reviews and Meta-analysis. It is NOT a handbook or guideline for writing/conducting the review. When referring to PRISMA, please state that it was reported using/according to PRISMA.

Not consulting a librarian. All of these problems could have been avoided if the authors included a librarian on the team. If your institution or organisation has librarians on staff, please consult them! They can be on the sidelines reviewing your strategy and providing advice, right up to full authorship with screening and commenting on draft versions before journal submission. Do you want your review to pass the first test in critical appraisal? Get a librarian on board!!

Joining a clinical trial

I have read countless reports of clinical trials but I had never been in one. Until I got an invitation to join one. Last month, I got a letter from the hospital I attend (not the one I work at) along with an appointment letter. I thought it would be interesting to have an inside view of a trial and I contacted the lead researcher. After a phone call assessing eligibility, an appointment was made for me to visit the main hospital for an interview, blood test and covid booster. This took about an hour. What is this trial? It is the BOOST-IC trial. It aims to determine whether an extra covid booster vaccination gives extra protection to people with solid organ transplants (that’s me), blood cancers (like leukaemia), or AIDS. Even though my kidney is now failing (last bloods indicated 12% function), I was still eligible.

After the initial clinic visit, I had to fill out an online form about vaccine side effects, which included taking my temperature every day for 7 days. Then nothing doing until the following month when I will go to one of the hospital pathology clinics to have a blood test. Then some more weeks will go by and I will have another blood test.

At the first appointment, I was asked if I would like a copy of the results when they become available. I demurred, saying that I will look it up and read the RCT. I asked which journal they were considering and they hadn’t decided on one yet. They have a wide choice due to the population they are studying. I also mentioned JANE (Journal Author Name Estimator) and that they could present at numerous conferences (AIDS, various cancer conferences, vaccinations/immunology and renal).

It will be years until I get to read it though!

If you are invited to join a trial or come across one that is recruiting that you are eligible for, I encourage you to join. It’s an interesting project to be involved in.

Catherine goes to prison

Last Friday, I went to visit a Forensicare researcher at the Melbourne Assessment Prison (formerly the Melbourne Remand Centre). Our library services a large range of groups who work in a variety of locations. Forensicare staff work in high security sites as well as in the community. This researcher worked in a prison and I thought it would be an interesting excursion to visit her (OK, I was very curious – I’ve never visited a prison before) and sometimes, showing a person how to search databases etc works better if they are in a familiar environment. It was raining when I left work to get on the tram and by the time I was at LaTrobe St, it was pouring heavily. My trousers were almost soaking! The researcher met me at reception and I signed in. Then I passed over my passport and staff ID for ID purposes to the guard and was then given a blue bracelet and visitor lanyard.. I had to put all my stuff in a locker – not even allowed to bring a pen in, but allowed paper. I had my sheaf of library bumpf and how-tos – that was allowed. Then we passed through a body scanner and then another scanner (this was for items being brought in ). Then to get to the staff meeting room with a PC, we had to go through 3 or 4 heavy doors with eye scanners. The researcher had to lift the flap and look into the scanner. At one guard point, it was looking in the eye scanner for her and for me, holding up my wrist to the window to show the guard I had been processed by the guards at entry. Past the first heavy door was the prisoner visitor communication booths. It was quite small and cramped looking. The building looks quite large from the outside but inside, from the areas I saw anyway, it was quite small. I guess the walls are very thick. I was warned it would take a long time getting through (it took about 30mins) and before I even got to this point, I had to provide the researcher with some personal details in order to be approved.

How can I end this post? If you have clients in multiple locations, it is worthwhile to do site visits. Not every client is able to physically visit the library for instruction sessions or research advice. Not only is it an excursion for you, it is also an opportunity to see what their work environment is like – what is technologically possible and what work-arounds could work. Next up is a visit to the Royal Park campus.