Last week we spent 16 hours on Zoom participating in the fourth Evidence Synthesis Institute cohort. The Evidence Synthesis Institute is aimed at library staff supporting evidence syntheses outside of the health sciences and is fully funded by the Institute of Museum and Library Services. There is no charge for accepted applicants. The institute’s instructors primarily came from the University of Minnesota, Cornell University, and Carnegie Mellon University, but there were also instructors from Canada, Africa, and other universities in the US. Topics covered include

• An overview of systematic reviews and similar methodologies
• Guidelines and reporting standards
• Search strategy development
• Software tools
• Quality assessment and systematic review services

Hu and Kathy’s Takeaways: 

• Evidence Synthesis (ES) is a growing research area outside of the health sciences. 
  • While ES is well-supported and structured in the health sciences and in medical libraries, it’s still relatively new outside of that area. However, it is steadily growing. A search for systematic reviews in almost any social science database shows a steady increase in the number of reviews published over the past several years. 
• Librarians play a key role in Evidence Synthesis. 
  • The core evidence synthesis organizations, including Cochrane (health sciences) and Campbell (social sciences) explicitly recommend that researchers include librarians in ES research. 
  • Librarians can take on various roles, from helping researchers develop and refine a research question to developing search strategies to being a full co-author.
• ES is bigger and more complex than expected.
  • ES projects can take just as long as an original research study (expectation is that these take at least a year, if not longer). 
  • This article by Grant and Booth (referenced multiple times in the Institute) outlines 14(!) types of ES reviews and methodologies. 
• Libraries offer a wide range of types of programs to support ES
  • Many programs offer a two-tier model: Consultant or Co-Author. What role the librarian takes is usually based on the librarian’s capacity. In some places, librarian support is limited to grant-funded ES projects.
  • In terms of who provides ES support, some libraries have whole ES departments; in others, ES as an official part of 1-2 people’s jobs; and in others, there is a small ES team made up of people that are also liaisons and these individuals partner with other liaisons to support ES. 
• Database searching on steroids! 
  • Essentially one whole day was devoted to search strategies, and we both left feeling that our database searching had improved dramatically!  
  • ES research requires a level of detail and precision that we don’t typically employ working with undergraduates. It also requires a different mindset, and we talked a lot about how to create searches that truly seek to capture ALL of the literature on a narrow topic. 
  • We also learned a lot about the advanced search features in several of our big databases, including Web of Science, Scopus, and PsycINFO, including combining queries and using search terms as limiters. 
• Value of grey literature for ES
  • In addition to the focus on using library databases, we also talked quite a bit about including grey lit in ES, and the value that it adds to your search. About 50% of conference presentations, for example, never turn into published studies; dissertations and theses are a wealth of information; and nonprofits, think tanks, and more are constantly creating materials that can speak directly to the topic of an ES research project. We also learned some strategies for finding it (since that piece is not as straightforward as database searching).
• Abundant resources are available to support ES, including cool tools and resources like: 
  • Publish or Perish-an application that retrieves and analyzes academic citations. For example, it can batch pull citations from a Google Scholar search and offer a variety of citation metrics.
  • Rayyan-a cloud-based web application for researchers working on systematic reviews and other knowledge synthesis projects that anonymizes and streamlines the screening process. 
  • Covidence-(currently licensed for the WFU Medical School) is another tool like Rayyan, used for primary screening and data extraction, and comes out of the Cochrane Collaboration. (The Cochrane and Campbell Collaborations both work to foster well-informed decisions drawing on relevant and sound evidence, with the Campbell Collaboration focusing on the Social Sciences.)
  • SR Toolbox-Developed and founded by Dr. Christopher Marshall and Anthea Sutton in 2014. The Systematic Review Toolbox is an online catalog of tools that support various tasks within the systematic review and wider evidence synthesis process.
• First steps – implications for grad students 
  • While we plan to be able to offer ES support as a service going forward, we wanted to begin by thinking about how we can support our graduate students. While most of them aren’t doing full systematic review/ES, these strategies have a lot of potential applications for the literature review that most of them do for their thesis. We hope to introduce some of these strategies to Communication and Psychology graduate students this fall. 
• Long-term – Broader ES Support 
  • As we build expertise on our team, we are looking toward developing a model of support that works for our context. In the meantime, if you hear of a faculty member who is interested in doing a systematic review or another form of ES, please send them our way! 

We covered a LOT of ground in 4 days, and we both left excited to begin implementing evidence synthesis support on our campus! We feel like we are now part of a community we can reach out to for guidance. ESI also has plans for a conference in 2024 for all past participants.

We highly recommend any librarians supporting or interested in evidence synthesis at WFU apply to the Evidence Synthesis Institute. The institute may be free, but the experience you gain is invaluable. (Note: The IMLS grant originally paid for the costs of an in-person institute at UMN, but since COVID, they have been able to move it online, and as a result, they can accept more than double the number of participants.) It will be online again in 2023, so if you are interested, we highly recommend applying for next year’s cohort!