Making literature reviews more reliable: lessons from systematic reviews
NEWS | 2015-06-05
Systematic reviews are ‘gold standards’ in evidence synthesis, but there will always be situations where traditional reviews are valuable. In a recent paper in the journal Conservation Biology, EviEM Project Leader Neal Haddaway and colleagues discuss how the reliability of traditional literature reviews can be improved by learning from systematic reviews.
Literature reviews are valuable summaries of research that can aid decision-makers and researchers in understanding a large evidence base. The volume of scientific research being published is rapidly increasing, but reviews provide a way to get an overview of an entire field of knowledge through one article.
However, traditional reviews are susceptible to a suite of pitfalls that can reduce the reliability and accuracy of their conclusions. These biases and inaccuracies can creep in, for example, through a search that is not comprehensive and misses vital research, or through selectively focusing on one group of studies without due cause.
Systematic review methods aim to minimise such sources of bias and inaccuracy through a range of different specific activities. These include writing a detailed plan, or protocol, outlining the review methods in advance; using an extensive, carefully constructed, comprehensive search for literature; screening and appraising all studies according to predefined criteria; and fully documenting all assessments.
Such work can be very time consuming and resource intensive. As a result, some topics may be unsuitable for systematic review: for example, if short timescales operate, or if resources are restricted. In these situations, traditional literature review may be the only option. However, lessons can be taken from systematic review methodology that can improve the reliability of these reviews whilst still being feasible within the existing constraints.
In their recent paper in Conservation Biology, Haddaway and colleagues outline the key steps that can be used to increase reliability at different review stages where full systematic review is not feasible.