Systematic reviews
A systematic review is characterised by meticulous planning, methodical work procedures and a transparent, complete record of all assessments carried out in the course of the work. Systematic reviews often clarify fairly limited issues, but since they are conducted with great thoroughness they can nonetheless be extensive.
Systematic reviews follow predetermined protocols
The first task for the team that is to conduct a systematic review is to formulate and finalise the question it will address, the review question. Next, the group establishes a review protocol, i.e. a detailed plan of how to carry out the work. At this stage, decision-makers and other stakeholders can also contribute their views.
The review team then attempts to identify all research findings that may be of interest in the context. Searches are performed, first, among articles in international academic literature. Supplementary investigations can also be carried out in official reports and other ‘grey’ literature. There may also be direct contact with scientists or departments that may, perhaps, possess valuable data. Unpublished data may also be useful. Throughout, the crucial requirement is for the material to be so well documented that its relevance and quality can be appraised.
The next stage is to screen the hundreds or even thousands of articles and reports identified by the literature searches. Eventually, perhaps only about one or a few dozen studies may prove to be relevant and well executed enough to be help answer the question that the review is intended to elucidate.
A research team that recently carried out a systematic review of how vegetation is affected by reindeer grazing found more than 6,000 articles in their literature searches. It was possible to exclude a high proportion of these articles, however, because of their titles alone. After continued screening, just 40-odd studies with sufficiently usable data remained. From Bernes et al. 2015 (EviEM review SR1).Synthesising results from different studies
After the screening, the usable data are summarised and their quality assessed. If the results are sufficiently uniform and comparable, it may be possible to combine them in one or more synthesised measures by means of a meta-analysis. Otherwise, a narrative summary of the findings is written. The documentation may also prove inadequate to clearly resolve the issue concerned; in this case, the review can pinpoint knowledge gaps that need filling in by means of further research.
Before the outcome of the review is finally published, decision-makers and other stakeholders get another chance to submit their views. This time, the focus is especially on how results are summarised and presented.
The figure above summarises results of a review of how biomanipulation (removal of fish) affects the transparency of nutrient-rich lakes. For each individual lake, the mean effect of biomanipulation on transparency (combined with a measure of uncertainty) was specified. At the bottom, the individual results are combined in an estimated weighted mean that implies an average transparency increase by about 22 cm during biomanipulation. From Bernes et al. 2015 (EviEM review SR3).Transparent, well-documented procedures
Systematic reviews are entirely based on existing studies. In this respect, they do not differ from ordinary literature reviews of scientific questions. The difference lies, instead, in the way of working. What characterises a systematic review is that every stage is planned and documented in detail. The final report gives an account, for example, of exactly which limits were applied to the choice of subjects, which search terms and databases were used in the article search, which articles were eliminated in the screening and the reasons why these were not considered usable.
Transparency of this kind is essential, not least if reviews of controversial subjects are to be capable of earning public confidence. When new research results emerge, it is also relatively easy to supplement the reviews and, where necessary, revise the conclusions.