In this guest posting, Dr Ruth Stewart, a collaborator on this research project, writes about why we are using a systematic review methodology in the first phase of our research project, and explains what exactly a systematic review is. Ruth is a research fellow at the Centre for Anthropological Research at the University of Johannesburg, where she co-directs the Collaboration for Environmental Evidence Johannesburg Centre, and she is a senior research officer at the EPPI-Centre of the Institute of Education at the University of London. (Photo: Ruth and Carina making ready for World Cup action in 2010 – own photo)
Part of the work of this Thuthuka research project is to review the science on climate change and water in South Africa. These reviews won’t simply be literature reviews though – they are using systematic review methods. If you’re not sure what this means, I’ll try and explain a bit more below.
Systematic reviews are essentially large pieces of ‘secondary’ research. To conduct a review, you need to search out, collect, read and combine all the available relevant research on a given topic in order to provide ‘the answer’ to a question of importance (Moyniha 2004). That question might be an exploratory, theoretical one, in which case you would seek out conceptual research, or it might be one on the effectiveness of an intervention, in which case you would seek out impact evaluations. Systematic reviews typically take a team of two-to-four people around a year to complete. They require specialist methodological skills and topic expertise. The review process is laborious, and typically results in a long and detailed report for the purposes of completeness and transparency. Reviews often begin by identifying many thousands of potentially relevant references, which are then filtered down by relevance and quality to include only the good quality pertinent research.
You might ask whether all this effort is worthwhile when more traditional literature reviews are so much simpler and primary research seems more ‘cutting edge’. But systematic reviews have been proposed as a means to avoid research sitting on a shelf unread, and instead contributing to understanding issues and finding solutions to the world’s problems, and enabling decision-makers to access the best available research evidence to inform their decisions. The suggestion is that, if you want your research to make a difference, then you need to engage with systematic reviews, even if you do not conduct them yourself.
For some people systematic reviews are synonymous with randomised controlled trials and ‘positivist’ knowledge, and indeed this has been their history, but the methodology is simply one that can be applied to a wide variety of evidence. As suggested by the name ‘systematic review’, in order to reach the final synthesised findings of the available evidence-base we must simply follow a logical process. This includes the stages employed in most research: establishing your research question and conceptual framework, collecting your data, analysing it, reflecting on the strengths and weaknesses of your methods, and drawing out your results and conclusions. Each step is set out in advance and described in detail in a protocol to ensure transparency and instil trust in your reviews findings.
Whilst relatively new in environmental sciences, this approach is standard practice in medicine, health promotion and some areas of social policy, where policy-decisions are not made and new research not commissioned without first understanding the combined findings of the best-quality and most relevant research evidence as reported in a systematic review (see the Cochrane Collaboration and the Campbell Collaboration).
Reflecting on the use of evidence in health-care decision-making, Pullin and Knight (2009) have noted that “environmental management has, up until now, had no formal shared evidence-base of this kind.” In the last ten years, Andrew Pullin and colleagues at the Centre for Evidence-Based Conservation at Bangor University, UK, have led an emerging paradigm shift towards environmental decision-making based on the systematic collation of rigorous evidence. This shift was formally reflected in the recent establishment of the Collaboration for Environmental Evidence (CEE).
Earlier this year the Centre for Anthropological Research at the University of Johannesburg agreed to host CEE’s first regional centre – CEE Johannesburg. The Johannesburg Centre includes a team of experienced systematic reviewers who have been using this methodology in health, education and development for many years (for example, see Van Rooyen et al (2013) on what a systematic review methodology for development means). We are now working to promote the approach throughout the southern African region.
Carina’s and colleagues’ research on climate change knowledge in South Africa will be using the principles of systematic reviews to understand different shades of evidence on climate change, including reviewing both ‘science’ knowledge and local environmental knowledges. In doing so, they will provide the first structured transparent overview of evidence on climate change across the region. I, for one, am looking forward to reading these reviews.
List of references
Moynihan R 2004 Evaluating health services: A reporter covers the science of research synthesis. New York: Milbank Memorial Fund
Pullin AS & Knight TM 2009 Doing more good than harm – building an evidence-base for conservation and environmental management. Biological Conservation 142(5): 931-934
Van Rooyen C, Stewart S & de Wet T 2013 Systematic review methodology for development: An example from micro-finance. Africanus 43(1): 65-77