UK Academic Research Contributions to the SDGs Observed Through National Assessment Submissions

Back in May 2020 we released our report, Contextualizing Sustainable Development Research, showcasing the growth in research around the UN’s Sustainable Development Goals (SDGs). We are continuing our blog series on SDG-related research with a post by Dr Juergen Wastl, Dr Briony Fane and Bo Alroe from Digital Science’s Consultancy Team.

Juergen Wastl is Director of Academic Relations and Consultancy at Digital Science. He previously headed up the Research Information team at the University of Cambridge’s Research Strategy Office and worked for BASF managing BMBF-funded projects internationally.

Briony Fane is a Research Analyst at Digital Science. She has a higher education background, having gained a PhD from City, University of London, and has worked as both a researcher and a research manager.

Bo Alroe has worked with research management and administration since 2004,  and currently as Director of Strategy with Digital Science. Bo is from Aalborg, Denmark, where he studied and lives with his family.

Introduction

Assessment of research in the UK helps fund and shape the development of academic disciplines in Universities. The recurring seven-year cycles of assessment exercises that Higher Education takes part in has a long history, starting in 1986 and evolving and culminating in the latest versions known as the Research Assessment Exercise (RAE) in 2001 and 2008 and the Research Excellence Framework (REF) in 2014 and 2021. The outcomes of these exercises tell us about the state of research that is taking place across the United Kingdom, from its academic strength and impact to the significance placed on research from social, economic, clinical and policy perspectives.

Reflecting on the last assessment exercise, REF 2014, outcomes would endorse that the Higher Education sector performs exceptionally well both academically, and in respect of the impact of its research in a real-world setting, as our in-depth post-REF 2014 analysis revealed.

The UN SDGs seek to tackle the biggest societal challenges in our world today, in part by asking us to fundamentally question long-held beliefs about economic growth and their compatibility with maintaining a healthy balance with the natural world. As higher education institutions have a uniquely important role to play in delivering solutions to the SDGs it would make sense that a proportion of the research submitted to the UK’s assessment exercises will have clear links to the SDGs. This focus on societal, economic and environmental challenges facing the world can be well captured by the need for the provision of impact (showcased by case studies) that was introduced in REF 2014.

We have used Digital Science’s Dimensions to shed light on the contribution of SDG-related literature to the UK’s REF 2014 submission. Dimensions offers a comprehensive collection of interlinked data in a single platform that reflects the entire research lifecycle. As Dimensions covers research inputs, outputs and data on pathways to impact, it is uniquely suited to teasing out contributions to SDG-related research. For both SDG and REF Unit of Assessment (UoA) classifications, Dimensions offers filters developed using supervised machine learning based on custom-made training sets. As a consequence, the UK’s assessment of research can be viewed in the context of sustainable development.

We start by examining REF 2014, in an attempt to ascertain whether SDG-aligned research is growing in content beyond the normal base rate growth of research as we progress towards the deadline for achieving the Goals in 2030¹.  Dimensions includes classification filters based on the REF 2021 UoA classification system comprising 34 UoAs.  These can be easily applied to the previous assessment exercises, despite research areas being defined slightly differently in previous assessments,  with 67 and 36 UoAs in REF 2014 and RAE 2008 respectively. Dimensions was used to retrieve SDG-classified research publications that were submitted in the REF 2014 period from 2008 to 2013.

Exploring REF 2014 and SDG-related research outputs in Dimensions

We start the blog series with examples of the percentage of SDG-related research submitted per UoA in REF 2014, and will address the following research questions:

1: Is research in the context of Sustainable Development submitted and deemed excellent research as defined by REF 2014?

2: What can the spread of SDG-related research submitted to REF 2014 tell us?

3: Can trends be found in SDG research that contributes not only to REF 2014 but across all years that have been part of research assessment? Is there evidence of SDG associated research before the SDGs were formally implemented in January 2016²? Will we see a steady increase in growth of SDG linked research across the research assessment timespan?

Table 1: Examples of the proportion of SDG-related research submitted to UoAs in REF 2014

Table 1 provides specific examples of the proportion of SDG-related research submitted in the UoAs featured. It highlights not only the importance of SDGs to academic research in units of assessment in clinical sciences and areas of social sciences, but also areas in the physical sciences where SDG research does not contribute significantly to REF 2014

1: SDG research is ‘excellent’ research

We can show that SDG tagged research is considered ‘excellent’ research in the UK in the context of the national research evaluation exercise.  By its very nature, we recognise that SDG-related research is, to varying degrees, ‘applied research’ because it addresses societal challenges, and ‘excellent’ research because it contributes to the national research assessment submissions. The evidence for this lies in the fact that, for research outputs submitted to the REF, we can identify what percentage are tagged as SDG research in Dimensions. If the percentage of SDG research is high, it demonstrates its level of excellence as perceived by the submitting academic and institution. The UK’s REF 2014 submission of research outputs has, in a number of Units of Assessment, a higher content of SDG-associated research than the average across the timeframe of the exercise.

Figure 1: Number of SDG-related research outputs submitted to REF 2014Figure 2: SDG-related research outputs submitted to REF 2014 as a percentage of total submissions per Unit of Assessment

2: What can SDG-related research tell us about the UK’s REF 2014 submission?

SDG tagged publications vary in number and share (see Figures 1 and 2 above) across the UoAs however trends, or hot spots, of individual SDGs per UoA are revealed. For example, SDG7, Affordable and Clean Energy (green) across Main Panel B UoAs in the physical and technological focussed UoAs, and SDG 3, Good Health and Wellbeing among Clinical Medicine and Life Sciences based UoAs (red), we see that SDG-related research coverage is considerable.

In order to have a summary view of the REF 2014 submission (and beyond) at the level of Main Panels, aggregating research in the context of SDGs helps to give us an overall picture with two main insights. Applying the SDG filter across the evaluation time frames of RAE 2008, REF 2014 and the current REF 2021, we can affiliate the SDG tagged research to the Main Panels over a period of 20 years (noting that the main panels have remained the same, but the UoA structure slightly varied across the three assessment exercises).

Figure 3: SDG-related research outputs submitted to REF 2014

Looking at figure 3, the first main insight that is immediately apparent for Main Panels A and B, which focus on STEM subjects, that there is a steady increase in SDG-related research, which for Panel A (in particular UoAs 1-3) climbs sharply as we get closer to 2020.  This can be explained by an increase in COVID-19 publications in the context of SDG3 ‘Good Health and Well-Being’.  The second insight is that when looking at Panel C, which has a focus on the social sciences, there is a notable peak in publications in 2013 which is the year for submission of publications to REF 2014, with a marked decline immediately thereafter. This potential REF 2014 effect seems very likely after we made a comparison with SDG tagged publications classified by UoAs across the same timeframe for Germany, seen in figure 4, which does not undergo the same research evaluation exercise, and reveals no marked spike and decline in publications categorised in UoAs related to social sciences between 2013 and 2014.  Panel D, which covers research in the arts and humanities and includes much more heterogeneous output types, make SDG and REF classifications less reliable as many outputs go beyond those having identifying features for classification purposes.

Figure 4: SDG-related research outputs in Germany, mapped onto the REF classification scheme in Dimensions

3: Can trends be found in SDG research that contributes not only to REF 2014 but across all years that have been part of research assessment?

When we look at year-on-year growth of SDG related research in each UoA, across both the REF 2014 submission period as well as across all RAE and REF time periods, we observe that the REF 2014 submission of research outputs has a higher content of SDG-related research than non-SDG research. Based on this, it would be our expectation to see REF 2021 contain even more SDG-related research, continuing the upward trend we see in previous assessment periods, where SDG research outpaces normal growth in research submitted to REF.

Conclusion

It has been argued that none of the SDGs can be achieved without the contribution of research carried out in higher education institutions.  Universities have the ability to generate, translate and disseminate knowledge relevant to achieving the SDGs. They also have the potential to increase societal impact through translational research.  Given this, using Dimensions to probe the vast corpus of research relating to both REF 2014 and SDGs and how the two entities interact will go some way to providing insight into the contribution of academic research to the United Nations Sustainable Development Agenda.

Footnotes

1: The SDG classification system is the most recent addition to classification systems in the Dimensions database and is not yet fully developed, not only in terms of the complexities involved in defining SDGs but also in difficulties with SDG language. It is rapidly evolving with more SDG-related research being classified on an ongoing basis

2: It is important to note that, although the SDGs were not implemented at the time of the two RAE exercises in 2001 and 2008, there was a previous global action plan, the Millenium Development Goals, which set out to reverse the poverty, hunger and disease affecting billions of people, therefore research associated with SDGs was already in evidence. It is likely that more funding was made available at this time to address SDGs