A multi-dimensional approach to assessing the impact of the UN’s Sustainable Development Goals (SDGs)

In this short, informal interview, Dr Briony Fane and Dr Juergen Wastl explain some of the methods behind their work on assessing how global research ties into the UN’s Sustainable Development Goals. Briony and Juergen have produced a number of key reports, whitepapers and blogs on the impact of research both in the context of SDGs and more widely; a list of their recent works can be found at the end of the article for further reading.

If you’re new to SDGs and want to get a quick idea of how to look at them through a number of different lenses, this is the introduction for you.

Quick links

What are the SDGs and FoRs?

Briony:

The United Nation’s Sustainable Development Goals, or SDGs, came into effect in January 2016. The 17 SDGs, are a call for action by all countries – developed and developing – in a global partnership. They recognise that ending poverty and other deprivations must go hand-in-hand with strategies that improve health and education, reduce inequality, and spur economic growth – all while tackling climate change and working to preserve our oceans and forests. And importantly, leaving no one behind. Every year, the UN Secretary General presents an annual SDG Progress Report, which is developed in cooperation with the UN System, and based on the global indicator framework, data produced by national statistical systems and information collected at the regional level. 

The Fields of Research (FoR) classification is a component of the 2020 Australian and New Zealand Standard Research Classification (ANZSRC) system, developed in 2008 and updated in 2020. It categorises all research and development (R&D) activity using a single system. The system is hierarchical, with major fields subdivided into minor fields (FoRs).

SDGs and research impact assessment

Juergen:

My overarching motivation is that I like to unearth interesting details about research, and how it connects to other things, and in particular those details and connections that aren’t necessarily obvious at first glance.

What does this mean in practice? Well, I usually like to start by thinking in two dimensions. For the first dimension we usually look at the different fields of research (FoR), but when we’re doing research assessment, having a simple uni-dimensional view like this can be somewhat limiting. So I like to open up a matrix / heatmap and spread whatever we look at into different corners to see what picture emerges, to see if hotspots emerge. Effectively doing a breadth-first rather than a depth-first search is a 2-dimensional data visualisation technique that represents the magnitude of individual values within a dataset as a colour.

At any point in time there is always a focus on a particular discipline or disciplines — nowadays an obvious one is computer science and AI — but there may be other characteristics to the particular pieces of research in that area. So by adding a second layer, a second dimension, you get a heatmap and can dive deeper into those areas that look interesting when seen through that perspective.

As an example, consider the field of engineering (FoR40). Engineering is very broad, so how do you narrow it down? You could start by choosing a particular branch of engineering, but in doing so you might miss some really interesting things. But if you look at it from the point of view of societal impact with SDGs, then you get a different, more interesting picture. We could, for example, then say: “Oh, we’ve looked at automotive engineering, what will it do in the future? Is e-fuels a thing? Is there already research? When did it start? Did it kick off in 2016 with the launch of the SDGs?” 

So SDGs are, for me, the second axis to open up another dimension in order to do a deep dive into existing research and to highlight it from a new angle. 

Briony:

To add my perspective, I would describe it as follows: using whichever SDGs are relevant, we can look at the associated research in a much more nuanced and detailed fashion. 

To me it is important we look at the sustainability side of research in ways that others might not.

For example, researchers might not look at the Global North – Global South divide — this is the high- and middle-income countries vs the low- and lower-middle-income countries, respectively — from the perspective of, let’s say, climate change, and putting the two together, like we did in our recent piece on Zoonotic diseases. So, whilst we’re not doing a meta analysis, what we do helps to see things from what you might call a meta viewpoint; it helps us to narrow it down in one axis whilst keeping it broad in others.

To me it’s important we look at the sustainability side of things in ways that others might not be.

Dr Briony Fane

Finding the needle in the haystack

Briony:

It’s a way of taking what would otherwise be a really big, intractable topic and giving us a particular angle to look at it from, or a particular filter or lens to look at it through.

It also enables us to showcase how much research has been labelled as being associated with one or more SDGs. The fact that we can identify and tag the research of academics helps showcase how much is being carried out that connects with the sustainability agenda. It’s a viewpoint we have because of the SDG coverage in Dimensions. Other databases don’t attach SDG labels to funding and policy documents, but Dimensions does. It enables this meta viewpoint which is really exciting, and allows us to look at the data in a more nuanced and thorough manner.

Juergen:

Yes, exactly! One of the things that we have done, particularly with our early SDG work, is to look at the potential citation advantage of SDG papers that underpin impact case studies, to determine whether there is an advantage or not. We also looked into how SDGs feature in a UK REF submission, as well as SDGs and impact case studies, using the narrative of a REF impact case study, classifying it externally via our classifier in Dimensions, and then carrying out different analyses. 

So it’s all about opening up new avenues, and looking at research from different angles. 

But we still have to be careful with SDGs. They are not as broadly available as FoRs are — FoRs capture 80% of publications in Dimensions. With SDGs that goes down to about 20-25%, because not all research is published in the context of an SDG. So in a way the SDGs are a good classification system, but not as broad as sometimes people think they might be. From a policy stakeholder point of view, and from a funder point of view it is a hot topic. So we like to look at any analysis that we have on FoRs and, where appropriate, map SDGs onto these.

The benefits for decision makers

Juergen: 

The flexibility of the classification systems in Dimensions and in this case, the SDG classification system, means we could in theory produce an infinite number of reports of course. However, I always try to focus on uncovering results that will be interesting and useful from a decision-making perspective.

In that sense, I’m particularly proud of the ways in which we link SDGs to Dimensions’ dashboards — for example, it took quite a lot of effort to include SDG-type functionality in the P&I dashboard.

It’s also worth pointing out that, unlike in many products and services where the dashboards that are provided only give you a visualisation of something without the ability to drill into the data behind it, with ours you can actually get right to the data. And however you slice and dice or filter your graphs, you can download the data for whatever is in that particular graph that you’re interested in. 

So that’s an important difference, and we’ve had feedback from our development partners that it’s really well received.

We’re providing the dashboards — the visualisations — to help leaders make sense of the data, whilst at the same time enabling them to go further with the underlying data if they want to.

Dr Juergen Wastl

Briony:

I’d echo that, and add that we are also driven and inspired by the conversations we have with people — whether it be colleagues here at Digital Science or the researchers or research administrators in academia and industry. For example, I get approached by colleagues wanting to know more about the topic. Digital Science is full of curious and interesting people, and I love that!

Conversations like this often lead to ideas for an analysis that might be relevant to a particular sector or industry, and if we can align these with our internal priorities, and find a good narrative then that’s even better. But even with this approach, for me the subject matter is as important as the underpinning motivation for the reports. It does not seem advantageous to just put an analysis on our website for the sake of it — there has to be a clear rationale for what we’re doing that ties the work to sustainable development goals or some broader related global challenge — and if that’s not there we will park it and focus on something else.

What lies ahead?

Briony:

Our early work was very UK-centric — related to the REF — but our recent reports, blogs and conference contributions have been much more globally relevant. I see this continuing; we’re now in a rather unique position (given the Dimensions data) to look at the global trends and impact of research. Through our work with the wider research community, we are keen to ensure this impact analysis is as available and accessible as possible, for instance through the dashboards Juergen mentioned.

Juergen:

Yes, as Briony says, in addition to the dashboards our initial SDG reports have already led to a number of notable publications with a global focus – most recently we presented jointly with the Prince Sultan University and the Times Higher Education an SDG analysis on all aspects on research in the context of the United Nations Sustainable Development Goals in the Developed and Developing World – with a particular focus on the Global South, which we have listed in the following section.

Our key findings

Below is a selection of our recent works covering SDGs, from their appearance in UK academic research assessments, to revealing disparities in efforts to combat the spread of zoonotic diseases amid climate change.

We are continuing to conduct further analysis, and will update this article with additional links as we publish our new findings. If you’d like to discuss potential collaboration opportunities, or to find out more about our use of Dimensions, we’d love to hear from you — please get in touch ✉