Report shows investing in academic excellence does lead to research impact
A report for HEFCE by Digital Science reveals the scholarly significance of research underpins its societal impact.
‘Publication patterns in research underpinning impact in REF 2014’ is the first report to analyse all research publications submitted for research assessment between 1988 and 2014. The study found that at least 42 per cent of the research cited in impact case studies in the 2014 Research Excellence Framework (REF2014) had been submitted as research outputs to the various prior research assessments. This suggests that research of scholarly significance leads to societal impact, but also emphasises the broad base of research from which impact stems.
The analysis covers almost 1 million submitted research outputs and more than 35,000 impact case study references across 25 years and five assessment exercises. The report considers the relationship between submitted outputs (as evidence of academic research quality) and impact case study references (as evidence of societal and economic impact). It explores what the data says about the selection of research outputs for assessment, and about changes in selection behaviour as a response to assessment.
Key findings include:
- Over time, journal articles have become a more commonly submitted output type (when compared with books, conference proceedings and other media). This is despite the fact that HEFCE analysis comparing quality profile by output type found that monographs were awarded the highest proportion of four star scores (Note 1).
- In early research assessment cycles, the research outputs submitted tended to be those published towards the end of the publication period. In Science and Engineering recent exercises saw submissions from earlier in the cycle in all relevant disciplines for all institutions.
- Of the impact case study references with digital object identifiers, 42 per cent were submitted to Research Assessment Exercises or the 2014 Research Excellence Framework (REF2014) as research outputs; these are evenly spread between publication years from 1996 to 2013.
- Earliest references are skewed towards recent publications for social science and humanities. In science and engineering, the date of the earliest reference in each impact case study is uniformly spread through the eligible publication period.
- The overlap between case study references and submitted outputs for assessment appears to be higher in more applied areas, and lower in the core humanities and pure sciences.
- For science and engineering, the case study references have highest overlap with the research outputs submitted in early cycles. The overlap between case study references and assessment outputs submitted to REF2014 is higher in social sciences.
Quote from Steven Hill, Head of research policy at HEFCE:
‘The innovative analysis in this report gives us, for the first time, evidence that research of all types leads to wider benefits for the economy, society and the environment. The findings emphasise the importance of supporting a diverse range of research to deliver impact.’
Quote from Dr Jonathan Adams, Chief Scientist & Head of Consultancy at Digital Science:
‘This huge body of work provides us with uniquely rich data on research activity, submitted outputs and case studies across 25 years and five assessment exercises. This is one of several perspectives that Digital Science has created on these data. The data will be available for the research base to mine in order to assess, uncover and understand the scholarly significance of the UK research base.’
About Digital Science
Digital Science is a technology company serving the needs of scientific and research communities at key points along the full cycle of research. It invests in and incubates research software companies that simplify the research cycle, making more time for discovery. The consulting group delivers data-driven analytical reports and technology solutions to research policy and management clients such as higher education institutions, charities and funding agencies, publishers and policy bodies. It is operated by global media company, the Holtzbrinck Publishing Group. Visit https://www.digital-science.com and follow @digitalsci on Twitter.
Note 1
