Doing research with impact

To talk about impact has become increasingly popular, particularly as “impact” is used to assess the effect of research on the economy and society by the Research Excellent Framework (REF) (the system for assessing the quality of research in UK higher education institutions), yet for some researchers there is still some uncertainty about what research impact is: When does research start to be impactful? How is research impact to be evidenced? How does impact relate to research output? In response to these various questions, I outline the difference between research with impact and so called impact case studies prepared for the REF.

Research with impact

Creating research with impact is what researchers all aim for and, in many circumstances, are actually doing. This means our research makes people look differently at the world and act differently in their daily life and work routines.

Research with impact enables you to explain changes that may have directly resulted from your research. Or, as the Higher Education Funding Council for England (HEFCE) puts it:

wider impact is an effect on, change to or benefit to the economy, society, culture, public policy or services, health, the environment, or quality of life, beyond academia

This impact can be generated at any point during the research cycle. It can be created at the start of a research process during data collection: Talking to and asking questions to interviewees can make people reflect and think differently about their working practices or more generally their lives.

But typically building up impact is the result of long-lasting efforts - months and years - getting to know people within organisations, communities, businesses or government bodies and maintaining trustful relationships where the external world is interested in the views of researchers and their research results. Then research can influence organisational practices as a result of the interactions that take place during the research process, as a result of direct intervention, dissemination or training.

Research very often has an impact as a result of the output we produce - a report, for example, that influences new policies or even governmental reforms. Traditional academic output can have wide reach in terms of how well it is recognised and received by the academic community - the UK is world-leading in terms of citation indexes and the impact factors of journals, with citations per article being higher in the UK than in all other countries.

The impact of our research on society however is more difficult to measure. It is not always possible to trace the precise impact of research on the behavioural change of an individual or the practices of an organisation. The impact may well be there, but it may be difficult to quantify or demonstrate in a convincing way.

The REF and impact cases

An impact case for the REF can only be prepared if there is a track record of underpinning research. The research needs to be finished or if ongoing long-lasting and documented in academic output. In many cases, impact cases are the product of a body of research over many years.

To develop an impact case, a compelling narrative needs to emerge that makes it clear what your research was about and how it influenced actors, organisations or institutions. There needs to be precise verification of how your body of research contributed to change in organisations, institutions or governmental policies.

This is where evidence comes into play. During the last REF, there needed to be at least 10 pieces of supporting evidence. Examples of supporting evidence could include:

• A letter by a CEO of an organisation stating how your research affected corporate policy.
• A copy of a government report where your work was acknowledged as having made a contribution, though this would need to be significant and the linkages between research output, ideas and actual change stemming from any government report would need to be transparent and unambiguous.
• The establishment of a new business association where the directors confirm that the organisation was heavily influenced by your work.
• A testimony of improvements in the way new products are designed.
• A testimony of the adoption of recommendations you made.
• The development of tools and guidance for businesses and organisations which are then used widely, training programmes you designed for managers and staff, improvement of communication plans and strategies of businesses and organisations, the list is extensive.

In many cases, demonstrating impact will be rightly challenging. While dissemination of research output to policy seminars or makers or the citation of research by government is indicative of good quality applied research, the dissemination in itself does not constitute evidence of impact. The evidence has to demonstrate that significant external change has taken place.

Summary

The main difference then between impactful research and impact case studies is that the REF sets certain requirements for turning ‘research with impact’ into an ‘impact case’. Research should alter behaviour in the public and/or private sector and this should be convincingly evidenced. The precise rules of what counts as evidence will be published with the next REF 2021 guidelines.

It is crucially important to recognise that not all ‘impactful’ research will meet the demanding criteria of a REF impact case study. Authors will need to be able to demonstrate, document and legitimately verify the impact and reach of their research, whether that has been on individuals, organisations or policies or broader society.