What is impact and how is it achieved in Business and Management?

The role of impact in the Research Excellence Framework (REF), with a focus on International Business.

Business School academics conduct research to increase knowledge, understanding, and impact in the real world. Impact, as defined in the Research Excellence Framework (REF) guidance, is “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.” 

The REF measures the quality of research conducted in universities and other higher education institutions in the UK against international standards of excellence. The role of impact in the REF has grown in importance, as evidenced by the increased weighting within the overall assessment (from 20% in REF 2014 to 25% in REF 2021). 

To help address the key issues of developing high-quality REF Impact Case Studies, a workshop organised by Leeds University Business School, Liverpool University Management School, and the Academy of International Business (UK & Ireland Chapter) took place on 1 December 2022. The workshop examined key issues and challenges in developing high-quality REF Impact Case Studies. It was held in a hybrid format with just over 100 attendees. 

Outline of the workshop 

The workshop was organised by Yingqi (Annie) Wei, Professor of International Business, Leeds University Business School; Nikolaos Papageorgiadis, Professor of International Business, University of Liverpool Management School and Frank McDonald, Professor of International Business, Leeds University Business School. 

It began with a panel of experts (listed below) who were members of the Business and Management REF panels. This was followed by an analysis of Impact Case Studies focussed on International Business and International Management that were submitted to the Business and Management panel in REF 2014 and REF 2021, and a review of the key characteristics of Impact Case Studies in Business and Management submitted to REF 2021 from universities that received overall high Grade Point Average (GPA) scores for Impact.  

The panel discussion and the presentations formed the basis for discussions on key issues to enhance the development of high-quality Impact Case Studies. The workshop concluded with presentations on practical issues in preparing high quality Impact Case Studies. See the full programme here.  

Panel discussion       

Panel Chair: Julia Bennell, Executive Dean, Leeds University Business School  
Panel Members:  

• Robert Blackburn, Professor of Entrepreneurship, Liverpool University School of Management, Member of Sub-panel 17 REF 2014 and Chair of Sub-panel 17 REF 2021 
• Nigel Driffield, Professor of International Business, Warwick Business School, Deputy Pro Vice Chancellor for Regional Engagement, Member of Sub-panel 17 REF 2014 and REF 2021     
• Terrance Warren, Business Consultant, Assessor for Impact Cases for Sub-panel 17 REF 2014 and REF 2021  

The panel members highlighted how the REF 2021 assessment worked, with particular reference to assessing Impact Case Studies (see slides 1 and slides 2). The key issues identified for high-ranking of Impact Case Studies were: 

• providing clear pathways from research outputs to users that extended beyond dissemination to meaningful impact,  

• and the provision of clear and identifiable evidence of significant Impact.  

Analysis of Impact Case Studies 

The first presentation (see slides 3) on Impact Case Studies focussed on International Business/International Management submitted to REF 2014 and REF 2021, reported on the major features of Impact Case Studies regarding author demographics (e.g. gender, single or team of authors, academic roles etc.), key features of the submitted case studies, and how the characteristics of Impact Case Studies had evolved between REF 2014 and REF 2021.  

The second presentation (see slides 4) highlighted the variety of approaches and topics covered in Impact Case Studies in the Business and Management Unit of Assessment submitted to REF 2021 by universities with overall high GPA scores in Impact in Business and Management. The presentation highlighted that a variety of themes and issues involving a wide diversity of private and public sector users at local, regional, national and international levels, are compatible with highly ranked Impact Case Studies.  

Discussion topics  

Workshop attendees worked in small groups to discuss three key issues highlighted in the panel discussion and from the presentations of the analysis of Impact Cases Studies:  

1. How can we present evidence that is transparent and verifiable with clear pathways from published research outputs to impact? 

2. Should there be citations of a variety of research output publications in Impact Case Studies, and should there be greater use of 4 and 4* journal outputs? 

3. How can we build research impact culture across the disciplines within the Business and Management Unit of Assessment and across other major areas of academic study? 

A variety of views emerged on these issues:

First, the collection of clear records with verifiable evidence is advisable on user involvement in research design, evaluation of results, and the development of recommendations from research. Record keeping is necessary on what, how, where, and when impact emerged from research. There should be transparent and verifiable statements by users of the value of impact. Guidance on the practicalities of these issues is available in the presentations on preparing high-quality Impact Case Studies (see slides 6).  

Some contributors cautioned against following a rigid format in the construction of Impact Case Studies as this could lead to a ‘box ticking’ exercise that did not lead to a convincing case of meaningful impact. The key is the construction of coherent, well-documented evidence that shows good-quality research was applied to important issues and that users confirmed that the application of the research made a valuable contribution.  

Second, a variety of research output publications were considered to be valuable but there appeared to be few references to publications in CABS 4 and 4* journals listed in Academic Journal Guide 2021 in Impact Case Studies submitted to REF focussed on International Business/International Management, and also probably in disciplines that are located within the Business and Management Unit of Assessment. This led to a debate about the role of 4 and 4* publications in Impact Case Studies.  

There was concern that there may be a trade-off between research leading to high-quality Impact and producing 4 and 4* publications. Some participants thought this was not a problem but others worried that an over-focus on Impact might reduce the flow of high-quality publications and this could be a problem, especially for early career researchers.  

A satisfactory solution would be for publications in 4 and 4* journals to be more focused on application to users and for research design for papers to be more connected to the interests of users. This however presents challenges as journals have their own agendas on what they consider as important. Moreover, REF-type Impact is a UK issue and many of the top journals are international and not based in the UK, and therefore likely to have limited interest in this type of Impact.  

The discussion highlighted that the growing importance of issues such as sustainability, resilience of global supply chains to shocks, and substantive changes to social, economic, political, ethical and technological milieu may enhance the willingness of top-level journals to publish Business/Management papers that are relevant to research users. It became clear in the discussions that linking high-quality research outputs in top-quality journals to REF type Impact was a challenging task that needed to be subject to more debate involving the key gatekeepers, e.g. editors, reviewers, publishers etc, of such journals.  

Third, although most people saw benefit from greater interdisciplinary research across Business and Management and with other academic areas including Science and the Arts, the means of doing this was not clear. One area thought to be of value was developing awareness of the major differences and similarities in what was regarded as Impact and good practice for Impact Case Studies across all the academic areas of research and especially among the various disciplines in Business and Management, as well as the Social Sciences. 

Presentations of practical issues of developing Impact Case Studies 

The workshop concluded with two presentations on the key issues in developing Impact Case Studies. The first by Krsto Pandza, Director of Research Impact at Leeds University Business School, focused on designing and executing projects that may lead to Impact Case Studies with commercial companies (see slides 5).  

Nikolaos Papageorgiadis, the author of an Impact Case Study, used his own Impact Case Study on an International Business topic submitted to REF 2021 to highlight the process of constructing an Impact Case Study.    

The workshop was a useful opportunity for participants to reflect on and learn from the characteristics of Impact Case Studies submitted to past REF exercises. It provided some useful indications to help advance thinking on how to develop high-quality Impact Case Studies for future REF assessment within Business and Management disciplines. 

If you would like to get in touch regarding any of the above content, please contact research.lubs@leeds.ac.uk. 

Presentation slides 

• Slides 1: Professor Robert Blackburn - Overview of the REF process and focus on impact  

• Slides 2: Terrance Warren - Leeds Impact Panel Discussion 

• Slides 3: Ms Malika Ben Kahla, Professor Nikolas Papageorgiadis and Professor Frank McDonald - The Key Characteristics and Evolution of Impact Case Studies Across two REF Periods: the case of International Business / International Management 

• Slides 4 - Kathryn Watson - The Analysis of High Scoring Impact Case Studies REF2021 UoA17 – Business and Management 

• Slides 5 - Professor Krsto Pandza - Engaged research with companies 

• Slides 6 - Professor Nikolas Papageorgiadis – The three stages of identifying, developing and achieving impact