The more things change with the REF the more they stay the same

The next steps in the evolution of research assessment are upon us with the publication of the consultation for REF2021.

Quite rightly, there will be extensive conversations and much exchanging of views before formally responding to a consultation, which, given the significance of the proposals on which it is based, is necessarily world-beating in length. For any higher education policy veteran, this is a familiar pattern of activity.

However, that simple description of the task in hand is deceptive and underplays the importance of those conversations in guiding us to understand just what the  proposals for the next REF will mean for universities. The overarching question those tasked with drafting responses need to confront is whether, and to what extent, the latest REF proposals will deliver an evolution which is helpful in supporting and growing the excellence of the UK’s research base in universities across the UK.

The challenge for the funders will be to interpret lengthy responses and consider how and to what extent all universities can be treated fairly in future, regardless of the size or intensity of their research activity.

In terms of the future of research, this consultation is probably the most significant for at least a decade, and this will need to be kept in mind. At the request of ministers, the proposals for next REF are based on recommendations produced by the Stern review. The proposal to submit all research-active staff to the REF, and the consequential proposals on decoupling and the non-portability of outputs, have received substantial attention and in my view are the most significant.

‘All-in’

Research assessment in the UK has a long history of selectivity, enabling universities to submit their best efforts for consideration by expert peer reviewers. The proposed ‘all-in’ exercise will tip the scales away from institutions with research concentrated in a few areas. In this scenario, these institutions are unlikely to be able to fully demonstrate their excellence.

It is difficult to see how this proposal provides a basis to fairly assess research excellence across the whole sector, comprised of small and specialist institutions, modern universities, and institutions with differing levels of research intensity.

Just as important is the knock-on effect of the proposed definition of research-active staff. There is every prospect that institutions will conclude they have little choice but to review staff contracts and definitions of research activity.

This might undermine the career choices of some researchers, including early career researchers. The employment choices of institutions, while perfectly rational in terms of an ‘all-in’ REF, may well prove unhelpful to the long-term health and dynamism of the sector.

The proposal to decouple staff from outputs could well promote the work of a unit of assessment more broadly. However, any minimum requirement of outputs for each staff member submitted will need to be considered in terms of the potential impact on equality and diversity.

There is also a risk that, although all staff will be submitted, a small number of staff will be responsible for the outputs. This could be the case regardless of any minimum of outputs applied.

‘Non-portability’

The ‘non-portability’ of outputs raises other questions. Though this may reduce the likelihood of late-stage ‘transfers’ in the REF submission period, it could still create destabilisation after the assessments are announced if individuals choose to move on, or are targeted for recruitment based on known results. The transfer market would remain, but would take place at a different time. The complexity of managing cut-off dates, and what is credited where and when should give many pause for thought.

REF2014 was highly significant in its greater regard for equality and diversity, and in properly acknowledging the work of early career researchers. An ‘all-in’ submission is unlikely to provide for the same considerations. New approaches should not discriminate against those at the beginning of their career, those who work part-time, or those who take extended breaks from the workplace.

Assessing impact

Alongside those concerns about staff selection, I have mixed views about the proposals for assessing impact.

Maintaining consistency of approach with the previous REF is beneficial, particularly as institutions will have made decisions about identifying the impact of research based on REF2014 experiences. Broadening the definition of impact and how it can be demonstrated will also be beneficial, allowing institutions to highlight a wider range of achievements.

But I worry that these improvements will be dampened by other possible changes. The importance of impact has been highlighted consistently since it was first proposed to be part of the assessment exercise, and the Government has been clear about its ambition to promote an understanding of the value of research investment. Given this, wrapping the information obtained via the impact template into the environment template could be a backward step.

The importance of impact could be strengthened by increasing the weighting to at least 25 per cent, and potentially reducing outputs to 60 per cent (thereby maintaining the environment weighting at a high enough level to make it worthwhile retaining in the assessment).

Leaving the profile weightings as they are is a missed opportunity. Despite the next REF potentially looking very different in terms of process, with a large amount of extra work created by an ‘all-in’ submission approach, the outcomes in terms of ratings and funding will end up looking much like before. Universities, particularly modern universities, may have to engage with a hugely disruptive process that they cannot afford to avoid taking part in, but without much reward for their efforts.

Unlike in teaching, where the Government has embarked on policies that seek to disrupt the established orders and perceptions, the REF proposals may well simply reinforce past findings, patterns and behaviours.


Look out for tomorrow’s guest blog post, when Matthew Guest from Guild HE will argue that research is all about impact.