I had very little to do with REF preparation this time around, being more absorbed with the (very high, by the way) quality of education across our humanities units. But I’m in it today! Exeter English managed to increase its grade-point average in REF2014, as compared with RAE2008, and yet slip a full 25 places in the table on this measure. If we adjust that for levels of intensivity (i.e. the percentage of people submitted) we rise to seventeenth. If we use the Times Higher’s ‘research power’ measure, which is the best predictor of QR dividends, we rise to fourth; on the Research Fortnight power rating, we’re seventh. So we’ll do ok on the financial side. But still, what looks to all the world like a fall, feeds into most league tables as a fall, and feels for all the world like a fall, just might well be a fall. We made an awful fuss when we did better than expected in 2008; this result leaves us with different stories to tell.
So what lessons might we learn from this REF? Here are some of my initial thoughts.
- Grade inflation is the big story. And don’t we know it? My first view of our figures left me with a warm glow – but only until I saw the tables. There are six English departments with grade-point averages over 3.4, which is almost unbelievable. I certainly didn’t see this coming.
- Don’t do the twos. Don’t believe anything you hear to the contrary: two-star work is counter-productive; one-star is poison. We’ve been very tolerant of two-star work, along the lines that everyone needs to do some of that sort of thing some of the time. But the real winners from this REF had ruthlessly minimized their sub-threes. Warwick English had 95% of work at three-star and four-star. Enough said.
- Impact is a good news story for the humanities. We were worried about this, weren’t we? But across the country, the English panel graded 83% of impact case-studies at three-star or four-star. So did History. Classics came in even higher, at 88%. So is that settled, or have these grading practices created a credibility gap? I predict some lively debate about this.
- But did impact really have any impact on results? Actually, impact didn’t have the effect that some people expected. There are some interesting winners and losers on impact in the English list: Newcastle, Swansea, Kingston and Bedfordshire at the top (putting to bed, in the process, the hypothesis that impact would favour the metropolitan universities); Cambridge and Bristol quite a long way down. But I don’t think anyone could claim that it’s had a hugely distorting effect. And for us, impact wasn’t the problem: we had 100% of impact at three-star or four-star. In terms of commitment of resources as we move forward, might there even be a risk of over-egging this pudding?
- A youth-first policy has its risks. Some of our rivals have a reputation for buying fully-fledged stars, at cost. We’ve done the opposite: we’ve bulked up (only Oxford and Cambridge entered more staff than us), but done so almost exclusively from the bottom up. That has always seemed to me an ethical approach, since we’re developing young careers. To be honest, also, we haven’t had an awful lot of choice, since we have needed bums on office-chairs to deal with the bums on the lecture-theatre benches. Other places haven’t had quite the same pressures, and have grown in different ways. At Queen Mary, one of the big winners, roughly 38% members of academic staff in English are professors – a figure significantly higher than the equivalent at Exeter. So maybe it’s time to admit that our approach, for all the efficiency of our wage-bill and all the excitement of developing young researchers, has its limitations. I’m profoundly sorry about that.
- The jury’s still out on creative writing. Warwick has invested in creative writing, and comes out on top of the table. But with 85% intensivity, we can’t absolutely say that creative writing is now (against popular opinion within our discipline) a REF winner. Ditto UEA: bloody good result, but obtained at a mere 63% intensivity.
- The AHRC really is prepared to plough a different furrow. Some of the big losers in the doctoral training partnership competition (e.g. Warwick, Queen Mary – Swansea, even) have been big winners in the REF. And vice versa. This debate has legs.
- Bedfordshire: good on ’em. They will get some headlines, outperforming Cambridge. But their intensivity rate is respectable, at 82%; they just shine out on this table as a very strong, albeit very small unit.
- How big is beautiful? Setting aside some anomalies – Oxbridge at one end, Bedfordshire at the other end – a well-formed English department looks from these results like one with 30-45 people. A number of the bigger ones (e.g. King’s, Birmingham, Leeds, Sheffield) are notable for having been tough on exclusion. There may be some lessons there.
- What happened to double-weighting? It’s impossible to tell from the data released to date, but this is a very interesting question. The most likely explanation for our blow-out (against internal predictions) in two-star grades is that we were overly optimistic about double-weighting applications. If some of these were rejected, some weaker work would inevitably have been pulled into the process. There will be calls for clearer, and earlier, guidance on this next time.
- How useful was the internal monitoring? We all did it, didn’t we? The internal grading, the external ‘experts’, the ‘mock-REFs’. And the cost, in terms of time and money, was huge. Somewhere – let’s say a university somewhat east of Exeter – even paid me for my opinions. But an awful lot of questions will be asked, in an awful lot of departments, in retrospect. Some departments might even be asking external advisors for their money back.
There is, doubtless, much more to be said, but that’ll do for a start.