The TEF: measuring excellence or adequacy?

On the whole, I’m a TEF-enthusiast. I think there’s good and bad teaching, and I think we should be celebrating the fact that in the UK there’s an awful lot more of the former than the latter, despite some public perceptions to the contrary. In the process, we should be exposing the exceptions. I also tend towards the view that it’s not beyond the wit of academics to create a credible basket of metrics that would serve the purpose.

In principle, I also agree with the current government that in order to have any clout, TEF results need to have financial consequences. But it’s the practice of this that might become problematic. The proposal, to the extent that it’s been articulated, is that ‘success’ in the TEF would enable a university to increase fees in line with inflation. That would send a strong message about the value of teaching, and in the process would solve the political problem of a fee structure that nobody thought to future-proof.

But might there be a problem all the same? What I want to do here is tease out some of the potential implications for the TEF of the financial consequences to which it appears that it will be yoked. These will be very different in nature to those we have come to accept from the Research Excellence Framework, and they will shape the impact and value of the TEF. Although there’s still plenty of time to address the possible pitfalls, I’m not yet convinced that this will be in the best interests of the TEF itself, to the extent that we want it to promote excellence.

Some of us may be unfortunate enough to remember the summer of 2015 as the summer of fevered REF noise. (Whereas I, perhaps even more unfortunately, will forever have four neat scars on my foot to remember it by – but that’s another story) Anyhow, at a moment when we should all be taking a break, here are my three TEF-questions (which, on reflection, are arguably all versions of the same question).

  1. Will it measure excellence or determine adequacy?

How important will the ‘E’ be in ‘TEF’? With the REF, we’ve become accustomed to finely-grained distinctions in quality assessments. We’ve maybe even got used to the fact that there are at least six ways to end up on top.

But the TEF now looks like it’s being set up – simply thinking here about the nature of the incentive – as a pass/fail test. If a university is ‘excellent’, it will be allowed to increase fees; if it’s not, it won’t. Maybe there will be finer judgements made. Maybe there will even be league tables: we’re pretty good at them in the UK, after all. In practice, though, I fear the model may lack incentives for a university to really commit itself to excellence.

So one question for me is whether the TEF can still be constructed so as to recognize and reward excellence. I can see the political logic of backing away from this goal, and we can all appreciate the challenges of distinguishing, say, between two-star and three-star teaching. But it seems to me, nonetheless, rather a noble goal.

  1. How excellent is excellent?

Any test requires a pass-mark. Some of the initial rhetoric surrounding the TEF suggests that the pass-mark – the level required by a university in order to be allowed to increase fees – might be quite high. This would concentrate minds and shift priorities across the sector; however, is it really very likely?

For, one way of interpreting the announced link between the TEF and fees is that the government wants to get more money into higher education, and is looking for a way of justifying doing so. Following this line of interpretation, one might assume that most universities are rather expected to pass. Surely the political consequences of, say, several Russell Group universities failing the test, would be more than the current government would want to tackle.

Hence my concern, again, that the ‘E’ might gravitate towards an ‘A’: maybe we’re heading towards a Teaching Adequacy Framework.

  1. How deep will it drill?

The beauty of the NSS is that it gives us finely-grained data at discipline level. I’m never too fussed about the difference between, say, an overall satisfaction rate of 90 or 94, despite the fact that this might mean twenty places on a league table. But every year, without fail, the NSS focuses my attention, in credible and specific ways. It did so when I was responsible for educational quality across multiple departments, and was sensitive to outliers; and it does so as head of department, when I’m looking for areas of weakness to target in the coming year.

But will the TEF give us discipline-level results? It seems hard to imagine a model that would allow one discipline within a university to increase fees, while the discipline down the corridor is denied this flexibility. That may achieve a feeling of justice from the outside; however, there’s not much sense in it as far as universities are concerned. But if this doesn’t happen, will pockets of under-performance be exposed at all? It’s very unusual to get a crap university, but much more common to find crap programmes within an adequate university. A successful TEF would identify them.

So, once again, I can see a tension here: between the need to construct a system that produces the neat pass-fail institution-level results, and the desire to create a model that genuinely recognizes different levels of performance.

Conclusion

I think a TEF can be done, but I’m not convinced it can genuinely drive forward excellence and drive out weaknesses if the financial incentives are established in the way that as been foreshadowed. The drive to promote excellence is just not wholly compatible with a mechanism to recognize adequacy. It will help, because it will provide a lever to increase fees, but I wonder whether we may look back on a missed opportunity.

Or maybe there’s still time to address these concerns.

Advertisements

3 thoughts on “The TEF: measuring excellence or adequacy?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s