View From Here

League tables and dullards quibbling about trivia are not the best way to assess anything

UNIVERSITY LIFE gets more like Wonderland by the week, making me feel increasingly like a cross between Alice - dazed by the constantly changing weirdness of it all - and the White Rabbit, forever rushing from meeting to meeting in a state of perpetual lateness. In the spring, things start to degenerate, as wodges of paper arrive by the carload, demanding more or less instant responses while claiming to be consultation documents.

The tightness of the turn-around time is obviously conditioned by the fact that even bureaucrats dream of holidays, and they want everything knocked on the head before summer starts to fade. This leads to not infrequent mistakes, so I felt almost sympathetic when a document arrived from the Quality Assurance Agency asking and answering the question "What is TQA for?" - obviously written by someone in the organisation who didn't realise that they stopped talking about TQAs (Teaching Quality Assessments) last year in favour of a different acronym.

My sympathy drained away like Alice's pool of tears, however, when I knuckled down and read the document, entitled "Graded Profiles: Interpreting the Numbers". To save any reader the trouble of wading through it, I shall attempt an English translation. Basically, the QAA set up a system of evaluating teaching in universities whereby teams of assessors go round assessing how departments teach and how institutions support teaching (these days, you have to say "teaching and learning" as though by some miraculous discovery the QAA had only just cottoned on to the fact that these words go together).

Points are awarded for success in six categories, so that a score of fours across the board will result in a total of 24, the maximum possible. Until recently, any score over 21 was held to be "excellent", but then the goalposts changed so now we're up to 22, not too difficult as institutions learned the ropes and worked out how to get the paperwork right. League tables sprang into being, with universities listing high scores all over the place and selling this evidence of excellence to any overseas student willing to pay the inflated fees.

But it now transpires that the QAA doesn't like its own system. "Does a number tell the whole story?" it asks. Not necessarily, it answers, a point so obvious that a five-year-old could have made it. You can get a 22, for example, with five sets of top marks making 20 and a two, which is pretty grim. This means the QAA is suggesting that what we now need is a system for deciphering the overall grades. We are commended to use a conversion table for this purpose, and tables and graphs are available from the agency direct. The purpose of this daft document, therefore, is to remind us that journalists have been using "crude and simplistic methods" to construct league tables and that this won't do. But the QAA set up this numbers system in the first place, you cry! Exactly so, and now they're trying to set up another numbers system to decode the first one, and we're all supposed to comment on it.

When I was younger, I was quite clear that the blame for social ills lay with Them, the ones in charge. I used to believe that They were corrupt and devious. Now, having joined the Establishment, I subscribe to the Cock-up Theory instead: They are often just plain incompetent. If Nato can start a war and sends in bombers without seemingly pausing to think about the results (thousands of displaced people, starving refugees, civilian casualties, world-wide opprobrium), it shows that no matter how many degrees, knighthoods, premierships or medals you are given, you can still be a complete fool. I once read somewhere that criminals and politicians have no sense of cause and effect, so that they fail to understand that if you do one thing, something else will ensue. I think we could add quality assurance agents to that list now, judging by the quality of thinking in this document.

Assessing the quality of teaching is a very good thing in principle because it reminds universities that they are accountable. Teaching assessment exercises can be very valuable and I am all in favour in principle. But setting up league tables and sending in teams of dullards to quibble about trivia is not the best way to assess anything, and to propose now that we need to start decoding the daft numbers game that the QAA set up in the first place really takes the biscuit.

You can just see what will happen next if this idea takes hold: it will be the QAA's equivalent of painting roses red to please the Queen of Hearts and playing croquet with flamingos.

The writer is Pro-vice-chancellor of the University of Warwick