Education: League tables - love them or hate them, they're here to stay

League tables make fascinating reading for parents and students but are deeply resented by many in the education business - and particularly by institutions which perform badly. We have had six years of official league tables in schools but nothing in universities unless you count the research assessment exercise. Only one league table exists - and it has come in for a good deal of flak.

Like it or not, people love lists - whether they are of football teams, unit trusts, washer-dryers or restaurants. They are easy to read and give digestible information quickly to stressed consumers. Newspaper league tables ranking universities on a range of indicators are now a fact of life in many countries. In France Le Nouvel Observateur does the job; in Germany Stern magazine; in Canada Macleans magazine; and in America US News and World Report, among others. For the past five years The Times has been irritating academics with its attempts to measure the performance of 96 universities.

The Times's education editor John O'Leary claims he's filling a gap in what has become a mass higher education market. "I get calls right through the year from students or their parents who find it difficult to choose between universities and find that the universities themselves - perhaps for understandable reasons - aren't producing much comparative information," he says.

"You can get as much information as you like on individual universities but often it's partial. We are able to put the data into context for people who don't know as much about higher education as our critics."

But league tables create fear because they carry potent messages about how one institution compares with another. They have the power to make or break schools and universities. In South-east Asia, for example, The Times's table is studied closely by local inhabitants and can affect efforts by institutions from the United Kingdom to recruit students.

There are signs, however, of a new acceptance of such tables in the higher education world - a sign perhaps that they're here to stay, so universities may as well make the best of a bad situation. The vice chancellors' committee, for example, is to hold a conference in the new year on the subject and is keen to see more publishers producing university league tables.

"In principle we're in favour of league tables which look at different aspects of university activity but we're against super league tables which pull together lots of different factors and give them subjective weightings," says Diana Warwick, chief executive of the Committee of Vice Chancellors and Principals. "Students have differing needs and aspirations, so we would like to see more league tables on the different aspects of university life."

Her comment is an implied criticism of The Times's league table, a poll of polls in which the results from eight indicators are converted into a points score and aggregated. Universities are rated on staffing levels, accommodation, degree classifications, graduate employment, research, teaching quality, entry requirements and library spending. Ever since it began in 1992, Cambridge University has been top, Oxford second and Imperial College London third.

New universities complain that the table is biased in favour of old universities because spending on old universities was always more generous until the elevation of the polytechnics. Other critics says its methodology is flawed, that it invests differences between institutions with a significance that is unwarranted, that it fails to acknowledge differences in the missions of the individual institutions and that the quality of the data is doubtful. Above all, they complain that it is subjective because the information you get out depends on the data you feed in. And, if you feed in material about research standing, spending on libraries and teaching quality, you're bound to have Oxbridge at the top.

Professor Manz Yorke, of the centre for higher education development at Liverpool John Moores University, has written the most authoritative critique of The Times's table in the journal Quality Assurance in Education. He says: "The thing is basically invalid in my view on the grounds that the methodology doesn't stand up. It's not methodologically sound and robust and doesn't inform in the way people think it does. It simply confirms reputation."

Another critic, Christine Hodgson, head of communications at the University of East London, has also produced a paper (as yet unpublished) that has been circulating in university circles. She complains that various items in the table, for example research and teaching quality, are given arbitrary weightings for which no justification is offered. Research scores are multiplied by 1.5 and teaching quality by 2.5.

"The process of aggregating the elements would probably produce different final rankings," she says. "In any case, higher education is far too complex a process to be represented by a single index."

In particular, she objects to entry requirements being drawn from the Ucas guide for university and college entrance. These are based on A-level scores, ignoring the fact that a proportion of students at new universities are admitted on other criteria. Moreover, the scores relate to A-level grades sought by universities rather than those achieved by students actually accepted.

New universities have a special beef about being rated for accommodation because they never set out to provide rooms for students. Located mainly in cities, they have seen themselves traditionally as local institutions catering for young people or mature students who live at home or have their own pads. Why should they be ranked against a university like St Andrews which provides comfortable rooms for all first-years and many others besides?

But universities such as Sussex also object to being rated for accommodation. It comes a mediocre 39th in the league table and says it supplies all the rooms that students demand. It also receives a relatively low score for staffing because it is more cost-efficient than many other universities, according to Sue Yates, Sussex's information officer. "The table claims an objectivity that it doesn't deliver," she says. "The indicators it is looking at and the weighting it ascribes reflect the reputation of universities, nothing more."

John O'Leary rejects some of the criticism, though not all. The accommodation category is being removed in next year's guide, he says, to reflect the objections. In fact, he is constantly changing the way he compiles the table in the light of complaints and as new data becomes available. He is also looking at the way entry requirements are measured.

Another item he is reviewing is the complaint that small differences in scores are translated to significant differences in the league table. But the weighting of research and teaching quality was restored this year on the grounds that both indicators were more important than others to students and staff. Research was given a weighting of 1.5 because that was the lowest multiple possible without getting into smaller decimal points and teaching quality was given 2.5 because that is the most important activity for undergraduates. Both indicators tend to favour the old universities.

"I accept that new universities are at a disadvantage but I would suggest that's because it takes more than five years for them to make up the advantages that the old universities have enjoyed from research funding and the superior facilities that they inherited," says Mr O'Leary. "Our league table is simply reflecting what is, not defending the way it's happened.

"We have chosen the indicators that we think prospective students who are looking at the national university system as opposed to their local university are taking into account and which academics value. Inevitably any compilation or aggregation of those figures is going to be to some extent arbitrary. But we're producing something that we think satisfies our readers and that's who it's for."

The Department for Education and Employment is not saying what it thinks about league tables for universities. Senior civil servants are thought to believe more league tables are inevitable. But Roger Brown, former chief executive of the Higher Education Quality Council who now works as an independent consultant, says the evidence is that students make choices on a whole range of factors and that quality of a given course comes well down the list. Other experts confirm that. The quality of the local football team or whether you have a friend at the institution seem to be as important as anything else.

That may change with the arrival of tuition fees. Tony Higgins, chief executive of Ucas, says: "There's going to be a greater concentration by parents and students on what they're getting for their money. If the Government continues to present league tables for schools, I see no reason why one day they won't turn round and say `Why don't we do it for higher education?' "

At the same time, like other higher education experts, Mr Higgins is opposed to rankings. There's no such thing as the best university or the second best university. Nor is there a division one or a division two. "The most important thing is what is the best university for a given student, what's the most comfortable environment and the sort of course they want to do. Some students might find the ambience of Oxford abhorrent. The best university for them might be Keele or the University of West of England."

David Roberts, chief executive of the Higher Education Information Services Trust, is another person who thinks parents are going to take more interest in quality when they start bankrolling their offspring through university. Parents of all social groups have become used to league tables in schools. They will expect them in higher education. If anything, they will take more interest in university league tables because they will be after value for money. Higher education is going to have to get used to them.