Higher Education: Who defines excellence?: Universities and academics are smarting over the first attempts to assess the quality of their teaching, reports Liz Heron

The Government's first attempt to assess the quality of teaching in universities started in earnest this month, and already pressure is mounting for the system to be scrapped or radically reformed.

Vice-chancellors and senior academics have attacked the logic, consistency, timescale, selection methods and assessors in the Higher Education Funding Council for England's teaching quality assessment exercise. One university claims that a visiting assessor tried to sell a department software and that another took away the department's report on its performance, saying it would be useful for drawing up his own department's assessment report.

The assessment aims to ensure accountability and value for money in the higher-education sector. Financial rewards and penalties for universities will be based on the council's ratings of teaching quality - excellent, satisfactory or poor.

Departments must rate their own performance, and the self-assessments are reviewed by two funding council assessors who may change the rating if they think it is wrong. Assessors visit any department suspected of having poor standards.

Twenty-nine departments were assessed in a trial run in the summer, and the results returned last month were hardly controversial. Six departments were rated excellent and the rest satisfactory. In this autumn's assessments, the first to use the council's full methodology, 179 out of 267 departments being assessed claimed excellence and 68 had their claims turned down. Nineteen have so far appealed against the council's decisions.

James Wright, vice-chancellor of Newcastle University, where three departments are appealing against assessors' decisions, has sent a circular letter to all vice-chancellors denouncing the council's procedures. The council can turn down a department's claim of excellence without assessors visiting it. 'The ability to write plausibly about your department's teaching and its excellence does not entail excellence in that teaching,' Mr Young said.

The funding council has already launched a review of its system. Alan Thomas, of the council's quality assurance division, said: 'We've put the review out to open tender and we expect a report on it late this year or early next year.' The Committee of Vice-Chancellors and Principals is so concerned about the costs and the administrative burden of the assessment that it has set up a working party to look at ways of streamlining the exercise and making it tie in better with the audit system run by the Higher Education Quality Council.

Assessors now spend two or three days in a department, before a report is compiled and published by the council.

At the University of Central England, Birmingham, the law department's self-assessment of excellence was changed to satisfactory without assessors visiting. The department's appeal against the decision has been turned down. David Warner, the pro vice-chancellor, said: 'This is absolutely ridiculous.' From 1989 to 1991 the department came either second or third in a national league table of Law Society finals results, and has matched or outdone its previous performance since the table was abolished. 'Given these sorts of results we find it difficult to accept the assessment of a group of people who don't even visit the place,' Professor Warner said.

At Exeter University, two departments claimed excellence and were rated satisfactory, and one, law, has appealed. Dr Bruce Coleman, until recently pro vice-chancellor in charge of quality assessment at the university, said: 'We were unimpressed by the nature of the visits. The people who came to our chemistry department had little experience of departments with major research activity and tended to take the view that if you were good at research you were bound to be bad at teaching.'

A survey on quality assessment in 64 universities compiled by the CVCP found similar concerns in a number of universities. Some senior academics were prepared to accept the assessment exercise if problems were ironed out. But for others, the methodological flaws - based on the question of how something as subjective as teaching quality can be measured - go to the heart of the exercise, and they want to see changes.

The Centre for Higher Education Studies at London University's Institute of Education has just completed a study of what good quality undergraduate education is, and how to assess it. Cari Loder, a lecturer at the centre, said: 'The study suggested that the only effective way to assess the quality of teaching was to ask the people being taught.' This will add weight to a campaign by the National Union of Students for students' views to be taken into account in assessments.

(Photograph omitted)