Hard-pressed to fit the pieces in quality jigshaw

Is the Quality Assurance Agency in too much of a hurry to implement its framework and methodology of quality assurance?

As one much involved with the agency over the past two years, I sympathise with and strongly support its approach in the wake of the Dearing report. At its core is academic review, which is intended to replace Teaching Quality Assessment at subject level. The basic difference is that whereas the TQA began from an institutional self-assessment, academic review will start with subject-by-subject benchmark statements, now being developed.

The intention is to combine several judgements on quality of provision with a single summative judgement on the standards set by each degree programme or group of programmes. The agency is concerned to reach agreement with funding councils about the quality judgements.

At stake is which aspects of TQA will be stressed and what will be the grading of provision. One vice-chancellor member of the QAA board has said that numerical judgements on quality as used in TQA are unacceptable.

There is much wider unease. The critical questions are: what aspects of quality will be judged? how will judgements be phrased? how will they be graded? Will the grading be essentially descriptive and developmental or will it be something susceptible to translation into crude numerical terms?

The agency's answers, when they come, are ones that the sector will need to absorb, ponder, debate, and finally, one hopes, own, if the new framework and methodology is to be effective.

The problem is that the work we have done over the past months has not taken us far towards an overall scheme of academic review. There is a fairly simple reason for this. Trials on the basis of three benchmark statements have been truncated. They have considered standards in isolation from quality and outcomes in isolation from process.

It is possible that a conceptual error at the heart of the agency's proceedings has thus drawn out the genesis of academic review in a way that reveals an almost impossible timetable.

The agency has not yet developed a model for the integration of quality and standards. Where, meanwhile, are we with benchmarking? The agency's plan is that the review of 21 subjects will begin on the basis of a new methodology next autumn. In one sense, benchmarking is clearly going very well. Three statements are in the public domain. From all one can gather they are well accepted by their subject communities. Nineteen other benchmarking groups are at work.

Benchmarking is seen in terms of partnership. Thus, some 250 to 300 academics are giving their time, free apart from the payment of expenses, to draw up the subject statements. But the chairs of the groups have not yet met, though a meeting is planned by the agency in October.

Apart from informal contacts, there is no sense of the groups working cohesively. The first groups are the agency's best potential allies. The history group, having won the interest and support of professional colleagues, is anxious to see the review scheme develop constructively.

It is unfortunate that this proposed partnership between the agency and the sector to establish academic standards is relatively undeveloped, for it is crucial to the success of the whole enterprise.

What the sector is looking for is a scheme that, while it fulfils the requirements of accountability, is true to the thrust and spirit of the Dearing report. Dearing encapsulated a notion of setting, monitoring and maintaining national subject standards in conjunction with practising academics.

The Dearing committee took it as read that quality is the servant of, and should galvanise, standards. Both managers and lecturers will want to be convinced that benchmarking creates the conditions for an academic review that properly integrates assessment of quality and standards, is intellectually coherent and purposive and serves the needs of enhancement besides satisfying ministers in Whitehall.

Many loose ends remain. Take the complex role of external examiners. In the new year the agency will issue a crucial code of practice. Externals and reviewers will work alongside each other. Yet the timetable shows reviews will begin in Scotland and Wales before institutions have had time to adapt and the code is mandatory. This is one reason why there are doubts about the pace.

The new framework is a jigsaw whose pieces do not yet fit in a logical and convincing manner. The danger is of an over-hasty and botched implementation.

Anthony Fletcher is professor of history at the University of Essex, chair of the History Benchmarking Group and an auditor with the Quality Assurance Agency.