A modernised, more anonymous process of application for research funds is required, writes John Smith.
The system of allocating research funds through peer review is one of those "old world" structures in need of modernisation. As young, struggling academics we are constantly appalled at decisions which we know have not been made with full knowledge or in the best interests of scientific research.
Peer review is the scientific world's answer to the political world of "chums". It relies for its strength on the lack of scientific literacy of most of the rest of society, and in particular on this same deficiency within the government bureaucracy allocating these sums. If the system, now creaking with age, is fundamentally reformed, valuable and scarce research funds can be more fairly apportioned on the basis of individual and scientific merit.
Coffee breaks in academic departments can be very enlightening. Hearing indiscreet professors chatting about proposals they have been asked to referee and thecomments on their own proposalsprovides a unique insight into the peer review process which is not available through official documentation. The source used for this article has not been restricted to the discussions within a single department. The supporting data presented here has been gathered from two physics departments and an electrical engineering department of very different standing in the research assessment exercise and in different parts of the country.
The official theory of the peer review system is as follows. A grant application carries the name of the applicant. The application is sent to "prominent" members of the same research community to be assessed, graded and, sometimes, constructively commented upon for improvement. The applicant does not usually know who the referees of the application were. However, this theory, as stated and advocated by the funding councils, omits an important point; the size of most of these research communities is so small, and their interaction at the senior levels so common, that the culture is closer to that of a gentlemen's club. If a referee is performing, or intending to perform, research of a similar nature then the context of that referee's assessment is often one of "Push that idea along for our mutual benefit", or "Nice idea but the wrong way to do it. Think I'll write an application myself on the right way."
If the referee is pursuing completely different lines of research in the same field then it is commonly a balance between "What a load of rubbish. He's not following the conventional wisdom", and "That seems like a good idea. I can jump on that bandwagon as well if he's successful."
The assessment of the research is based more on a mixture of mutual backscratching and backstabbing than on objective scientific merit. After all, there is only a finite amount of pie to be sliced up and one would rather have it distributed among chums than outsiders - even if the outsiders do have the occasional good idea.
Reforming this old boys' club is straightforward. The following proposals were put together over morning coffee by some young academics. After half an hour we decided on the following structure and asked "why hasn't this been proposed before?" The answer to this is that no one has bothered. It is often easier to work within the system you have inherited than to alienate the old boys by saying that it needs changing.
However, change in this system is crucial to the development of younger academics since the reins are being held too tightly by the old boys in this shrinking and more narrowly defined fiscal climate.
The keys to objective distribution of research funds based more fairly on individual and scientific merit are heterogeneity of referees and anonymity of all in the process. Committees of the funding councils and the pool of referees should comprise an equal mixture of junior academics and senior academics so that ideas which may not follow the conventional wisdom (aka dogma) will receive a fairer assessment. The committees should also contain a mixture of members from industry and academia or a mixture including other interests in the humanities and social sciences. Membership of the committee should rotate but should be more rapid, and membership should not be based solely on prominence or eminence.
Where possible, anonymity of applicants and referees should be regularised through the use of standard forms and formats. Although the "case for support" can often reveal enough information to senior members of the research community to indicate who the applicant is, this can be discouraged through tighter guidelines and restrictions on what can be mentioned. Applicants who might dissent from this format can be discouraged by the knowledge that their case for support will be edited if felt to be in breach of the guidelines. Although ensuring the anonymity of the applicants from the referees can be the most difficult of these reforms to guarantee, it is one of the most important reforms required to prevent research communities becoming "closed shops".
For members of the committees, clear, unambiguous policy guidelines should be enforced so that personal viewpoints and individual interests and influences in the panel cannot be brought into the process. Perhaps this is best achieved if the committee is anonymous to each other. This would also serve to reduce the effects of personality factors.
Clear guidelines should also be issued to referees requiring that comments on proposals must be extensive and should fall within the context of the same policy guidelines as the relevant committee. Rejecting proposals with simplistic, one or two line rebuttals does not serve the profession well or improve research.
As the concept of reform is firmly on the agenda in higher education we should make a proper job of it and not neglect one area which, if not reviewed and overhauled now, will leave considerable damage which will only be more expensive (in monetary and human terms) to repair when the present younger academic generations eventually form their own club of chums in two or three decades.
The author is one of three academics who decided an article of this kind was needed. A single pseudonym has been chosen to protect these three academics now working in the same department, and the indiscreet professors and their departments who have provided some invaluable information.