Test answer is anybody's guess

The problem facing Dr McQue and his multiple-choice examinations is a common one (Teaching, THES, February 4).

Certainly, the reliability of the examination would be improved by increasing the number of options (branches) in each question. Even an increase from four to five would be helpful. But increasing to six or seven is likely to be difficult.

In my experience, question-writers tend to write the question stem first and the branches second. This makes the items more difficult to write and tends to produce tests of factual recall rather than ones of interpretation and application. Dr McQue might like to try writing a series of branches on a topic (for example, "antibiotics") and then construct a question stem based on one of them (that is, the one that is the right answer). This would be more likely to produce high-level items such as the following: "A patient complains of (symptoms) and on examination you find (clinical signs). Which of the following antibiotics would you select to commence treatment with?a) aaaa; b) bbbb; c) cccc; d) dddd; e) eeee, rather than a test of factual recall such as: "Which antibiotic is the first-line treatment for (infection)?' a) aaaa; b) bbbb; c) cccc; d) dddd; e) eeee.

Dr McQue could also increase reliability by increasing the length of his examinations. He might also consider introducing "extended-matching items" that are a modern development of multiple-choice questions (MCQs) allowing a much longer list of options (stems) to be presented, yet still retaining the advantages of being optically markable.

But I would urge Dr McQue to reverse his decision to attempt to minimise guessing by deducting marks for wrong answers. The fact that candidates can gain marks through partial knowledge and blind guessing is a permanent flaw with MCQs and the history of test development is littered with attempts to overcome it through negative marking (more correctly called penalty scoring). The only thing they have in common is that none works.

The best we can do is to contain the damage through good item-writing, increasing the length of the examination and increasing the number of question branches.

Penalty scoring makes things worse because it introduces a new, uncontrollable variable into the exam - that is, each individual student's willingness to attempt an answer when they are in some doubt about whether it is correct. Indeed, in the hands of test-wise candidates the outcome may be exactly the reverse of what the examiner intended because, even with penalty scoring, "guessing pays".

Gareth Holsgrove Cambridge Medical Education Consultants