Being an examiner - part 1.

One of the nice things about not being the P4 examiner is that thousands of scripts no longer invade my kitchen table twice a year. It’s easy to tell they are P4 scripts - blood and tear stained they lie begging for mercy. However, there are still the many hundreds of university scripts to take my mind off the World Cup and the nagging feeling that England was robbed. Tooled up with marking schemes, several coffees and a large number of red pens its time to start the test mark.
I normally double and sometimes triple mark the first 10 scripts making sure that I have brought the exam paper and marking scheme fully back to memory. I need to make sure that I am back into the logic of the paper. My style, though examiners differ, is to set papers that have steep mark gradients. What that means is that there will be a number of elements in each question where the candidate can harvest a good number of marks cheaply. My target, working with a 50% pass mark, is to get the reasonably well prepared student to 25% or 30% before piling on the pressure. Other elements of the question are differentiators -that is sections of each question designed to help me differentiate between the levels of student achievement. I may also set a question which is generally easier than the rest again with the objective of settling the reasonably well prepared candidate into the paper.
The advantage of my style is that it allows me to differentiate between candidates of different abilities - to 'stretch the distribution' to use the jargon. The disadvantage is that only the very best candidates manage to complete a question at all levels it has been set. This makes them very anxious post exam but later very surprised when they pass. It’s a common refrain of my students – I hated the paper but how did I pass? The answer is – that’s the skill of examining. In my view it is pointless setting papers where the average student could gain 100%. In my view the best papers are ones which allow the normally well prepared student of average ability to pass comfortably but to really make the brighter students work hard for their high marks. A good exam paper should also test thinking ability as well as rote learning.
Now is the moment of opening the first script - candidate 007's work is exposed to the savage light of day as I begin my scrutiny. I normally mark my trial set as a whole rather than question by question. Indeed I prefer to examine all scripts this way but sometimes when under time pressure I may take the easier option. Reading and marking the whole script allows me to get a sense of the candidate's level more clearly and this can be important especially when a script is on a margin - a potential 39, 49 or indeed 69. Like most examiners I hate to leave a script on a nine but will spend considerable time working backwards and forwards through the paper looking for the odd mark which might make the difference.
At the end of the trial mark I now know what I am looking for again. It may have been some months (indeed 18 months with the ACCA) since the paper was originally set. Getting it all into memory is a long job but once done speeds the process considerably. I would normally expect to assess 4 - 5 good scripts an hour, poor scripts somewhat less but I find I can just mark about 30 a day before giving up.
Marking under the UK system is a protected process in the sense that if done properly the candidate cannot appeal against academic judgement. They can appeal on the grounds of administrative or procedural error but not otherwise. The price we pay is that papers are double marked and sometimes reviewed by an external examiner. With professional examinations the examiner would normally review a substantial sample of scripts from the marking team. An adjudicator may well sample review the quality of the marking process as well. I much prefer this to the system in other countries where scripts are single marked but are subject to appeal.
As I bury myself in a script there is a real sense of engagement with the candidate - sometimes answers are genuinely thought provoking, sometimes they improve on a model answer, often though they are wrong. Being wrong is not a failing offence - giving up is a failing offence. Sometimes a question does not work out well but as an examiner our job is to analyse the candidate’s thought processes. On innumerable occasions I find myself working out an answer using the candidate's mistakes to see how they have followed through.
Then come the essay questions. Marking these is where every ounce of judgement comes into play. No two essays will be the same and with a good essay question there may be many multiples of answers. I adopt a tick or check mark system where I put ticks in the essay at the point I believe a new mark has accumulated. Then at the end I make a summative judgement marking up or down depending on my assessment of the essays quality as opposed to its content. Other examiners like to read through, sit back reflecting on the key points and make an overall judgement. However it is done it means hard thought - serious brain time - and that is why most examiners like heavily quantitative questions where the answer can be seen and ticked off quickly. But essays do give us an opportunity to assess higher order skills.
Then the marks are in, the distribution assessed and the pass rate reviewed. In a well prepared academic examination I would normally expect about 5% outright fails. About 5 - 7% will be in the distinction or first class band. The bulk will be around the mean which with a 50% pass mark should be about 55 - 58%. Professional students by and large rarely perform at that level even with equivalent papers. There are many and complex reasons for this that I will not delve into here. Different professional bodies have different procedures for handling low pass rates - some shrug them off as a necessary price for quality others do other things but however it is done once the pass list is produced our job as examiners is nearly over. Inevitably there are the queries and appeals to resolve and sadly the sorting out of the examination offences. In another blog I will let you know the secret of detecting plagiarism.