Post-its, tick. Coffee, tick.

Series one
Bonus: Episode six | 1 July 2019

Chief Examiner Gill leads a team of principal examiners writing exam papers for AQA, and she plays a key role in ensuring that the mark scheme is applied fairly across the board. Join her for a day behind the scenes at the AQA offices, and see what goes on in an awarding meeting.

Featured in this podcast

Gill – a chief examiner with AQA

Transcript

I'm a chief examiner with AQA, and I’m going to take you through a day of awarding, where I’m going to be chairing the meeting, and we are going to be looking at the grade boundaries of A and E grade for A-Level. As a chief examiner, I look after a team of principal examiners who write exam papers for AQA. I also write two exam papers each year for AQA.

During the exam marking period, I attend standardisation meetings for all the papers and I make sure that the mark scheme is going to be applied fairly throughout.

I think one of the key reasons for people getting involved in examining is to get a better understanding of how the examining system works, and also to get a better knowledge of their subject and how the examining process is actually carried out. I think that other people will get involved in examining just because they have a real passion for the subject and they want to be in a role where they are encouraging young people to take up their subject, and they want to be able to influence that.

I started as an assistant examiner in 1985, largely, because I was on maternity leave with my first son and I was quite bored, so I thought I would find something else to do. I’ve always really loved statistics and so I decided to get involved in that particular subject, and eventually I became a principal examiner, then a chief examiner.

It’s 9:30 in the morning and I’m just meeting up with a couple of colleagues, and we’re going to walk to the AQA offices and we’re going to sign in, ready for a day of awarding.

Unfortunately, there’s a long queue at the reception desk, as usual. It’s nice to see a few other faces that I haven’t seen for a year, so I’m just going to go over and have a chat while we wait to be allowed in. Just finding out where everybody’s going to go on holiday this year after awards are over, and also, who’s had an operation this year, and why, and is anybody moving, anybody retiring.

Well, having got to the awarding room, the most important thing is to locate the coffee machine and grab a cup of coffee and decide where everybody’s going to sit. I am going to check the paperwork with the AQA representative and then I’m going to start to welcome the committee to this awarding meeting.

There are about 15 people in this room at the moment. There are the principal examiners, who are ready with their reports to go through how their exams went. There are also the experts from AQA, who are going to take us through some procedures and technicalities and, the most important thing, the statistics involved in the exam results. And also, people with huge piles of exam scripts, all around the room, who are going to locate exam scripts with the correct marks so that we can have a look through a range of candidates’ exam papers when we’re deciding where to put the grade boundaries.

I’ve now explained the importance of the awarding meeting and the fact that we must consider every candidate, and our decisions are very important for each and every one of them. And I’m now going to ask the principal examiner, for the first paper that we’re going to be looking at, to go through his principal examiner’s report.

Everybody is now listening intently to the principal examiner and checking through the mark scheme themselves, and making some notes, and now we’re just waiting for the AQA support officer to go through the statistical recommendations for grade boundaries.

I’m now just waiting for the AQA support staff to go and have a look through their big packets of exam papers so that they can find us all some scripts around the grade boundaries for us to look at. The first grade boundary we’re going to be looking at is the grade E boundary. We’re also all waiting for the archive scripts so we can have a look at scripts around the E grade boundary for last year.

My teams are all getting busy now, looking at exam scripts, and they are all scrabbling around on the desk to try and find Post-Its so they can scribble their initials on a Post-It and put it on the candidates’ scripts that they’ve looked at. And they’re also desperate for a pencil so that they can record the quality of the scripts that they’ve been looking at. The reason for using the Post-Its is to make sure that principal examiners don’t look at one script more than once.

Everybody’s now very busy in the room, looking at the principal examiner’s report and the mark scheme, and looking at the candidate’s paper in front of them and deciding whether they think that that particular paper qualifies for the grade that it’s at the recommended statistical boundary for, or whether it doesn’t quite make the grade.

As usual, for example, I can hear people at one end of the table bartering and asking people at the other end of the table whether they happen to have a script at 29 because they’ve only seen two of them and they want to see more. And then the people at the other end of the table are now saying, yes, you can have one of my 29s as long as I can have one of your 33s.

I’m looking through some scripts now myself over the range of marks that we have available and I’m having a careful look through each paper and comparing it with the important key points made by the principal examiner in his report, and also, I’m taking a look at the mark scheme and seeing where the candidate actually gained their marks.

Now I’ve had a good look through the script, I can make a decision about whether I believe that this is well within the statistical recommended grade boundary for this qualification, or whether I think it doesn’t quite make the grade. As I’ve been doing this role in awarding for very many years, I’m pretty good at identifying whether a candidate meets the E grade criterion or not.

I’m a little bit uncertain on the candidate I’ve got in front of me now and so I’m going to just take a look at the archive paper from last year and compare the quality of the responses on that paper with the quality of the candidate’s responses this year.

I’m very happy that I’ve looked at sufficient scripts and, looking around the room, everybody’s starting to chat, so I think everybody else has seen enough scripts and we’re ready to move on to completing the tick chart.

Everybody always looks forward to the tick chart, because we now have an AQA officer coming along with a flipchart and a marker pen, that may or may not work, and they are now going to hand-draw on the flipchart a chart where we’re going to record each mark that we’ve been looking at. And we’re going to record each person in the room, and go around in a circle - and this is where the person going first is always a bit nervous - and they’re going to record either a tick, if the person believed that at that mark the candidate qualified for that particular grade boundary; a cross, if they believed that the candidate at that mark did not qualify for that grade boundary; and a question mark if there was a level of uncertainty that couldn’t be resolved about that particular mark.

I can see a nice tick chart full up with two rows of ticks at the top and two rows of crosses at the bottom, and I’m going to ask for a line to be drawn under the two rows of ticks at the top and above the two rows of crosses at the bottom. And in the middle, there’s a mixture of ticks, question marks and crosses at the two marks that are left available. One of them is the principal examiner’s recommendation and one of them is the statistical recommended grade boundary. And we are now going to collectively come to a decision about which mark we should choose for the E boundary.

Well, today the principal examiner is very determined that the mark he chose should be the eventual selected mark, but the statistically recommended grade boundary seems to be perfectly acceptable to me. And as Chair, I see no reason to move away from that one as that’s based on a great deal of information on the candidates’ backgrounds and the way they would be expected to perform in a paper of this type.

I do seem to have developed skills that I never knew I’d have to develop in chairing awarding meetings. There are lots of pretty strong characters here with very determined views, and I’ve had to learn to be able to mediate between those and also be quite firm on occasions to make sure that candidates are all awarded the grade that they deserve.

So I’m now telling the whole awarding team my decision that the statistical recommended grade boundary is going to be the grade boundary that we definitely go with this year.

I can see everybody now piling up the papers and making sure that the archive scripts are collected separately, and the AQA helpers are coming round collecting up papers, and everybody is anxiously standing up, going off to get a cup of coffee before they come back and look at the A grade boundary.

Everybody seems to be back and ready to work again now, so I’m going to now ask for the AQA staff to get candidate scripts for everybody on the table around the statistically recommended grade boundary at A grade. I’m also reminding both myself and the rest of the awarding committee that we’re now looking for that top quality work to decide where our A grade boundary’s going to be.

I’m just going to ask the principal examiner to run through the mark scheme and just remind everybody of the questions on the paper where we would only expect those top A grade candidates to achieve the marks. That way, the whole team knows which parts of questions they’re really going to be focusing on to decide on the quality of response for an A grade.

Everybody is now busy again with their Post-Its and pencils, recording their name on scripts and also recording their view on the quality of a candidate’s script.

We’ve now moved back to the flipchart again, a new one this time, all hand-written out again, and this time looking at the range of marks for the A grade boundary. And the ticks and crosses are now all available. And I will make a decision, along with the team, about the mark that’s going to be the A grade boundary mark this time.

Well, after three hours of hard concentration on candidates’ papers and hard concentration on mark schemes and reports, I think it’s time now to call the meeting to a halt and go off and grab some lunch somewhere.

I think everybody’s back from lunch now, so we’re now going to move on and look at another paper in this qualification, and start following the same process again to decide on the E and then the A grade boundary for that particular paper.

The reason we are only looking at the E and the A grade boundaries in these meetings is that it is fairly easy to identify those candidates who are the weakest in the cohort, but who just about qualify for passing the exam. And it’s also relatively easy to look at the top end and identify those candidates who really are outstanding in their performance and deserve the A grade. The B, C and D grades are then allocated evenly between the A and the E boundaries.

Everybody is looking pretty tired now and, I think, trying to focus on any more papers is not going to be a good idea, bearing in mind the importance of the decisions we’re making. So I think I’m going to call the meeting to a close for today. And we’re all going to decide where we might meet up to have a meal this evening and catch up a bit more with all our colleagues on what’s been going on.

We will continue this process for another two days. And at the end of that, we will all meet together and go through all the grade boundaries that we have decided upon, and then the AQA awarding officer will sign off all the paperwork, and that will be the end of awarding for this year.

Right now, though, I’m off for a well-earned rest.