Mathematics

This video focuses on how mark schemes are developed in GCSE Mathematics. It covers the principles that underpin Maths mark schemes, including how marks are allocated to be fair to all students. We look at how a mark scheme for a problem-solving question is created and refined and how some questions need a very different approach.

Transcript

Hi, I'm Andrew Taylor. I'm Head of the Maths Curriculum team at AQA. Hopefully you'll have seen our general assessment video about the principles of assessment, and that covers things like reliability and validity and what those terms mean. What I'm going to do in this video is talk about how those principles apply in mathematics, and specifically, I'm going to look closely at how maths mark schemes help to contribute towards high-quality assessment.

In maths, the important thing about mark schemes is that they have to reward method as well as the final, hopefully correct, answer. And of course, students work through problems in a variety of different ways, and we've got to have mark schemes that can consistently and accurately and reliably reward effort, however a student approaches a problem. And that can sometimes be a challenge.

So, in this short video, we're going to try and look at three things. We're going to cover the principles and the approaches that we use in maths when developing mark schemes; we're going to look at how marks are allocated for a particular question, and you'll get an opportunity to try that out for yourselves; and we're also going to look at how important it is that we use student examples, as well as our own experience of how students might answer a question, to make sure that a mark scheme can cover all eventualities fairly and robustly. So, those are the three areas we're going to look at.

First of all, let's have a look at the principles of mark schemes that we apply in mathematics. Our first principle is that we should always reward any valid method that students use. Doesn't have to be the most effective method, doesn't have to be the most efficient method. For example, if a student is carrying out a multiplication and they decide to use repeated addition, then as long as they show us their intention and it’s clear what they're doing – chances are they'll get it wrong because it gets too complicated – we'll still try and give them credit for a method that could lead to a valid solution. And it's that ‘could lead to a valid solution’ that's important. In terms of the number of marks for a question, one of the considerations is how long it's going to take the student to answer. And we generally work on a rule of thumb of about a mark a minute. So in GCSE maths, we have 80-mark papers, and students have got 90 minutes to work their way through them. So, in maths, we want to give credit for the first step towards a solution, so particularly in GCSE we'll often give the first mark for the very first step in that journey towards getting it right. And even if students don't take any other steps, they'll get some credit for doing something mathematically worthwhile.

On the other hand, there are sometimes steps in questions, often towards the end, where if a student picks up the last method step, they will get, for example, the correct answer. So, they will either get two marks there, or they will get no marks. And in that situation, we might take a view that we're just going to give one mark for those last two bits, because otherwise we're unfairly differentiating between those who can and those who can't, and that is something we want to avoid.

Of course, every mark scheme has got to give credit for the skills that the question was written to show. So, in awarding marks, yes, we want to award marks for the first step, we want to make sure marks are fairly awarded, but we have to make sure that we are actually rewarding the skills that are intended to be tested. And of course, in high stakes assessment, we have to apply that mark scheme consistently, across hundreds of examiners and across thousands of students. And sometimes that involves compromise; in order to achieve that reliability, we may not be able to be perhaps as fair as we would like to be to every single student. Sometimes, we'll see a student has done something that looks worthy of some credit, but if it doesn't work in the mark scheme, then we can't give that credit, because we’re dealing with consistently and fairly marking 200,000 scripts.

And finally, a mark scheme has got to allow for all possible responses that might come from thousands of students. One way of dealing with that is to have the most common likely solutions expressed as different mark schemes. Now, that can become confusing for the examiner, and can really increase the cognitive load when they're looking at three or four different methods. So sometimes a better approach is to state generally what each mark is for, give perhaps a typical solution, and then trust the judgement and expertise of the examiner. It puts more onus on them, but it probably ends up being a better approach to marking.

One way that we think of possible responses to a question is to consider the expected responses, and then to consider the actual responses that students make. So, in the diagram, you can see the red ring is what we expect. So, we can allow for that in preparing the mark scheme. And then the green ring are the whole set of actual responses that students produce that are worth some credit. Of course, in any exam, students respond in ways that are obviously wrong, and we can dismiss those. The ideal is to have that central zone, the red and the green ring, to be perfectly overlapped, so that what we expect is everything that we see, and that's the perfect mark scheme. And that's what we see in Zone One, there.

Of course, sometimes we expect students to respond in certain ways and they don't. And that’s Zone Two. And that means we've wasted perhaps a little bit of time writing a mark scheme for a solution that doesn't appear, but that's no big deal, really. The more worrying zone is Zone Three, where a credit-worthy response is seen and it's one that hasn't been allowed for. That's why, after first writing a mark scheme, once the exam has been sat, senior examiners spend days looking at solutions, adding to the mark scheme, so it can deal with every eventuality. And even then, there may be occasions where a credit-worthy response doesn't meet the mark scheme, and that's why we're able to refer scripts, we're able to consult with senior examiners, to make sure that every student can be fairly assessed.

So now we're going to look at how marks are allocated across a question. The example here is from a higher-tier practice paper. It's a problem-solving question from quite late in the paper so it's quite high demand, and it's worth five marks. What I want you to do is to decide how to allocate those five marks across the solution to the question. What I suggest is that you have a go at the question, have a look at it, and then decide where the five marks should be given. So, if you pause the video now, and spend about ten minutes looking at that, and then we'll come back and have a look at the mark scheme that we did for this question.

Okay, so I hope that was an interesting exercise. And now we're going to look at the mark scheme that we published to go with this question. So, the first thing to notice is there are two alternative mark schemes given, but they're actually quite similar, so they could probably be condensed into one. In both versions, there's a lot to do for the first mark. Students have got to work out a method for the area up to point B on the graph. For the second mark, they've then got to get an expression for the area between points B and C, and there are a couple of different versions of that given in the two different schemes.

Again, for the third mark, there's quite a lot to do, because they've got to come up with a complete expression for the area under the graph, and then that's got to be equated to the distance covered. The fourth mark in Method One, that's to get a simplified equation. In Two, it's just to get those final numbers to be added together, and in both cases the fifth mark is for the final correct answer. So, on the right-hand scheme, the fourth mark is for their 13.5 + 14, and the last mark is for the final answer of 27.5. And it seems unlikely that any student at this level who got 13.5 + 14 wouldn't actually go on to get the 27.5. So it's worth looking at that and deciding whether that's the best approach to this scheme.

So, now, we're going to look at a different approach to that same mark scheme. So, this possible mark scheme is taking an approach that's more similar to the one we use in A-level, where on the left-hand side you've got general instructions for markers, then the marks in the middle, and then a typical solution – but it doesn't have to be the only solution – on the right-hand side. So, looking at this scheme, mark by mark, the first mark is for a method to find an area under the graph in the section up to B. So that could be the triangle, it could be the rectangle, it could be the whole trapezium. Any one of those methods will get the first mark. So, that fits in with that principle of awarding the first mark for the first step in a solution, unlike the earlier scheme.

Then, the second mark is about an expression for the area between B and C, and we've condensed this, the two ideas from the last one, together, so it could be using t−14, or it could be just a sign in a letter, for the height of that trapezium. The third mark is for that total area expressed and equated to the total distance. And it doesn't have to be tidied up, it could be quite messy, as long as we can see that expression and that the unknowns on either side are consistent – so if the students are using t, then they're using t on both sides. If they've introduced an x, then they've got to be using that on both sides. The fourth mark is for the simplification of that equation. And then the fifth mark, again, is for the correct final answer.

And I think that mark scheme, it requires a certain amount of judgement from the examiner, but it seems to work more flexibly for this kind of problem than the two alternatives we saw earlier. Of course, neither of the mark schemes has actually been used in anger because the question was from a practice paper, so it hasn't stood the test of being sat by thousands of students.

Not every question in maths exams is that kind of method and accuracy, following a solution through approach; there are some quite different styles of questions, and they need quite different approaches to the mark scheme. A good example is from our Level 3 Mathematical Studies Qualification, our Core Maths qualification, where one of the requirements of the specification is for students to make fast, rough estimates of quantities and calculate with them. So, here's the question – students are being asked to estimate the amount of water drunk by the population of a small town in a month. Of course, we're not expecting accurate answers here, and a method-accuracy mark scheme just wouldn't work. So, let's have a look at the mark scheme we did use for this question.

Okay, so this was a 5-mark question requiring estimation skills, and this is the mark scheme we used. It's worth noting that this question was answered fully correctly by 65% of the students in the actual exam back in 2017. So, the first three marks are for making sensible assumptions for the number of litres a person will drink in a day; the number of days in a month – that's the easy bit; and then, how big is a small town? And we give huge ranges, so the amount of water drank can be anything between 1 and 10 litres a day; the size of a small town can be anything from 1000 to 100000. As long as students state those assumptions clearly, then they will get those first 3 marks. And then, the fourth mark is simply to combine those assumptions, those values, in a sensible way – in this case, to simply multiply them together. And then, the final accuracy mark is their answer to their calculation. Sometimes we might insist on sensible rounding, and sometimes we might not, for that final mark. So, a very different approach to a mark scheme for mathematics.

But that's not the whole story for a question like this. Because there are so many possible responses that we might see from students, we've got to consider all of them so we can fairly mark the question. And that's where the additional guidance comes in and is so important in a question like this. The additional guidance is, as this name suggests, additional notes that are given to examiners after the mark scheme to help them focus in and mark accurately. So, let's have a look at the guidance for this particular question.

So, the additional guidance just gives more detail for the examiner about what's accepted and what isn't accepted in this question. So, it talks about what we allow for the number of days in a month, for example. It talks about what to do when a student works in millilitres, and what to do when they do extra working that doesn't necessarily get credit in the mark scheme. The additional guidance is vital for almost every question, but particularly in those questions where students are likely to have a wide, wide range of responses, whether they're offering an opinion, or a bit of analysis, or evaluation.

Okay, so that's it. Hopefully, this short video has given you some insight into the work that goes into developing a mark scheme that is fair and consistent for all learners. And of course, we're just scratching the surface and talking about mark schemes; there are lots of other considerations about how we write questions, about how we avoid bias, about the care we use with language. But they'll have to do for another time. Thank you, and I hope you found this useful.

Questions you may want to think about

• How can you use these insights to prepare your learners for exams?
• Do your internal assessments reflect the approach of the exam? To what extent do you want them to?
• What’s the most important or surprising thing that you’ve learned? How might it influence your teaching?