In this short video I'm going to be talking about mark schemes. Mark schemes are really important in assessment. The purpose of the mark scheme is to make sure that markers reward the right thing in the assessment. This is actually, as we discussed in one of the earlier videos, a really hard task to do. And so, I wanted to start the section with a quote from one of our researchers at AQA and an ex-colleague of mine. "Mark schemes should be designed with cognitive demand in mind. Clear, concise and simple mark schemes are likely to elicit more reliable marking". So, as well as thinking of students’ cognitive load, we also need to be mindful of markers. They are doing a difficult task and we want to make sure that mark scheme really supports them in knowing precisely what they should be rewarding and so they can really focus on inferring, from the student, what they know, understand, and can do, and can reward it appropriately.
Mark schemes come in many different forms. Effectively, there are about three types. You have objective mark schemes, which are really for multiple choice questions where there's only really one right answer. The next type is points-based mark schemes, which are for fairly constrained questions, where you can be reasonably prescriptive about what is right in the student's response. And then you can have levels of response mark schemes, which are really designed for more open tasks, like essays, where the marker needs to look at the level of response and take a best-fit approach to rewarding the marks appropriately. Here they are looking more at the quality of the answer rather than a list of correct points.
When you're trying to choose the sort of marking you should be using, really, you can think about the sort of response that you're asking from students, first of all. You can think of the sorts of responses on a continuum, from a very open response, all the way through to a much more closed response, like a multiple-choice question. The type of mark scheme that would be most appropriate to use would be dependent on that response. So objective mark schemes would be used for multiple-choice questions. You might be using a points-based mark scheme for the more constrained questions in the middle. And then, for the more open questions, actually, a levels-based approach would be a better fit.
So, to bring this to life, there's a couple of different questions on the screen. What sort of mark scheme do you think should be used for each one? Take your time, press pause, and play the video when you're ready to continue. So, Example One, is a pretty constrained question and a points-based approach would be the most appropriate thing for that question. Question Two is an open response question, and so, a levels of response question where you can really reward the quality of the answer is the best approach to be taking. Examples Three, Four, and Five demonstrate the judgement calls involved in selecting the appropriate mark scheme for short answer questions. With these types of questions, we could be assessing either the correctness or the quality. So, whether we want to use a points-based or a levels of response mark scheme depends on the question we've asked, and the nature of the responses we want to elicit. If we asked you for a list of things, then it is likely we would use a points-based mark scheme. Whereas, if we ask them to evaluate the usefulness of a source, we might decide that a levels of response scheme is more appropriate. In short, there are some question types where either a levels of response or a points-based approach could be taken. It's a bit of a grey area on the spectrum.
So, we are now going to look at some guidance for best practice in writing mark schemes. So, when it comes to writing mark schemes, what we're really trying to get at is that level of guidance for markers so they can mark really consistently. When it comes to levels of response marks scheme, that consistency comes from the level descriptors. What we're aiming to achieve with the level descriptor is a descriptor that is qualitatively different from the level above and below, and also has a sensible progression between the different levels. This can be actually quite hard to achieve. I'm going to talk through an example now to show you exactly how.
So, the reason it's difficult to get that progression, is that each level often incorporates a number of distinct assessments and themes. So, in this example, one such theme is a quality of conclusions. And even this theme involves two elements, the clarity and complexity. In this example, there's enough daylight between each level, and there's a reasonable progression across all levels.
So again, I have a little task for you. On screen, there’s going to be two mark schemes where the level descriptors have been jumbled up. What I want you to do, in a second, is to pause the video and see if you can put them in the right order. The correct answer is now shown on screen. Did you find two easier? This is because the progression in Example One is not very clear. Where does 'guide the reader' sit? Is it asking something else? Where progression isn't clear in the levels, that's where we may find some inconsistency in marking, where markers struggle to put students into the right band. I'm now going to be talking a little bit about points-based mark schemes. With points-based mark schemes, validity can be lost where there's a mismatch between the question and the mark scheme. In this example, the question asks students to explain two short term effects of exercise, but the mark scheme awards marks for simply stating short term effects. The command word in the question is not consistent with the type of answer credited in the mark scheme. It's possible that students will write much more than they need to and will take more time than needed, and possibly increase the chance of them mentioning a single effect.
Another way validity can be lost is through a lack of transparency in the marking rules. Questions can have marking rules that test takers are not aware of. In this example, the mark scheme is requiring two separate reasons, one mark for each. But the question does not make that clear, so that some students might think that a developed single reason would suffice for two marks, when actually they'd only get one single mark, according to the mark scheme. Another way validity is lost, is when a mark scheme categorises the response space in a way that's not appropriate or correct. In this example, in the first three bullet points, possible responses are grouped together under a single mark in a way that students would not realise. For example, they might answer 'Jesus rose up from the earth into heaven. When he got to heaven, he met God'. But this will only get a single mark. The first two points listed in the first bullet of the scheme are really separate points, each deserving a mark in their own right. But the way the mark scheme categorises them means they are treated synonymously.
To summarise, here are some of the key takeaways for mark schemes. Be clear on what you're assessing, in order to know the type of mark scheme that's most appropriate. For levels of response mark schemes, ensure that each level is clearly distinguishable from the adjacent levels. For points-based mark schemes, ensure that the mark scheme fits the question, and there's nothing hidden in the mark scheme. All mark schemes should be designed with the cognitive demand on the marker in mind.
So, I hope this video has given you a bit of insight into the work we do around mark schemes to make sure they support our markers to mark consistently and fairly to students. Thank you for watching the videos. I hope you found them interesting and useful.
