Religious Studies

This video considers the key principles of assessment as applied to Religious Studies at both GCSE and A-level. It covers question types, assessment objectives and marking, looking at both points-based and levels of response mark schemes.

Transcript

Hi. I'm Ester Zarifi, and I'm here to talk to you about understanding exam assessment, in Religious Studies.

To best prepare candidates for their summative exam and assessment in RS, it's important that teachers understand the way that this type of assessment works. Designing an exam assessment is a lengthy process that involves many, many layers of scrutiny to ensure that all candidates have fair access to the final examinations. There’re all sorts of factors such as language, accessibility, and avoidance of bias that need to be considered when writing assessment material. All of these things can prove challenging.

In this presentation, I'll discuss the key principles of assessment, question types, and assessment objectives, specific to religious studies, and then mark schemes and marking. That is point marking and levels of response marking.

When we're designing assessments, some of the principles we take into consideration are that the questions reflect the content that's set out in the specification, that students can complete the paper in the time that they're allowed. The questions need to clearly tell the students what they need to do, and the mark scheme then needs to reward students, of all abilities, for what they know and what they can do.

Hopefully, you've seen our short video on the general principles of assessment. If so, you'll be familiar with key terms like validity, reliability, and comparability. However, I would like to remind you of one of the key things we must strive for in assessment, and that is differentiation. This is achieved when an exam paper provides opportunities for students, across the ability range, to show what they know, understand, and can do. By differentiating effectively, an exam is likely to distinguish between students. If they all get full marks, we haven't provided a good assessment. By ensuring that the entire mark range is used, we can separate students based on the evidence of their knowledge and skills, as they've shown in their answers.

The way we gather this evidence of what students have shown us that they know, and they can do, is by measuring their performance against assessment objectives. Our RS papers follow a clear and consistent assessment design. This enables us to measure students against AOs that are set by Ofqual, for both A-level and GCSE qualifications. In RS, we have two AOs, or assessment objectives. AO1 is split into three strands at GCSE, that is for A-level, and it tests knowledge and understanding. AO2 looks at skills of evaluation and analysis. Both of these skills are important in the study of this subject, as students need to be able to demonstrate knowledge, but also to be able to apply and analyse this knowledge, in order to be critical thinkers, and effectively engage with important issues. You can see that, although both levels follow similar AOs, the wording at A-level increases the demand and moves that thinking on, to a higher level.

Our assessments follow a clear structure that is based on the two assessment objectives. As you can see, for example, on the GCSE spec A, the one-mark multiple-choice, the two-mark, and the five-mark questions, are always designed to assess AO1.1, knowledge and understanding of beliefs, practices, and sources of authority. The four-mark questions on Paper One, Beliefs, will always ask for influences, which assesses AO1.2. And the 4-markers on Paper One, practices, and on the themes, will always assess the third strand of AO1, that is the contrast and similarities within and between religions and beliefs. You can see the breakdown for spec B and shirt course on screen, or on the teaching guide webpages.

It's also noticeable that the command words change as the question types move up the mark tariff, and they, therefore, demand more in terms of skills. So, we move from a multiple-choice to a 'Give' question, and then onto 'Explain', and then those 'Evaluate' questions. AO2, analysis and evaluation, is always assessed through, at GCSE, through the 12-mark extended answer questions. They always give a statement and then use the phrase 'evaluate the statement' as the command. At A-level, 10-mark AO1 questions will be asked throughout the philosophy, ethics, and religion sections. AO2 is then assessed in the 15-mark questions. Then finally, in Paper Two, Dialogues, students are assessed by 25-mark questions, that combine both the AO1 ten-marks and the AO2 15-marks. This allows for synoptic links to be made across spec content.

Right, I've got an activity for you now to put what I've just been through into practice. Using the assessment objectives and the tables that I've just shown you, alongside your spec content, I'd like you to write some quick questions on the following sections. So, for GCSE, have a go at a two-mark question on themes, a four-mark question on Paper One, Beliefs, and then a four-mark question on Paper One, Practices. Make sure you check which assessment objectives they need to relate to. And then for A-level, a ten-mark question.

Hopefully, in doing that activity, you've come up with a question for GCSE that asks about influences, that is the second strand of AO1, for the belief four-marker, as that will always be the question type there. And for Practices, I hope you came up with a contrast question. For the two-marker, there shouldn't be any requirement to contrast or ask for an influence. The item should just ask students to 'give two'. The A-level ten-marker you've written should be an 'Examine' question, from any area of the spec content. Doing this again across all item types, I think could be a really useful activity that will help in designing your own practice questions to use with your students in the future.

Understanding where the assessment objectives appear in the examinations, especially at GCSE, can also help with specification delivery. For example, when you are covering the spec in Beliefs content, you now know that the four-mark question would always ask your students about influences, so you can build in those activities or that discussion about how they belief in. for example, the resurrection of Jesus, would influence Christians today, and so on and so on. And so, you can build some activities that relate directly to how those assessment objectives are going to appear on that final exam.

We'll now look in more detail at one question type that's used at GCSE, and that is the multiple-choice question. Recall of facts or terminology might be assessed through these shorter questions, which means also, that the first question on the paper is accessible to the full range of student abilities. This can help build their confidence as they progress through to those items with the high levels of demand. The accessibility of the multiple-[choice questions also result in fewer unanswered questions on the paper, overall, which ultimately helps with standards setting and awarding. Multiple-choice questions, or MCQ's, when they're designed well, they can validly test various areas of the spec in a short amount of time. On the slide, you'll see that there's several paths to a multiple-choice question. The stem is the beginning part of the item that presents the question. The options are then all the possible answers. So, in our case, there's four options. The correct answer is the key, and the incorrect answers, we call the distractors. The question stem needs to be written as succinctly as possible, in order to allow all students to understand what is being asked of them. Any unnecessary information needs to be removed, and the wording should remain minimal.

MCQs, such as the one shown on screen, can be really helpful in the classroom as a way of testing the knowledge of topics and revising an area quickly. They could be built into starters or plenaries as a way to assess understanding from a lesson, and quickly, to identify misconceptions, and this allows teaching to be reactive. They can also be a tool for starting discussions around a topic and a quick revision exercise.

If you were going to write your own MCQs, it's worth us looking at some useful principles in how we design them. The question needs to give a clear and concise instruction. The options should be of comparable lengths, so that no one option stands out too much as the correct, or incorrect, answer. The options should also include plausible distractors. In other words, they need to be, at least, related to the subject matter, so that they can't be eliminated by students who do not know the correct answer.

Answering MCQs should take students one mark worth of exam time. So, in GCSE RS terms, that's about a minute. So, the item shouldn't be too wordy, or it'll take them too long to read through the question stem and all the options, and then think of their answer. Have a look at the MCQ on the screen. Based on the principles I've just been through; how could this question be improved? So, here we have a question that says: 'The festival that remembers the Buddha's birth, enlightenment and death is called?' A) Enlightenment, B) Karuna, C) Wesak, or D) Holi. If we remember the principles that the distractor should be in alphabetical order, we ensure all distractors are plausible, and we shouldn't use an option within the question stem, as it stands, it wouldn't assess the content as accurately as it might. Firstly, the distractors aren't all plausible, as they aren't all Buddhist practices, which means a student who's revised about Buddhist practices could rule out option D) Holi, straight away, as that's a Hindu festival. So therefore, that reduces the level of demand. Secondly, the term 'enlightenment' is in the question and in the options. And so that could be really confusing, especially if it was the correct answer in this case, and it would compromise the validity of the question. Also, when we're writing an MCQ, the distractors are always listed in alphabetical order, to remove any sense of bias or pattern, which could be unintentionally written into the question. Overall, the phrasing of the question isn't as clear and concise as it might be, either.

The question when it appeared on our GCSE paper actually looked like this, which you can see works far better and meets all those design principles. The question actually read, 'Which one of the following is the festival that remembers the Buddha's birth, enlightenment, and death'. And so, we've got a question that clearly tells the student what they need to do. And then the options were Karuna Samatha, Wesak, and Zazen. So, they were all terminology from the Buddhism paper, and they were written in alphabetical order. They were all about the same length, and none of the words from the question appeared as options, as they did in my previous example.

As we've tried to show you through that quick look at an MCQ example, the language used, as well as other factors, need to be carefully considered when we're drafting questions. As such, during the question paper production process, all our items go through rigorous checks and a series of revisions if needed. Our teams consider things including the use of command words, whether the questions relate clearly to specification content. Checks might also ask things such as, is the question concise? Could the order of the wording be improved? What answers do we actually want to elicit from the students? In other words, which part of the spec content are we wanting to test them on? I asked our assessment design team to share some examples of what this process looks like. And as you'll be able to see from this 2021 GCSE question, careful revisions were made here. The draft and final version, although we assessed the same content, are clearly different. The final question is far more concise and removes the unnecessary, and possibly confusing phrase 'the worldwide community'.

I'll also show you now, an example of a 12-mark question statement from the 2022 GCSE Christianity paper. You can see that the draft version was quite long and the key idea about adult or infant baptism was somewhat lost. That would, or could, have impacted, quite significantly on the accessibility of this question. And the final version, though, that reads 'Only adults should be baptised into the Christian religion' is far more concise and tests the subject knowledge more directly, which ultimately, is going to give us a more valid assessment outcome.

So far, we've talked about assessment principles and question types. Finally, I would like to spend some time considering the two types of mark scheme, and therefore, the two approaches to marking that we have in Exam RS. Alongside the exam paper, the mark scheme is crucial in ensuring effective standardisation across students, and therefore, the reliability of our assessments. A mark scheme is developed alongside the question paper, and before the final word and have questions is decided, the mark scheme is structured to ensure that the focus of assessment is rewarded correctly. It's the same as planning a lesson, essentially. Teachers know and understand the objectives of their lesson before they design the content and activities, as ways to deliver and achieve those objectives. In the same way, we know our assessment objectives, where they fit into the question structure, and the content that we want to assess. The wording of the mark scheme is critical and needs to ensure that it will allow students to be credited for correct responses, even if they're not the answers that were originally anticipated. During the standardisation process, once scripts have been seen by senior examiners, the mark scheme content is often added to, or amended to reflect common or previously unexpected answers.

In RS, we use two approaches to mark schemes and marking, that is point-based marking, and levels of response marking. Point-based marking is used in short or low-tariff questions at GCSE. This type of mark scheme enables examiners to reward students on a point-by-point basis. For example, on a 2-mark question, ultimately, examiners are looking for whether the student has stated two correct points in their answer. Or on a 4-mark question, one mark is awarded for a simple point. The second mark is awarded for development of that point through the inclusion of additional detail, or examples. Essentially, then the examiner is counting up a point and a development or detail, and then a second point and a development or detail, to award that mark out four. For open-ended questions, it's often best to use a levels of response mark scheme, as this gives you some flexibility in awarding marks for merit, even if the student's response is unexpected.

Although assessment writing teams try to consider many possible answers, until the exam has been sat, it's hard to gauge exactly how a question will be received. For this reason, indicative content is built into a mark scheme to highlight the most likely answers. Therefore, levels of response mark schemes are used often when there's multiple ways a student could approach a question and still be successful. They're also used when assessing skills. In our case, in our AO2 evaluation questions, in addition to just the knowledge-based questions. Levels of response are used throughout the A-level and GCSE for the higher tariff, 12-mark questions. Using the levels of response mark scheme requires a different approach to marking and from a points-based mark scheme, and this distinction is really important. When you're marking a longer response, such as a GCSE 12-marker, or an A-level answer, you would be taught as an examiner to read the whole answer before awarding the level. In other words, you're not checking off points as they are made like you would on a 2-marker or another short question. This way, the full response would be considered and would be that you'd get the best fit in terms of the level identified.

There’s full instructions on how to apply levels of response in the opening section of all our published mark schemes. The levels might include a cap on marks dependent on criteria. For example, as you can see from the table, at GCSE, there is a cap of Level Two for a one-sided response. That is one that includes a point of view as the levels say, and also a cap for answers that are purely secular. So, on reading the whole answer, you can easily see if it would be capped at 6 out of 12. Either way, you then need to read the level descriptors and see if the answer meets all the criteria set out.

To achieve Level Four at GCSE, answers need to include judgements. In other words, students need to include an evaluation of the statement. Do they overall agree or disagree with the statement? And they need to give the reasons for this judgement. For example, are arguments and evidence strong or weak? Why? Students can include evaluation throughout their answer, if they wish, in a conclusion at the end, or even make that judgement at the beginning and then build up their argument. The mark scheme for these questions is as inclusive as possible. All relevant material will be credited, but the key to the high levels is making sure that the statement from the question is fully addressed within their answer. The 12-mark question is a very effective differentiator, and so, although aiming for Level Four is great, for many students, being able to gain 5, 6, or 7 out of 12 on an extended response question at this level would be a fantastic achievement.

Remember, a good assessment will differentiate between students, and thus, produce valid results overall. Our A-level RS assessments are always marked using levels of response. There are separate level descriptors for AO1 and for AO2. You can see here, that AO2 descriptors include a cap at Level Two for a one-sided response, and that for Level Three and above answers need to include evaluation supported by reasoning.

It's really useful, I think, to share these mark schemes with students, at both GCSE and A-level, so that they can not only judge the level of their own work, but by understanding how their work is assessed, they can then develop understanding of how to improve their exam techniques. Mark schemes can be useful for marking assessments too, as they provide ready-made feedback that you can give to students. As well as self and teacher assessment, students could also be shown example answers across the question types and could mark them together or individually. These activities can help to demystify the examinations and then build confidence in students as they prepare for their final assessments in religious studies. Thank you.

Questions you may want to think about

• How can you use these insights to prepare your learners for exams?
• Do your internal assessments reflect the approach of the exam? To what extent do you want them to?
• What’s the most important or surprising thing that you’ve learned? How might it influence your teaching?