Economics

This video looks at how general assessment principles are applied to Economics and contribute to the development of high-quality assessments in Economics. It considers the importance of assessment objectives and of quantitative skills in the development of Economics assessments and identifies the role of different types of questions.

The video also explains the relationship between the assessment objectives and mark schemes and finishes by providing some examples of student responses to some past questions.

Transcript

Welcome to this presentation explaining how AQA ensures that assessments in Economics are fit for purpose. I'm Stuart Luker. I've worked for AQA for a number of years since roughly 1985. I'm currently the chief examiner for Economics. I started teaching economics in 1974 when Harold Wilson was the Prime Minister.

Hopefully, you will have seen our introductory video on the general principles of assessment and will be familiar with some of these principles. In this presentation, I'm going to look at how those principles are applied to Economics and contribute to the development of high-quality assessments in Economics. After that, I'm going to consider the importance of assessment objectives and of quantitative skills in the development of economics assessments. I will then move on and identify the role of different types of question and explain the relationship between the assessment objectives and market schemes. And finally, finish by providing some examples of students' responses to some past questions.

When developing assessments in Economics, the fundamental objective is to ensure that these assessments conform to the fundamental principles of assessment. Assessments aim to measure a student’s knowledge, understanding, and skills. The principles are: that assessments should be valid. In other words, they should measure what they're supposed to measure. Secondly, they should be reliable. They should be consistent, and different assessments and different examiners should provide the same result. Assessments should be comparable exam papers from different years and different options within a paper should have the same level of demand. Assessments should also be predictable, but predictable in the right way. The style of the paper should be familiar to the students, but not the content that's covered. Assessment should discriminate between students. Questions and exam papers should generate a range of marks. Unless they do this, you don't reward the right students with the right grades.

Assessment objectives are the description of what the student needs to demonstrate in response to the questions in the examinations. The assessment objectives for GCSE, AS, and A-level Economics are set by Ofqual, and they were developed in consultation with the exam boards and are the same for all specifications of the same type for all exam boards. So, for example, the A-level Economics assessment objectives are the same for AQA and the other exam boards. The AOs for GCSE Economics are similar, but not the same as those for AS and A-level Economics. The equivalent AOs at GCSE are less demanding than those at AS and A-level.

The validity of assessments is measured by ensuring that the assessments measure how well the students have addressed the different AOs. Economics is a quantitative subject and it's important that students sitting economics exams at GCSE, AS, and A-level are assessed on their ability to demonstrate quantitative skills as they apply to the subject content. Ofqual, in consultation again with the exam boards, has laid down exactly what these quantitative skills are, and they're specified in the various specifications. So, the quantitative skills for GCSE are different from those that AS and A-level. At GCSE, students are expected to demonstrate quantitative skills at key stage three level. At AS and A-level, it needs to be at level two quantitative skills.

AQA specifications lay down the weighting attached to each of the assessment objectives. These weightings conform to Ofqual's requirements. For each question, for each paper, and set of papers, the marks allocated to each assessment objective are fixed. This ensures that the papers conform to the AO weightings in the specification, and that the assessments in different exams are comparable over time. When the papers are constructed, they're also checked to ensure that the quantitative skills requirement is met, and that there is a reasonable coverage of the different quants over a number of exam series. When we construct the papers, we don't only consider the assessment objectives and the quantitative skills. It's also important that we consider which sections of the specification are being assessed in each question and across the whole paper. The aim is to make sure that there is a reasonable coverage of the specification in each exam series, and over time. This process helps to maintain the validity, the reliability, and the comparability of the individual papers, and sets of papers, in a particular exam series and in a series of papers over time.

Economics is an applied subject. Although it has a well-established body of knowledge, it is also applied to what's going on in the real world. The applied nature of the subject is reflected in the assessments at GCSE, AS, and A-level through the provision of source material and also providing context for individual questions, for example; essay questions. As well as ensuring that the assessments conform to the general principles of assessment, it's also important that questions and source material are accessible, but they're suitably challenging, and they're expressed clearly through a language that is appropriate for the students that are taking the particular assessment.

Although the A-level Economics specification identifies four distinct assessment objectives, these assessment objectives are not independent of each other. For example, knowledge and understanding will be demonstrated through application and it can also be demonstrated through analysis. Evaluation, if it's going to be good evaluation, will be analytical. Although examiners must consider the extent to which a candidate has been successful in demonstrating an individual assessment objective, often, they must take an holistic view of the skills demonstrated in the response to a question, and they need to judge how successful a student has been in using economic ideas to answer the question.

Some questions are constructed, so they just assess one assessment objective. Other questions are designed to assess several assessment objectives. The command word that is used in the question indicates the AOs that the question is designed to assess. For example, a question that has the command word 'explain' will at AS and A-level assess assessment objectives AO1, AO2, and AO3, but it will not assess evaluation AO4. The command words, assess, evaluate, discuss, and justify, are likely to be used in high tariff responses that require extended writing. These command words indicate that at AS and A-level, all four assessment objectives are being assessed. At GCSE, they indicate that AO2 and AO3 are being assessed. Although, it does need to be borne in mind that AO3 at GCSE includes both analysis and evaluation.

We're now going to look at a few questions and link them to the assessment objectives. The first two questions are taken from a GCSE Economics paper. I want you to consider which assessment objectives each of these questions is designed to assess. The first one 'State two types of economies of scale' for two marks. This is clearly going to be AO1. Which assessment objective is the following question designed to assess? 'Using Table 1, calculate the monthly profit of the café Show your working.’ This will assess AO2 but is also assessing quantitative skills. The question below is taken from an A-level Economics paper, and which assessment objectives is this question designed to assess, do you think? It says 'To what extent do you agree that reducing the budget deficit is more important to the UK's macroeconomic performance than reducing the current account deficit on the balance of payment? Justify your answer.'. In this case, it will assess all four assessment objectives that students will be rewarded for; knowledge and understanding; applying that knowledge and understanding to the context; for their ability to analyse the issues that are raised by the question; and because the command word is 'justify your answer', they will also be assessed on their ability to evaluate, and the extent to which they are successful in judging whether or not, or the extent to which reducing a current account deficit, is more significant than reducing a budget deficit.

Okay, we're now going to look at command words for questions that assess quantitative skills. The following command words, clearly, are going to assess quantitative skills: calculate; draw; explain how the data show; and with the help of a diagram, explain. When the command was 'calculate', or 'draw' used, they're also likely to assess AO1 and/or AO2. When 'explain' is also part of the instruction, the question at AS and A-level will also assess AO3.

On the A-level Paper Three, Section B: The Investigation, Question 31 consistently uses the following instruction: 'To what extent, if at all, do the data…'. And then, it depends on the nature of the data and the context for the particular question. But the instruction 'to what extent, if at all, do the data…' indicates very clearly the quantitative skills are being assessed. Because it says, 'to what extent?', the instruction is also indicating that all four assessment objectives are being assessed, in particular: evaluation. They need to make a judgement about the extent to which, if at all, the data does actually show whatever the question is actually asking. For this question, the same instruction is used consistently from one year to another. This illustrates good predictability. The style of the question is familiar. However, the context and the subject content is not predictable. In other words it is good predictability, in terms of the style of the question, but they don't know exactly what the context is or what we're actually going to examine.

Okay, on this slide, I'm showing an example of a question from a GCSE Economics paper that assesses quantitative skills. As you can see, the command words are 'draw' and 'label'. The question is assessing quantitative skill four, QS4, and that quantitative skill is: construct graphs from data including supply and demand curves, which clearly this is relevant to. The question is also assessing AO2: apply knowledge and understanding of economic concepts and issues to a variety of contexts.

I'm now going to look at multiple-choice questions. Multiple-choice questions contest AO1, AO2, and AO3, and they're also an important means of assessing quantitative skills. However, they cannot test evaluation. At A-level, the number of questions assessing each AO and quantitative skills can vary, but only within a very small range. This ensures that the demands of Section A of Paper Three is similar from one year to another, and hence the assessment is reliable over time. Similarly, at GCSE and AS-level, we also make sure that the number of questions testing each of the AO is relatively fixed, and that the number of questions testing quantitative skills is also fairly fixed. We also pay attention to the number of diagrams and the number of questions that have got tables. We want to ensure through multiple choice that there is good coverage of the content of the specification, but we also want to make sure that the demands of the multiple-choice section are consistent from one year to another. Multiple-choice questions are a reliable method of assessment, in that the mark awarded is not affected by the judgement of an examiner. As with other questions on the paper, the multiple-choice questions are checked by several subject experts to ensure that they are valid and reliable and will discriminate between students of different abilities. The key must be unambiguously correct. The distractor should be plausible and likely to attract students who do not have the relevant subject knowledge and or skills. Some multiple-choice questions are more demanding than others. However, the multiple-choice sections of each of the papers are checked to ensure that they are comparable and that overall they have a similar level of demand from one exam series to another.

The following multiple-choice question is an example taken from a GCSE Economics paper. Firstly, I'd like you to consider which AO this question is testing and then to consider why you think it is a good question, and I hope you do. This multiple-choice question is testing AO2: ‘apply knowledge and understanding of economic concepts and issues to a variety of contexts.’ What are the good features of this question? Firstly, it's valid in that it is testing an important section of the specification, i.e., price elasticity of demand. The stem, including the instructions, is clear and concise. All the options are plausible and hence the question is likely to discriminate well. The numbers can be easily determined from the two figures in the stem and a candidate that doesn't really understand the economic principle could go for a plus as opposed to a minus, hence, the extractors are likely to work effectively.

Assessments in Economics, at GCSE, AS, and A-level are predictable in a good way. At both GCSE and AS-level the structure and the style of the assessments is the same for Papers One and Paper Two and is also the same each year. At A-level, the structure and the style of the assessments is again the same for papers One and Paper Two, and again, is the same for each year. However, Paper three is different but is nonetheless consistent over time. Students going into the exam should be familiar with the style of the assessment and the demands of the different types of question. However, I want to emphasise that the content of the assessment and the subject content is being assessed is not predictable. Most questions in economics are assessed using a 'levels of response' mark scheme. The descriptors in the different levels of response are closely linked to the assessment objectives that are being assessed by the particular question. When marking, examiners start by annotating the response to indicate where the student has demonstrated the different assessment objectives. To determine the mark, the examiner takes an overall or holistic view of the response to determine the level that best fits the response before they decide on the mark to award within that level. The mark schemes also include indicative content. I want to emphasise the students are not expected to cover all of the issues mentioned in indicative content, but neither is indicative content meant to be exhaustive. That expert examiners can use their judgement to determine whether or not something that's not in the scheme, but is being discussed by the student, is worthy of credit. Although the indicative content in mark schemes will vary, where possible, the same levels of response mark scheme is used to assess questions that are assessing identical assessment objectives. For example, at A-level, the same levels of response mark scheme is always used to assess the first 15 mark part of the essays on papers One and Two. The same levels of response mark scheme is always used to assess the last 25-mark part of the data response and essay questions on papers One and Two. Similarly, at GCSE, the same levels of response mark scheme is used for the six, nine, and fifteen mark questions on both examination papers. Using the same levels of response mark scheme helps to maintain consistent marking between questions, between papers, between examiners, and over time. Where there are optional questions, the use of a common levels of response mark scheme helps to ensure that the optional questions are comparable, i.e., that each question has a similar level of demand.

I'm now going to show you extracts from an A-level mark scheme and look at the link between the descriptors in the different levels and the assessment objectives. I've chosen two levels; level five and level two. As you can see, the first descriptor in both of these levels is to do with Assessment Objective one. In other words, knowledge and understanding. The second descriptor is to do with the application of knowledge and understanding, the third analysis, and the fourth evaluation. So, there is a clear link between the assessment objectives and the levels which are actually used to determine the mark that is given to a particular student for their response.

I'm now going to move on to look at a student's response to a particular question. The first example we're going to use is for a nine-mark question taken from a GCSE paper. The question is 'Using Figure Two, assess whether the government is right to always aim for economic growth as an objective for the economy'. Note that the command word is 'assess' and at GCSE, this indicates clearly to the students that they need to demonstrate their ability to apply knowledge and understanding, AO2, and to analyse and evaluate the various issues, AO3. What I'd like you to do is to read the student's response and then pause for a minute, move on to the next slide and use the levels mark scheme on the next slide to decide in which level you would place this response. This is the GCSE levels of response mark scheme for the question that you've just had a look at. Note that the statements in the levels of response mark scheme correspond to the assessment objectives. For example, in Level Three, the top two statements relate to AO3, in other words, through evaluation and analysis respectively. The third statement corresponds to AO2; the application of knowledge and understanding. Making sure that the level descriptors in the levels of response mark scheme correspond to the assessment objectives helps to ensure the validity of the assessment. Whilst it would have been better if the student had started by defining economic growth, it soon became clear that they understood the concept. They could also have referred to and used 'though this may not be shared out evenly' mentioned in the extract as part of their response. However, they apply economic principles effectively to help answer the question, and for the most part, they use specialist language appropriately. The arguments presented are sound, with developed logical chains of reasoning. They evaluate throughout and their conclusion is reasonable and it's fully justified. Using a best-fit approach, this was judged to be a Level Three response.

We are now moving on to look at a second example of a student's response. This time it's taken from an A-level paper. It's actually an answer to the first part of an essay question from A-level Paper Two: Question 13, Summer 2019. The question is 'explain the main causes of a rise in inflation'. The command word for this question is 'explain', which indicates to the students they're expected to demonstrate their ability in AO1, AO2, and AO3, but they're not expected to evaluate, i.e., they're not expected to demonstrate AO4. This is a student's response to the question 'explain the main causes of rise in inflation'. Read it very carefully. Note that the various assessment objectives are indicated by annotations, and in the exam, there would be a final comment made by the examiner. Pause for a minute, read the student's answer, and decide what mark you would give to this response. This was judged to be a Level Three response, and I hope you agree. It was without any hesitation awarded the maximum of 15 marks. It fully satisfies the Level Three criteria that you can see on the screen. The student has produced a well-organised response that demonstrates all three assessment objectives at a very high level.

The third example I'm going to show you is a student's response to the second part of an essay question taken from the A-level Paper One, Summer 2022. The title of the essay is 'Assess the view that high-speed internet connection is a necessity for modern life and should be provided by the government free of charge to all households'. The command word is 'assess', and this indicates to the students that they're expected to demonstrate their ability in all four assessment objectives; and that evaluation, AO4, is prominent in this. Again, please read the student's response, note the annotations, including the final comment. This student's response goes over two slides, and you'll need to read both slides. What you will also notice is that there is a context to the question which adds some realism and some application and may be used by the student as part of their response to the question. Once you finish reading this part of the student response, please move on to the next slide. This response largely satisfies the Level Four descriptors but has some of the characteristics of a Level Three response and so is placed at the bottom of Level Four. The analysis is not always fully developed and the evaluation, including in the conclusion, is often superficial and in these respects is more of a Level Three response than a Level Four response. However, taking a holistic view, it was placed at the bottom of Level Four.

Thank you for taking the time to listen to this presentation. I hope it's provided some insights into the development of assessments in Economics, and you find it useful when teaching your students.

Questions you may want to think about

• How can you use these insights to prepare your learners for exams?
• Do your internal assessments reflect the approach of the exam? To what extent do you want them to?
• What’s the most important or surprising thing that you’ve learned? How might it influence your teaching?