Assessment: exercising validity and re-visioning resits

Craig Barton: Hello, and welcome to Inside Exams. I'm Craig Barton, I've been teaching maths for 15 years, so by this point there's little I don't know about classrooms of kids. But walking into exam halls can still feel a bit like stepping into the unknown. So, in this series I'm seeking out knowledge that we can take back to school, to give our students the best chance when they open a question paper.

As always, I want to know what you'd like to know.

George: I'm George and I'm a physics teacher. I'm keen to write more of my own topic tests in classroom assessments, what factors do I need to be aware of to make sure that my exams are just as valid as national qualifications, can I easily follow some of the same principles?

Craig Barton: Right, this is what I know of validity in the real world. You've probably got a number of mates who track their steps on fitness watches, and they tend to be dead proud of telling you how many steps they've done, you know the type. But get them to count their steps out loud and their watch will come up with a slightly different number.

That invalidates the data they're trying to collect. Basically, the instrument isn't measuring what it's meant to measure. I can feel I've just crushed the dreams of so many eager walkers. Now unfortunately George, that's pretty much the extent of my knowledge on validity. So to find out about some of those key principles that make an assessment valid, I'm going to meet AQA's Director of Assessment and Curriculum, Dave Mellor.

OK Dave, thank you very much for the invite to come to AQA, I'm very much looking forward to discussing assessment validity, and I'm going to start with a deep question. What's the purpose of assessment?

Dave Mellor: The purpose of it is to test the students on the content of a specification in our case, but to try and find out what they know, and what they can do. And what you want an assessment to be able to do is to indicate to employers what skills and knowledge those students have or to further study for university or whatever. And that's the real purpose of the assessment.

A good assessment should separate better students from weaker students in that subject area.

Craig Barton: Now I'll tell you where I start to get confused because there's two words bombing around with assessment, because you've said good assessment but of course that's very much open to interpretation. So you've got validity on the one hand, and reliability on the other hand, will you be able to just distinguish between the two for us, is that alright?

Dave Mellor: Yeah, validity is about whether or not, and there's lots of definitions of validity here, but for us it's about whether or not you're assessing the right thing in the right way. So that better students at geography for instance, get a better mark and hence, a better grade.

Reliability is about the repeatability of the assessment, and whether or not the questions are marked in a way that is consistent, the test will give you the same results within a reasonable margin of error. And if you think of it as a bit like archery, the validity is, are you hitting the bullseye with your hours. The reliability is, how consistently are you doing that, are they all nicely clustered around it.

If it's not valid, well you might be hitting top left of the target.

Craig Barton: Yes.

Dave Mellor: If it's not reliable, you might be hitting the target, but they're scattered all over the place.

Craig Barton: Nice.

Dave Mellor: And that is a way of thinking about it, if you like, so the narrower the cluster of the arrows is, the more reliable your assessment will be, because the outcomes are the same test after test for the same student, or they're consistent across students of the same ability.

Craig Barton: I love that analogy, fantastic. I want you to image that you've got an assessment in front of you, you or one of your team, how do you go about determining whether it's a valid assessment?

Dave Mellor: So when you're designing an assessment, there's two real areas that you're looking at. You're looking at what the content is that you need to assess, so content is decided for GCSEs and A-levels predominantly by the Department for Education.

And then the boards take that content and they can add to it in some cases, but ultimately the core content is determined for us. And then the assessment objectives, and sometimes I think there's misunderstanding about the assessment objectives are for. Sometimes people ignore them completely -

Craig Barton: Yeah I've been there.

Dave Mellor: Sometimes people teach the kids about the assessment objectives, and whilst they're not saying that's necessarily a bad thing, I'm not sure it's a good thing either -

Craig Barton: Oh that's interesting –

Dave Mellor: It depends on your students and so on. Because all the assessment objectives are an indication of the skills that those students need to be able to demonstrate.

Craig Barton: How does that feed into validity then, so once we're aware of these different assessment objectives, if you're either creating an assessment, or you're looking at one, how do these assessment objectives determine, or perhaps invalidate the validity of the assessment?

Dave Mellor: With validity, the really important thing is, are you assessing one what's on the specification, so it's got to match the content, and sometimes that's a bit grey. But two, the assessment objectives themselves, they always have a weighting attached to them. And they're the same across all the boards, so that you have comparable demand of assessments.

Because if we had loads more AO1 than AO3, then our test would be easier than a test by a different board. So when you're designing your qualification there's a number of things you have to do. You have to say, how much assessment do I need to cover all the content.

Now, in an ideal world, for the most valid form of assessment, you would want to cover all the content every year.

Craig Barton: Yes absolutely.

Dave Mellor: But you can't do that for GCSE, you'd have GCSEs of 10 or 20 or more hours of assessment, so how do you sample across the content so that you get a decent understanding of the kids knowledge across the whole piece, and the student can show successfully what they can do, and what they know, and so on.

And that will tell you how big your test is, and how many marks you need, and what sort of questions. And then once you've got the size of your assessments, you then need to work out, well how much of that assessment is AO1, how much is AO2, how much is AO3. And different specifications have different approaches to that.

So a specification like science, the questions are all different every year, and they're assessed different combinations of assessment objectives together, like mathematics as well. For subjects like history, the structure of the assessment is the same every year.

But what changes is the context and the content that you assess. So for history there's always a relatively short question at the start, that allows the students to engage with the examination and get going. It's always on AO1, there's a source with it, there is a sort of structure there. And that helps the students, because there's an element, it's sort of good predictability, they know what to expect.

In the science they don't know the order that things will appear, and then they don't know the detail of the structure, but they know the types of questions they'll get. But what you don't want is bad predictability, and the bad predictability is, actually I know what content is going to come up, I can second guess the content, I can see a pattern.

And we work really quite hard to make sure that we don't create that bad predictability, and yet we still have to assess the whole content over the life of the specification, and it's a juggling act. And then you start to look at, well what does that mean for the demand of the assessment.

Because you need to make sure the demand of your assessment is not so high that your lowest ability students can't really engage with the assessment. Because if you are a weaker student, and you're coming into a subject, and you can only answer the first question, and then can't do anything else, what are you doing for the next hour and a half, it's demoralising.

But equally, you need it to be demanding enough that you stretch and challenge the most able students. For GCSE that's quite hard, because you have a very broad range of abilities. And for some subjects like maths and science we tier papers to accommodate that.

For other subjects like English, we differentiate by what the students can show and can do on some of the essay type questions, in literature for instance. But you don't want to combine say, in science, assessment objectives that are more challenging with really hard content in too many cases.

Because otherwise all of a sudden you've ran up the demand of your assessment, and it will be much harder than last years. In an ideal world you want the assessment to be a similar demand year on year. Because if the assessment changes over time, then your grade boundaries fluctuate over time, it's harder for teachers to know how their students are doing, when they use them for mocks and practices.

So you want to try and keep that consistent level of demand.

Craig Barton: So how do you validly assess the full grade range, 1–9?

Dave Mellor: It really depends on the different subjects and the different history that they have, and different ways of doing things. For something like English literature for example, the questions are designed such that every student can have a go at them, and you differentiate on the basis of outcome.

For something like mathematics, or sciences, we often ask questions of varying demand throughout the paper, and generally we tend to ask easier questions at the start, and the harder ones at the end. But it's not, it's not like a universal progression throughout the paper.

Because otherwise students will get to a certain point and think right, I couldn't answer that question, I won't carry on.

Craig Barton: Absolutely.

Dave Mellor: So, you know, in a GCSE… I was helping my daughter do her maths homework the other day, trying to do a few questions out of a higher tier maths paper, and we got about halfway through and I was starting to struggle. But I said, well carry on because there'll be some questions further on where at least you might be able to do, or we might be able to do, part A or the first parts of the questions.

So whilst there is a sort of ramping of difficulty of questions throughout, it's not universal and we try and make sure that every students encouraged to get to the end of the paper, because they may pick up marks.

Craig Barton: And it's important the kids know that isn't it, as well, because again, I've had kids exactly like that who will get to question 12, can't do question 12, what's the point of me looking at question 13, and so on.

Well already I'm sensing just how complex this process is of putting together these assessments. I just want to go back to that idea of sampling the content, because as a maths teacher, I've always been there where, when it was in the old specification, there was a bit of predictability in there, there was some fairly safe bets that were going to come up each year.

So you'd definitely make sure you'd focus on those. And if it paid off fantastic, your kids probably did a little bit better than maybe they should have done, for want of a better phrase. If it didn't pay off, well then maybe they did a little bit worse. How do you sample the content because it's a massive part of the validity of this, isn't it?

Dave Mellor: Yeah, the challenge with sampling content is, how important is some of that content. If it's some underlying fundamental principles, then that's pretty much guaranteed to come up every year and you would want to include that.

So there will be some content that basically says I always appear, it might appear in different forms, but it will always appear.

Craig Barton: And is that literally kind of a list of content and these are kind of green ones that are definitely coming up, is it?

Dave Mellor: Pretty much, yeah. So we often will sit and try and map out in rough, at least, what does the content look like over say a three or a five year period. Which topic areas do we want to assess? Because the last thing you want to do is end up with the assessment in Year 5 that's got all the little obscure bits of topic, that are not necessarily fundamental to what you're teaching, and they're more peripheral and so on.

So you need to try and map it out over time, so that you don't end up either over assessing something, or under assessing it. And we have sort of assessment grids, which basically list all the content down the side, and which paper it's going to appear in, in which series that we map out.

So for one series for say GCSE combined science, each paper has an A3 grid of content coverage and skills coverage that we balance. And we'll often say we'll assess this content on this assessment objective in this series, and then marry them up.

And then the clever bit, and the really creative bit, and the bit that assessment writers are brilliant at, and most of them, or many of them are teachers or ex-teachers, is then saying, how could I make that a really good question, how could I make that engaging.

So we had a chemistry question on A-level about biodegradable plastic glass and the chemistry behind that. And inspiration struck the question setter at a beer festival where these biodegradable glasses were being used.

And he said I can use this for a chemistry question, flash of genius, and really innovative and clever and a really good AO2 type question. So that's the clever bit.

Craig Barton: So I appreciate that validity is a super complex idea or concept, I'm just thinking about whether I'm creating an assessment, or I'm looking at an assessment. So I've got in my head now that there's got to be this balance of assessment objectives, I've got that there's got to be this coverage of the domain of the content that needs to be assessed.

I've got this balance of predictability, it can't be too predictable, but it can't be so bizarre that when the kids sit an exam, they've never seen anything like it before. Is there anything else that, kind of a general, give a general idea that this is a valid assessment?

Dave Mellor: So there are some sort of more basic rules that you can look at. And we talk about construct, the construct is what is it you're trying to assess, what's the test. And what you want to try and do is avoid construct irrelevance. And construct irrelevance is just basically your test is assessing something that it isn't supposed to assess.

So in a mathematics paper for example, if you have really wordy questions with complex language and so forth, actually you start to test the kid's ability to understand the language as opposed to can they show that they can do the mathematics.

And language is one way of creating construct irrelevance. You might ask a really, really hard question that only 10 percent of the students are going to be able to answer. But you should never ask a question that only 10 percent of the students can understand.

Craig Barton: That's nice, yes.

Dave Mellor: Because it's not, the question is just telling the student what it is they need to do. If it's not clear what they need to do, then that is not assessing the right thing. It's construct irrelevance.

Craig Barton: And is, I can imagine that the literacy or the reading demands are the biggest example of this, are there other examples?

Dave Mellor: Yes, so use of diagrams that they, or pictures or maps that they don't actually then need to answer the question. Because often the students will go, oh why did you put that in there, I can answer this just from my own knowledge, I don't need to refer to the source material to be able to answer it.

Is there, for want of a better word, cultural capital, so for something like Shakespeare, you have to understand some of the context in which the plays were written, and that cultural capital is essential to enable you to answer the question.

But if you ask a question, let's say there's a piece of source material for a GCSE English. And you put a question in about, or a source about fly fishing, or a gymkhana or something, then how many students are actually going to have any real understanding of what these things are.

Because they don't have this cultural capital to actually access these things.

Craig Barton: We’re building up quite a list here of things we have to consider when we're talking validity and assessment. Is there anything else on that list that we need to be aware of?

Dave Mellor: So one of the aspects of things like language, is actually not just the context of the language, but whether the words used are appropriate. We publish lists of command words for all of our specifications, and we publish what they mean, because sketch in art versus sketch in maths are both legitimate command words, but you would end up with a very different result if you asked for the same thing.

So we try and be quite specific about what they are. And where possible, we try and be consistent. I think the other area is around ambiguity of the tasks. You have to try and put yourself in the shoes of a 16 year old, 18 year old, when you're writing the questions.

The use of language can be different, but also life experience can be different. And really what you want is your task to be as clear and as ambiguous as possible. So if you're asking a question and you're looking for say three reasons why something works like this, why not tell them, give me three reasons why this works.

Because then they know exactly what they're expecting. Now the fact that there might be three marks attached to it would probably give them a good indication. But be really specific and be really tight and try and make your tasks unambiguous because if the task is ambiguous, and the students answer in an unexpected way, you could look at this and you can say it's not what we originally intended, it's not what the design was intending to do. But actually, I can see how they're read the question to respond in this way.

And it would be very unfair not to award marks for what is a good answer to a question that's been interpreted in a different way. So what we then do at the standardising process is actually make adjustments to the mark scheme to accommodate appropriate answers. It maintains the validity to a degree, but you've not assessed the thing you thought you were assessing, you've assessed something maybe a little bit different.

But actually, it's fair to the students to reward them for that. You design your specification to try and ensure it's valid, you design your assessment structure to try and ensure it's valid.

You design your marking to make sure that the examiners all understand what the standard is that's being applied, and what the right answers are and what's not acceptable. Where's the boundary between one mark and two marks, all of those sorts of things.

You then go on and monitor the marking and try and protect that validity to ensure they mark at the right standard. Your award, where do you put the grade boundaries, impacts on the validity.

And then what you do post results, and all the way through this chain, it's a bit like, you know in the westerns, the old western movies where someone's house catches fire, they have this bucket chain, they're passing buckets from hand to hand, trying to put the fire out.

And the aim is to get as much of the water as fast as possible from the well to the fire. And if you slop the stuff out then it's gone, the waters gone. Well validity is a bit like that. You can start with the perfect design, but if at any point, you manage to lose some validity for some reason, you can't get it back in, it won't go back in the bucket.

You can try and ameliorate it, but it doesn't correct it. Validity we think about as an end to end piece, all the way through the process. If we do this, what's the impact on the validity of the assessment.

So it might even be, actually quite a few kids have misunderstood this question, but the question is unambiguous, and it's very clear, and what they're written is technically not correct. You might say, well we could give those kids the benefit of the doubt. But actually you invalidate the assessment for doing that because you're not separating those kids who really did know what they were talking about, from those who kind of knew, but weren't quite accurate enough.

So decisions you make at, you know, standardising, at marking, can actually invalidate your assessment, even though the design is good. So that bucket chain is something we try and preserve.

Craig Barton: I like that picture of the bucket, and I wonder how do you know you've put the fire out, what makes a successful valid assessment, at what stage of the process can you sit back and think, yeah that works that?

Dave Mellor: I think the proof in the pudding is in the eating. It's only once you've got all the results through at the end of the day, and it's all been marked and you can look at it in the cold light of day and say, did that assessment achieve the things that we wanted it to.

You can put checks and balances in throughout the process to try and ensure that you maintain validity in your assessment. And it's not an on off thing, it can be more valid, or less valid. It's not a binary feature, although we often do talk about this is a valid assessment as if it is some mythical, you know, there's only one way of doing it, when that's not true.

At the end of it all, what you want your assessments to do is, to differentiate, you want to spread the students marks as much as possible, if you spread the marks out, the grade boundaries are likely to be quite widely spaced. And students having a good day or a bad day, are much less likely to be affected by being close to a grade boundary if they're all spread out.

So it's one way that we try and ameliorate that challenge of sampling, where, you know, a student could get lucky or unlucky. And if they're a few marks lucky or a few marks unlucky, if your grade boundaries are far apart, then it works well, and it doesn't matter because they don't change grade, which is, at the end of the day, what's important to the student.

Craig Barton: Well this has been an absolute revelation for me, I didn't know much about validity, I think I'm a bit more clued up now, so thank you so much for your time.

Dave Mellor: No worries, thank you.

Craig Barton: Clearly, a huge amount of work goes into making sure as much water as possible stays in the validity bucket. However, not all students achieve what are considered good grades. And this means they may have to resit in order to progress. So, how can we encourage them to try again with a resit.

Julia Smith is a former maths teacher, and author and now works as a teacher trainer. She has some pretty innovative thoughts about motivating students to give assessments another shot.

Julia, I am so pleased to be talking to you today, because I think, so I've been teaching 15 years, and I think for most of those years I've routinely failed to teach my GCSE resit students particularly well.

And I don't think I'm getting much better at it, so I'm hoping your going to help me here. So kick things off, what are some of the challenges of teaching students who need to resit their GCSE?

Julia Smith: I think Professor Susan Wallace has kind of coined this phrase, they're Rhinos, they're really here in name only, they're a bit thick skinned, they've got some attitude as well. They go to college or they stay on at school to do other things. And the resit happens to them as well, and it's not very welcome in a lot of cases.

So the challenges, motivation and value in that qualification, National Numercy have hit it spot on the head, when they mention about the student valuing the qualification, believing that they can do it a second time round with you, and then they might put some effort in.

Craig Barton: Fantastic, well we're going to dive into some practical strategies later on. Just thinking about GCSE maths in particular, do you have any stats on how many students are resitting, and how many actually pass?

Julia Smith: Yes, so this year 143 thousand students resat their GCSE, mandatory, so they would have had grade 3s or below. And out of that 143 thousand, 25 thousand achieved a grade 4 out of their resit qualification.

So on a macro level that's absolutely brilliant, it was six percent up from the previous year in terms of June entries. So on a macro level 25 thousand students gained the grade 4, which is that Willy Wonka golden ticket to riches and success, like no other qualification.

That's absolutely brilliant, but on a local level, the national outcomes are 17.8 percent, which means in a class of 20, you'd be lucky if three or four pass.

Craig Barton: Jeez, and of course that's spread out and you're going to get some colleges where maybe no students are going to pass.

Julia Smith: Yes absolutely. And it's a very tricky situation in terms of motivating the staff to keep them going -

Craig Barton: Yes I've been there, yeah.

Julia Smith: Getting the students involved and attending, that's one of the major issues. And then there's the whole kind of keeping the momentum going, because they have to keep going subsequent years and years and years. So we had, there was one student this year, Lauren, who passed eventually got her grade 4 GCSE after nine attempts.

Craig Barton: Nine.

Julia Smith: Nine attempts, and that, you know, all credit to her, that's fantastic but she needed it, she needed it to go to university to do occupational therapy. But we do now see quite commonly, students that have sat it seven times, and that's a real challenge.

Craig Barton: I tell you what Julia, I'm on a bit of a downer now about this, so pick me up a bit, why are you so passionate about it?

Julia Smith: Because I think there are thousands and thousands of students out there for whom that grade 4 will open so many doors, will absolutely open so many doors, and I think they need to be championed. I also, there's so much complexity to it that really becoming a specialist in the resit arena I think has allowed me to have some significant impact.

And now we're very, very fortunate to be supported by AQA with the five Rs approach, which is now being rolled out, we've got 88 colleges taking part in this massive trial, looking at ways in which we can help the students, primarily, that's what it's all about.

But also, give the teachers some really good tools so that what they're delivering doesn't look and feel and sound like school. Because if it does, you're just going to get the same result.

Craig Barton: What are those five Rs?

Julia Smith: The five Rs, it's a lesson structure, the five Rs are Recall, you revise one topic every hour for 15 minutes and you talk about it, so you revise a topic that's the third R. You repeat that topic but just looking at exam questions, so the revise part is talking about it, discussing it, looking at things, remembering what you can from before.

The repeat part is looking at that same topic, but just focusing on three or four exam questions, and the final part is about exam readiness because exam technique is really key as well for a grade 3 moving to grade 4.

They make so many numpty errors, and those are the quickest ones to sort out for a resit teacher, so.

Craig Barton: That's a technical term there isn't it, the numpty errors –

Julia Smith: Yes, the numpty errors, it's not the student being a numpty, it’s the numpty errors that they make, and everyone makes those.

Craig Barton: And these five Rs, is this essentially a lesson structure?

Julia Smith: It's a lesson structure, an hour structure, so you have, it's a spiral so it repeats again, so if you have three hours delivering, you go through the five Rs one cycle, then again. It's all based on attention span, based in the [interleave] practice, nothing lasting longer than 15 minutes, and you're keeping coming back to reminding them, constant reminders over time.

Craig Barton: This is great this. And part of me is fuming because I haven't heard of these five Rs before, but I'll tell you a couple of things I love about this. I don't know about you Julia, I love a structure and a routine, I love something to follow.

And I think the kids like it too, they know what to expect, it's consistent and that brings me to the second thing I like about this, and that is it will feel different to what they did at GCSE. And it's now OK, we're not just going to repeat what we've done for the last five years, this is GCSE resit, here's a structure in place, it's different, and it's designed to give you the best chance of getting that elusive grade.

Julia Smith: Yeah, the whole premise of it was about revision because if you think about the word revision, you're revisioning, you're seeing things differently to first time round, because you didn't quite get it all.

Craig Barton: I like that. Revision. I like that.

Julia Smith: Yes so it's a whole revision year, because they've seen fractions since Year 3, Year 4, they've seen mean median and modes since Year 5, Year 6.

Craig Barton: Do you know, that's fascinating that, I think all my teaching of GCSE resits has been a few minutes of revision, they've never seen it before, so I just crack on.

Julia Smith: No, no, they've seen everything, everything they've seen, they've seen enough, there's no, you know, place for new topics until the summer term in my kind of viewpoint. They've seen enough to pass, it's just you need to sort out what they sort of know, carry on with what they definitely do know, but it's about that point about practising until you cannot get it wrong.

Craig Barton: Yes I like that.

Julia Smith: Rather than practising until you get it right. And also, the fact that they have to do some daily maths. So there's a big push on giving them the resources within the five Rs project that we have running.

Craig Barton: I've been going wrong with assessment you know, because again, the only thing I'm looking at when I, if I'm honest, when I start planning to teach a GCSE resit group, is their GCSE results, so I've got their grade, and maybe I'll get from whatever awarding body they've done, a little breakdown of what questions they got right and wrong.

But of course, that's just a single exam that samples from a very wide domain, and if I'm starting to base my planning just on that, and you've got the fact that kids of, in that class, some kids have nailed that particular topic according to the GCSE, some kids haven't and so on.

So how do you, what assessment data do you use and how do you use it?

Julia Smith: We're looking at the basic skill test, so there's three papers, nine questions, one question on addition, subtraction, multiplication, division, fractions, decimals, percentages, scale and ratio. Those are the nine basics.

Craig Barton: Good knowledge there, you did good there.

Julia Smith: Well I wrote them. So I should remember them. The nine basics underpin the curriculum, if they cannot multiple they're going to struggle with area, and if they can't subtract, they're going to struggle, etc.

So three basic skills tests, one is very easy nine questions. Two is slightly harder, nine questions. The third one is another nine questions, so there's just 27 questions, do them fairly early on in the term. There's a full set of teacher's notes as well, that I wrote, which will tell you if they're getting this wrong, it's because of this.

Craig Barton: Yes. How long do the tests last?

Julia Smith: 10 minutes.

Craig Barton: 10 minutes and are they sitting them back to back or over three separate -

Julia Smith: It's entirely up to you how you want to present that.

Craig Barton: What would you advise?

Julia Smith: I have delivered it, and I've done one per session, so I had three one and a half hour sessions, so I'd do one in the mix of things in that one and a half hour session. So very manageable, but they give you a huge piece of information.

If they've got a grade 3, they will have some gaps or some misconceptions in those nine basics and those are the first ones to sort out. And when we trialled it with AQA early on, it doubled outcomes. Doubled outcomes for a number of providers, which was kind of, whoa, what's going on.

Craig Barton: Jeez, this is working.

Julia Smith: This is working, really working, so I was banging on about it even then, with the huge support from AQA I have to say, and then some schools started saying, oh we're using it for Year 11, it's brilliant, bottoms hit Year 11, it's great.

Craig Barton: Yeah of course.

Julia Smith: Some people have morphed it to six Rs, because they add a reflection point in. So it was very adaptable, but it was just a very simple approach. But it's using things that didn't look, feel, sound like school.

Craig Barton: Let me ask you this, what happens if you, you've identified these nine areas and you've got a class of 20, 25, 30 kids or something, and its clear that 20 of them multiplication is a gap. But for 10 of them it isn't, what are you doing there, are you putting those 10 off on something else or are you saying, well actually multiplication, we need to really make sure you've hammered this, because again, this happens throughout all schooling but particularly in resits, kids have got different areas of strength and weakness, how are you managing that?

Julia Smith: Yes, so it really is about introducing the student padlet early on, so if they're OK and sound and secure in some bits, they've got the mechanism by which they can go off and do something.

Craig Barton: Yes and is that within the lesson, are they, got it, they need a device so they, right -

Julia Smith: Yes and in sixth form that's generally a lot less of an issue than it is in Year 11. It is about keeping it very simple, working on what the chief examiner's reports are telling us, but we know what they're struggling with, working on that so that they have got ever opportunity just to do loads of maths. Just do shed loads of maths, simple.

Craig Barton: Got it, I tell you what, a picture in my heads forming now, I've got a structure now, I've got some assessment data to go along with it. For me the missing piece here is getting the kids onboard, the motivational aspect. What's that early messaging that you're giving to the kids because they're coming in in September, they failed their maths essentially, as you say, a lot of them, they're Rhinos, they don't want to be here, they'd rather be somewhere else.

That initial messaging is so important isn't it, what are you saying to them?

Julia Smith: So the starting point I've always loved the National Numeracy website, and there's a fantastic video on there, I could even probably recount it to you, we've got a problem in maths, it goes on, and it's just three or four minutes long, and it sets the scene.

So you set the scene with students and I play it to everybody and everyone who'll listen. And it tells you in half the adult population have the maths skills of an 11 year old, of primary school students, so it talks about the wider picture in society.

And then there's some bits and pieces in the resources, so things like, we've got the video where the boy takes the slip, his pass slip home and his dad howls crying in the kitchen. It's a very funny video, again, three or four minutes long, but it just is a very good talking point about putting students into the moment of when they pass and get their exam slip next time.

Recognising that, well I do things a bit differently, it's not going to look, feel, sound like school. You've got a part to play, it's just revising, we've got to revise for the whole year, you've seen enough, it's just we need to sort out what you don't yet know and sort out some of the easy stuff that you're struggling with. And that's enough to move them from, they've already got a grade 3.

Craig Barton: You mentioned before a particular success story of a child who'd sat this nine times and eventually passed, it's a different kind of success isn't it, when a child's repeatedly struggled at something, and eventually gets it.

Versus a child who, for example, was on a grade 7 all the way, and then managed to get a grade 8, it's a different feeling for the child, and a different feeling for the teacher as well. Do you have any other favourite success stories to share?

Julia Smith: I think it's the fact that very early on when I started teaching GCSE in maths, I got the Monday night GCSE resit class, which was mainly adults. So, for adults again, a lot of this isn't revision, a lot of it is kind of they're fitting in their jobs and they're fitting in everything else around it.

So to give them a mechanism by which they can do a lot of work for themselves very easily, seemed a big thing. And actually one lady, she got really, really angry with me, and I don't know, I was a very green teacher then, and just say well if you can't add fractions, have you seen this method, if you can't multiply with decimals, have you seen Napier's Bones. And half an hour of this she'd had enough completely of me.

And I said why are you so annoyed, you know, what have I done, and she said it's not you, she said, I'm 40, I want to be a nurse, I can't get onto nursing without my GCSE, a C grade as it was then, she said no one has bothered to show me another way, I thought there was one way to do things, I always thought I was thick.

She'd never been able to help her children at primary school, she was a single mum, wanted to go back to work and she got so annoyed because she felt I'd done in half an hour with her, what someone else could have bothered with. And that really resonated with me. So I was playing around with alternative methods a long, long time ago, back in the day.

And that really stuck with me, it really did stick with me.

Craig Barton: Well that is superb, Julia, I'm not just saying this, this has been an absolute game changer for me this, because I feel empowered now, I've got a structure, I've got a strategy, I've got my messaging, you put those together and it's got its best chance of working, so.

Julia Smith: Yes you won't get 100 percent success with all of them, but if we can double outcomes, I think everyone would be very, very happy with that.

Craig Barton: Absolutely, Julia thank you so much for your time.

Julia Smith: Thank you Craig.

Craig Barton: As I'm sure you've probably already gathered, this idea of literally revisioning the same content for resit students, has blown my mind a little bit. So that's certainly something I'll be going away to think about.

Dave talked about standardisation being particularly important in creating a valid assessment. Our last episode was all about standardisation, so head back to your podcast feed to listen to that one.

I'll be back in two weeks' time, helping to answer more of your questions. But in the meantime make sure you rate, review and subscribe to the podcast. Do also join the conversation on Twitter using #Inside exams. Until next time, goodbye.