assessment – frogphilp.com

Lessons Learned #2: Don’t colour the numbers

Here's some numbers with pretty colours... — Here’s some numbers with pretty colours…

I’m a big fan of conditional formatting. Look how you can put a number in and the whole box changes colour! Amazing!

I had done a lot of conditional formatting by the time Ofsted arrived last June. But Ofsted don’t want colours. They just want the numbers. And by the time they arrived, whilst I controlled the colours, I had no idea what was going on with the numbers.

The core of the problem is that what I thought was the assessment system, was actually only the data capture system.

There are a whole load of things that need to happen before a number appears on a spreadsheet to indicate how well a child is doing. The child needs to be taught something. They need to then really learn that thing so they can apply it ways that aren’t in the routine lesson in which they were originally taught the thing: like a test, or a piece of extended writing. They need to evidence they have learned the thing too. Tests are fine, but remember that books are king, so the evidence has to be in the books. After that you can quantify how much the child has learned the thing. Then you can put that quantity into a spreadsheet. You can total it, average it, filter it and apply a whole load of other formulae to it.

And if you’re like me you’ll want to colour it.

That last part is the data capture system. But the whole paragraph is the school assessment system.

Somewhere in that system, you will have defined the criteria ‘good’ and ‘bad’ looks like. Unfortunately I wasn’t in control of that either – I just coloured the numbers.

By the time the numbers reached the inspectors, the criteria for ‘good’ colours had managed to exceed the criteria for ‘good’ numbers. This meant that the colours looked a lot better than the numbers. But the inspectors didn’t have the colours. And with the criteria that we had given the inspectors the numbers did not look good. And when the numbers don’t look good, it’s easy to go and look in books and find evidence that proves that the numbers were never so good in the first place.

With the new assessment without levels and every school building up their own system from scratch, it’s vital that school leaders don’t lose the big picture of what their assessment system actually shows. If you say to inspectors “our children are making better than expected progress, and look, our assessment system shows it,” the inspectors will simply pick up some Year 2 books and some Year 4 books and see if there is evidence to show the ‘better than expected progress’.

If that evidence doesn’t exist, it doesn’t matter how much you have coloured the numbers, the assessment system is broken.

The problem with a best-fit approach

Scattergraph — Here’s a handy graph, just ready for a best fit line to be drawn through it

It is not often that I read a 50-page government-commissioned document and get wildly excited. But that is what happened when I read the “Final Report on the Commission on Assessment without levels” (September 2015).

I found pages 12-18 particularly inspiring as the commission explains the rationale behind why levels are bad and ‘not-levels’ are good.

This statement was pertinent:

Levels also used a ‘best fit’ model, which meant that a pupil could have serious gaps in their knowledge and understanding, but still be placed within the level.

And this one:

Levels were used to measure both end of phase achievement and lesson-by-lesson formative progress, but they had not been designed to fulfil the latter purpose, with the result that formative assessment was often distorted.

Just as in yesterday’s post, when I tried to explain a character flaw that had held back my practice, here was a practical example of something I was doing day-to-day which was not actually any good.

As a key stage 2 teacher I had experienced this over many years. Children came up to my phase ostensibly as ‘a level 2 child’, but the best fit approach meant that while they could add a pair of 2-digit numbers with the support of a 1-100 number square, they had no idea of any other strategies for doing the same thing, nor really any sense of the the size of the numbers they were dealing with, nor the purpose of the equals sign.

And likewise I must have been doing the same to my secondary colleagues for years: children from key stage 2 going up to key stage 3 being able to scrape enough marks to get a level 4 in a SATs test, but not with the necessary number or problem solving skills to really go any further. In fact at my school, analysis of SATS shows me that number skills have always been high, but problem solving really low – the children have been able to scrape through with good routine number skills, but lack the fluency to really excel in maths.

What is even more insidious about the ‘best-fit’ approach is what it does to ability groups within a class, particularly low ability children. But that’s tomorrow’s topic.

Image courtesry of: https://upload.wikimedia.org/wikipedia/commons/a/a2/Scatterplot_r%3D.24.png

Remember that you are dust, and to dust you shall return.

Showing Progress in Fractions

One of the great things about teaching fractions to my Mathemateers group is that they knew pretty much nothing about them. This meant that whatever they learned would show oodles of progress – always good for impressing line managers.

(Not that it matters in this circumstance. I am the booster (remedial) teacher for the Year 6 group, therefore reporting to the Year 6 teacher who in turn reports to the Key Stage 2 Phase Leader. However I line manage both these people, so it’s less of a line and more of a circle…)

The assessment system we use showed that all the children in the group struggled with fractions. For example, Sarah‘s profile in ‘number’ looked like this:

Sarah's lowest 'ticks' were in Fractions — Sarah’s lowest ‘ticks’ were in Fractions. My school used the Incerts assessment system.

Of course when I asked Sarah some questions, it transpired that her prior assessment in fractions was, shall we say, over-ambitious, in that the assessment system said “she is developing the ability to use simple fractions that are several parts of a whole”, when actually she couldn’t do this question from ‘Recognizing Fractions 1‘ in the Khan Academy (which I have written about previously).

Most of the children couldn't do this question when we started. — Most of the children couldn’t do this question when we started.

Of course there’s the whole issue about performance and learning here. Sometimes children really do know something, but for whatever reason they don’t show it. This is performance. Performance variation is one of the main reasons for the difficulty in carrying out accurate assessment in education.

But for me as a teacher, this is great. I can now teach some stuff to the children and show great progress. And that’s what I did. Pretty soon the children had motored on to ‘Recognizing Fractions 2’ and even managed to do questions like this by the end of the first week.

No I’m not saying this is world-record teaching, but it does show progress. And what’s great is there’s an image, you can talk about it with the child and then the child has to write down the answer in fraction notation. It’s the perfect move from the Pictorial to the abstract. The downside, if you only use the Khan Academy is that children don’t write down what they did in their books and so their progress isn’t there for external visitors. And that’s not good if you’re a very book-scrutiny focused kind of school.

What would be great would be if we had already moved on to the New National Curriculum. However, as you well know, Year 6 are still working to the old curriculum. You see Incerts have just released their tracking system for the new curriculum and it looks fantastic. Here’s a picture of the ticks I could make about Sarah’s fraction learning:

What the new assessment for fractions looks like in Incerts.

However I can’t use that for my current group because they’re in Year 6. Nope. I’m going to have to cope with the learning that’s actually happened in the children’s brains and their SATs results in a few weeks time. Speaking of that, the final tool I’ve used to show progress is the Testbase tool that is a store of all the previous SATs questions. Sounds boring, but it’s really, really handy at the stage of the school year when teaching in Year 6.

If only there was a tool like Khan Academy

So I was speaking to an inspector a few months ago who was trying to look a bit more deeply into my schools maths data. She asked out loud, “couldn’t you make a system that finds out how well children are doing in each individual area of maths, rather than these overall numbers?”

Broadly speaking, that is the problem with data in schools. There’s always the danger of there being so many interpretations and approximations between the numbers that come out of the system and what’s actually in a child’s brain that the data becomes meaningless. Here’s how assessment works:

we decide what children should be able to do by particularly ages or stages and write it down in sentences.
we assess how well children can do the things we wrote down.
we turn those assessments into numbers.

Sometimes those assessments are called tests, at other times they are called observations. Either way it’s more or less the same process. However, quite often as teachers we get distracted and over-focused on the last stage of the process – on the numbers and less on the ‘what the children can actually do’ part of it.

This is where Khan Academy is brilliant. I’ve been using it this term with my Mathemateers group, and even though it doesn’t entirely match with the UK National Curriculum, it does help spot the gaps that children can’t do and provide the children with ways to practice skills that they are still shaky on. I also like the way I can focus the children on a particular skills at a time so that I’m not having to teach each child individually. For example for a few weeks I was focusing on fractions, so I directed children to activities that helped them visualise and practice fractions. I used Google Classroom quite often this – I would post a link in the Google Classroom assignments that would take the children directly to the Khan Academy challenge I wanted them to do.

Why Khan Academy fits in to the inspector’s question is that it gives a brilliant assessment of how children are doing in each area. For example, when setting my fractions challenge I mentioned earlier, I could see that one child had already mastered it, another was struggling at it and the rest had never tried it – it meant I could focus the challenge precisely on what I wanted the children to learn, support the child who was struggling and set a harder challenge for the child who had already mastered it. Ace.

What AfL is for

Rob Coe recently posted an interesting essay about how AfL might well be over-rated.

I broadly agree. And of course I’m in no position to argue against him – my experience only relates to the impact of AfL on 12 teachers in a small primary school of around 240 children. However, my experience of AfL has been really positive and I’ll explain why…

Everyone knows that there are only 4 things that improve teaching, and one of them is subject knowledge; the purpose of AfL is to increase subject knowledge.

Teachers have curriculum strengths and weaknesses – this is particularly apparent in the upper reaches of primary school, where the required qualification in English and maths for a teacher is a ‘C’ grade at GCSE. There are significant numbers of children at this level who may be working close to that level, hence the teacher’s subject knowledge may simply be not high enough to meet the needs of the students.

This is where AfL comes in. Assessing the children closely against rigorous banks of knowledge statements such as those found in the APP materials for English and maths, means the teacher discovers holes in their own subject knowledge – they find out what their students can do, they can see the next steps and they can determine whether they have the subject knowledge to teach those steps. At this point, if they don’t have the subject knowledge, it’s either time to panic, or seek help from their senior colleagues.

It is exactly at this point that things go wrong – senior colleagues (in other schools, I might add) are often keen to tick the AfL box rather than address the underlying problem. Unfortunately it is far easier to make things look like AfL is happening than to actually increase the subject knowledge in your staff – this involves a level of skill and compassion that is beyond many senior leaders in our education system. In this culture, rather than seeking the improvement they need, teachers who need to develop their own subject knowledge will develop all sorts of strategies to conceal it. In fact one of those strategies is writing the letters WALT and WILF on your whiteboard – a point that Professer Coe alludes to.

It is the culture of the school that makes a difference here. In my school we are all learners and my headteacher repeatedly reinforces a ‘no blame’ culture. Only yesterday, my year 6 teacher (whom I line manage) was teaching me what modal verbs are. Similarly we are all happy to educate each other so that we increase each other’s subject knowledge. We have found systems such as APP and Incerts (an online assessment system based on the old National Curriculum) really useful because they have helped us identify what we are good at teaching and which areas we still don’t know much about. We use them as assessment for learning, but really that means increasing our own subject knowledge so we can teach better.

On the teaching of handwriting

I’m writing this both as a parent and as a follow up to a previous post concerning roleplay in schools. You may ask what is the relationship between handwriting and roleplay? Well, I suppose I’m asking a wider question – what are schools really for?

When my son was in Reception, my wife and I were concerned about his letter formation. We told the teacher about it. She assured us it would be alright – he’s a boy after all – he just needed some time. She gave us some photocopied sheets of letters to practice at home .

When my son was in Year 1 we were concerned about his handwriting. We told the teacher about it. She told us that it was early days and he just needed some work on his hand strength and motor control.

When my son was in Year 2 we were concerned about his handwriting. We told his teachers. They told us there was still time. The important thing was getting him to write in sentences for his SATs. He was assessed at level 2B in writing, despite being level 3 in everything else. His handwriting had held him back apparently.

When my son was in Year 3, the teacher told us she was concerned about his handwriting. She implied that we hadn’t done enough as parents and we should be supporting him more at home. We too were concerned about his handwriting, but now we learnt that it was our fault, we gave him writing exercises and paid for piano lessons to build up his hand strength some more.

When my son was in Year 4, the teacher told us she was concerned about his handwriting. She put him in a small group so that during times when the rest of the class were doing something he was already good at (like reading, or maths), he could practice his handwriting. By the end of of Year 4 we were less concerned about his handwriting.

When my son was in Year 5, the teacher told us that our son was the best writer in the year group, but he did sometimes forget his capital letters and full stops. She didn’t mention his handwriting.

As parents we made the mistaken assumption that during his first years at school he would be taught handwriting.

As a teacher I know it is easy to assume that things like handwriting will just sort themselves out.

As a child educated in the 70s, I went to a school that, whilst it hadn’t completely sold itself out to ‘discovery learning’, it didn’t teach things like pencil grip or handwriting. My handwriting is pretty dire as a consequence.

In following up that post I mentioned on roleplay, my question is what are schools for? Should they ensure that children are brilliant at handwriting? Or should they make up for the lack of open-ended play that takes place in the modern home by focusing more on roleplay? Do I sound too much like Michael Gove if I suggest that schools should focus on academic skills such as handwriting, reading, mathematics, to the detriment of play?

In fact, I don’t think this is what has happened in the case of my son. I don’t think it was a war between play and academia. I think the school focused heavily on their needs to have my son perform well in various assessments – maths, reading or writing, and unfortunately handwriting is an insignificant part of the writing assessment.

A good week for an inspection?

This week could be a good week to be inspected by Ofsted. That is, if the lead inspector is Welsh.

Some time in the not too distant past my school was inspected at around the same time that the Six Nations Rugby tournament was on. Our lead inspector was Welsh and it was in one of those years that Wales won the Grand Slam.

I remember that at the start of the inspection we had proverbially shot ourselves in the proverbial foot. Although we knew we were doing a good job and had the evidence to prove it, we had toned the achievement section of our self-evaluation from ‘good’ down to ‘satisfactory’. It is not a wise idea to be self-deprecating in these documents. The first thing the lead inspector said was – “so as a leadership team you claim to have moved the school’s achievement from ‘good’ to ‘satisfactory’. I that’s true it would mean that you have inadequate leadership, so I must put you into a category.”

We spent the next two days proving that our achievement was actually ‘good’ and as a consequence came out with an overall outcome of ‘good’.

The point of me saying all that, was that our inspector had given us a chance to get right what we had got wrong in our self-evaluation. I have spoken to other school leaders who have told me that their inspectors would never have given them such a chance. Our inspector was fair to us, generous even, some might claim, in giving us that chance. Was this because he was a nice guy? Is it because all Welsh people are nice? Or was he in a more positive mood because Wales were winning in the rugby at that time? Or could it even be that actually most Ofsted inspectors are actually pretty fair people?

Looking back over the Ofsted inspections I’ve been through – eight in total – I have to say that most of my experiences have been pretty fair. I’ve been praised for good stuff and criticised for under-performance. There are one or two disappointments – like the time when I saw the inspector coming to watch my outdoor PE lesson, but it was a grizzly day – he just stopped in the corridor for a minute or two and then moved on. Or the time when Ofsted came a-calling in the penultimate week of the Summer term. I was teaching Year 6 at the time and my plans revolved around perfecting the Leaver’s Production. Neither the inspection or the Production went particularly well for me that year.

Back to the inspection in question and I happen to know that the lead inspector wrote up his report over the same weekend that Wales won the Grand Slam. He wrote words in his report such as ‘blistering pace’ to describe one particularly good lesson. His report was particularly effusive I thought – colourful even. Did this partly result from his positive mood caused by Wales winning? It could well have contributed something. Yet, he was also fair with his criticisms and gave us good guidance on how to improve the school.

We have to face facts – Ofsted inspectors are individuals not robots. They have real lives with ups and downs just like the rest of us. Some of them will even have favoured teaching styles that they prefer to see. Catch them on a bad week and you might have to work harder to convince them of the job you are doing. But the inspection system is still in the main doing a good job, a far better job than the other main accountability measure – league tables. All league tables do is make schools turn the assessment system into a game to be won or lost, rather than use it for individual students to measure their success.

If you get the call this week, good luck and remember to stay positive. I even hope your inspector is Welsh.

What is Number Ninja?

The ultimate 'Number Ninja' badge. Nobody has achieved this level yet. — The ultimate ‘Number Ninja’ badge. Nobody has achieved this level yet.

Number Ninja is essentially an assessment system for maths that uses badges to reward children for making progress. It’s for children in the Primary age range from 5-11 years of age.

I had become frustrated with some of the maths assessment we were using at my school. I felt they were either too slow or too lenient and they didn’t reflect what I consider to be the golden triangle of maths achievement – understanding, pace and accuracy. For example we were using a ‘99 club’ – several iterations of this exist. It was a good system, demanding recall of multiplication facts and division facts. Each class would do the test once a term and the children would be rewarded with a certificate in an end-of-term assembly, with a class prize going to the class that had obtained the most certificates. However the one we were using allowed 10 minutes for each test, which was just too long for my liking.

We also use a system called Incerts which calculates a national curriculum level for each child based on the number of statements ticked. I find this system both convenient and accurate, but in my opinion the current National Curriculum under-rewards children for mental mathematics skills.

A second problem was inconsistency across the school. While we had two main whole school systems, teachers were using a range of assessment systems within their own classes that didn’t continue beyond that year group. For example some people were using the excellent mental maths assessment developed by Wigan LEA some years ago (get it while you can – this website is no longer supported). Other people used their own times table systems that they had borrowed from other places or created themselves.

What I wanted to do was create a wider system that demanded more from the children in terms of pace but still maintained a whole school rewards system in a big termly assembly – this is useful to keep a high profile for mathematics.

So I did 4 things:

I kept Incerts and 99 Club – there’s no point throwing away good stuff if it’s already working.
I developed a new mental maths assessment system called ‘Grid Club’. This is much more pacey than 99 Club.
I introduced Khan Academy. While I’m not overly impressed with the instructional videos on Khan Academy, I do really like the assessment system that goes alongside it.
I created a spreadsheet to collect all the scores from the different systems and calculate an overall Number Ninja score – this is what I use to award the badges.

Here’s an example of what the Number Ninja spreadsheet looks like. It’s from the Year 3 class. Teachers enter the numbers in the coloured columns and the number on the far right is what the spreadsheet generates. It is colour coded to tell you what ‘Number Ninja’ level the child is on.

Any questions or suggestions about Number Ninja – I would be grateful to hear them!

Staff Incerts Training

incerts1.mp4
Watch on Posterous