I have never much liked the word ‘progress’ as a description of how successful a child has been at school. To me it has a vague, annoyingly fussy and managerial feel which obfuscates the true purpose of schools and, wherever possible, I try to use ‘learning’ instead. Whatever it is called, measuring how much students have improved over time has become increasingly influential on the way in which teachers and schools are judged. Recently, progress as the primary measure of a school’s effectiveness has become formalised and embedded by the introduction of Progress 8.
On the face of it this appears fairer than judging schools on the attainment of exam cohorts. This is because, the thinking goes, attainment is, at least partially, the result of intelligence; a school with lots of clever children is likely to get better grades than one with less intelligent ones and, because intakes are, theoretically at least, out of the control of schools, it would be unfair to judge a school’s effectiveness on these results. Progress seems a fairer measure because children are assessed from the point at which they started.
As well meaning as P8 might be, it is riddled with problems and is actually a very unreliable measure of school effectiveness, which may come to have a damaging effect on the education of some children.
The statistical problems with P8 have already been very robustly covered by others, as has the unpleasant way in which it pits schools and their pupils against each other in a drier, more academic version of Battle Royale. I’d go even further though, and argue that the measure is actually conceptually flawed because it is based on three incorrect assumptions:
The KS2 data used as the benchmark against which progress is assessed is safe.
I have already written lengthily on this here. Briefly, KS2 tests assess the attainment of children in discrete disciplines (English and Maths). They are not tests of a child’s capacity to learn. The grades children achieve in these tests are the result of a very wide range of factors of which their ‘intelligence’ (I use quotation makes here because of the lack of consensus as to what this is) is only one. Using the grade a child achieved in primary school in two subjects is not a safe starting point against which to assess their attainment in another five years later. This is continually misunderstood with most schools (and Ofsted) wrongly referring to students as high, mid or low ability based on these attainment tests.
If we accept these arguments, and that KS2 data is not therefore a safe measure against which to assess learning, P8 loses credibility.
Even if we put aside concerns around the safety of KS2 data and blindly accept averaged scores in English and maths as reliable and appropriate benchmarks against which to assess the learning of pupils in other subjects, further issues remain.
Socio-economic context makes no difference to the speed at which children learn.
I do understand why socio-economic context is ignored in progress measures. Recognising the influence and taking it into account can, of course, lead to lower expectations of certain groups of children, which may embed differences in attainment. This, of course, is why Michael Gove got rid of the Contextual Value Added measure. But we all know it makes a huge difference and ignoring it won’t make it go away; children from poorer backgrounds lack significant advantages held by their wealthier peers and, as a direct result, are less likely to do as well at school. Teachers and leaders know that it is harder to get good results in a school in a disadvantaged area than it is to get them in an advantaged one, which means given the choice they are more likely to choose to work in more affluent ones. When combined with performance related pay, this creates an educational landscape in which recruitment and retention are much more difficult for some schools than others. Of course, some schools in very disadvantaged areas buck the trend and achieve outstanding outcomes (the school I attended was one of these), but using these as evidence that it as easy to run a poor school than a rich ones is to indulge in whataboutery and, in my view, knowingly disingenuous.
Perhaps worst of all, ignoring context means ignoring the issues that most disadvantage some children. For example, white English boys with prior poor attainment are the group least likely to make ‘good progress.’ If we know this then we should accept that a school with a high proportion of these children is less likely to achieve a good P8 score than one with a smaller proportion regardless of the quality of teaching. This should be acknowledged and a system-wide focus on the reasons for this underachievement would almost certainly be more productive than ignoring the issue. Not doing so creates conditions in which schools may be incentivised to reduce their proportion of poorly-performing groups, which is clearly neither desirable nor the intention of the P8 measure.
The difference in value between the different GCSE grades is uniform.
An intended consequence of P8 is that the progress of all students now matters equally. Previously, when schools were measured on the percentage of students achieving Cs or above, a lot of effort was put into children struggling on the D/C border with other children perhaps neglected as a result. P8 should, at least theoretically, end this with the value difference between a new 1 and a 2 being the same, or close to the same, as between a 3 and a 4, or a 4 and a 5. This should mean all children are pushed to improve, not just a few key groups.
The problem with this is, outside of the measure itself, the different GCSE grades are of different value. Colleges do not offer children places based on the progress they have made. Nor do universities, or internships or employers. Life differentiates by attainment, not achievement. This means that schools may actually be quite right to care more about whether a child achieves a 3 or 4 than they do about whether or not a child achieves a 1 or a 2. A 4 might get a child into a college when a 3 does not, whereas it is far less likely achieving a 2 instead of a 1 will make a difference to a pupil’s future opportunities. I suspect that many schools and teachers will, either consciously or sub-consciously, recognise this, which will lead to tension between how a school is judged and what is best for the pupils who attend it.
I worry that the focus on progress for all, as well meaning as it is, may create a system in which what happens inside school becomes less and less relevant to the world outside. Teachers, schools, Ofsted and the DfE may care very deeply about progress but outside education it is poorly understood and typically ignored. I don’t think parents, as much as we might think they should, typically care very much about how much ‘progress’ children make at a school; they care about attainment because they know it is this that really determines their child’s future opportunities.
It seems to me that an important unintended consequence of our preoccupation with progress has been the creation of an accountability measure largely irrelevant to the world beyond the rarefied ecosystem in which it was conceived and incubated. This is, of course, not to say that progress should be ignored; doing this could easily cause schools to teach to the middle and, by so doing, de-prioritise the learning of both lower and higher attaining pupils.
If we really need a system to judge schools against each other, it must be based on credible data, acknowledge the role of context and reflect both achievement and attainment. While I’m happy to be put right on anything I’ve got wrong, as it stands, I don’t believe P8 does any of this.