David Reynolds takes a searching look at the IWA’s report on Making a Difference at Key Stage 3: Learning from Five Successful Schools
The IWA’s study on what makes for ‘effective’ schools for the eleven to fourteen year olds at Key Stage 3 was released at a day conference in Cardiff earlier this year. The attendance of over a hundred people – some great and some good – was impressive in the run up to the holidays. But was the content and the message?
The research does have a lot going for it. It is, firstly, about school effectiveness – about what works in education in Wales – and is only the third of its kind over the last 35 years. England has had – literally – hundreds of studies like this, and probably fifty or sixty countries across the planet now have their own ‘effective schools’ knowledge bases.
Secondly, the report focuses upon an area of maximal Welsh educational failure – Key Stage Three. On the assessment tests at the end of Key Stage One (age seven) and Key Stage Two (age eleven) we in Wales are level pegging with England. By age fourteen, we are very badly behind, which continues and intensifies by Key Stage Four (age 16) and the GCSE results.
Thirdly, in an educational climate within Wales that is depressed and depressing after our calamitous performance in the PISA tests, released in December 2010, the IWA’s study is a ray of sunshine. It shows that in some locations at least we are getting it right.
Fourthly, it is significant that the study comes from the IWA, and not from the Welsh Government. It is extraordinary that in the two most significant areas of educational policy in Wales – the issue of small rural school closure and the issue of what makes for ‘effective’ educational provision in an ‘ineffective’ system – the Government has not used its research budget to investigate the issues. Their research budget is small, admittedly, but that research into ‘core’ educational issues has to be done by an even more impoverished think tank is, frankly, pitiful.
At the same time it is best to state openly that there are a number of significant methodological caveats about the IWA’s study that should go on the record. Any study in the public domain should be subjected to evaluation of this kind.
Firstly, the study is based upon school level data which aggregates together all the children in a school, rather than upon individual pupil data. The latter is a much better way of controlling out the effects of intake to leave the ‘school effect’ to stand out.
Secondly, out of the sample of five schools that were meant to be exhibiting ‘success’ at Key Stage Three, two in fact did not.
Thirdly, the use of ESTYN judgements about what – from their routine work with inspections – makes a ‘good’ school adds rather little to the picture. Too often their judgements are circular – so, for example, effective special educational needs provision is simply characterised by where it is working.
And – most importantly of all – we need to know how much variation between schools was controlled out by the data on the schools’ backgrounds, the extent to which their pupils were in receipt of free school meals and so on. We need this information to assess whether we really do have a report on Wales’ most successful schools.
Having said all this, it is interesting that the picture of what makes for ‘success’ in Welsh secondary education is very close to that already existing in the thirty or forty countries around the world that have done research of this kind. In this latter research from the school effectiveness community, five things are usually considered key:
- The quality of leadership
- Data richness
- High quality teaching
- School ethos
- The belief that all can learn
In the IWA study, it is a very similar five:
- Leadership/evaluation
- Data usage
- Innovation in teaching
- Pupil ethos
- Attending to all pupils’ basic skills
Given this, it is rather strange that the report emphasises how in fact the schools are not really all that similar, when in fact across the five areas it is the similarities across the five schools that are repeatedly mentioned in the text. One senses here an attempt to hold on to the historic Welsh curse – that since good practice is so different in different contexts we can be excused from doing anything to evaluate what it is, and from doing anything to tell anyone what to do.
The criticisms of Welsh Government education policies – as expressed through the School Effectiveness Framework for example – are somewhat wide of the mark in some instances. PISA is argued as needing to promote a debate about strategic direction. However, there is no need for a debate given the move across the planet to the skills based approach that PISA measures. Diversity in response to national needs is argued for – but isn’t that exactly how we got into this mess in the first place? And the focus upon the basics – a core of Welsh Government Strategy – is debated when the basics are the door through which all educational outcomes are obtained.
In a particularly telling passage, the report argues that “Development of pupils’ literacy and numeracy skills is critical, but must not be pursued at the expense of teachers’ freedom to innovate…”. Isn’t this the second great historic Welsh educational curse – that teachers’ freedom is more important than whether the pupils are learning or not as they exercise it?
Nonetheless, the research has numerous important implications. We need better data systems in Wales – similar to the individual pupil database in England provided by the Pupil Level Annual Schools Census – to promote more research. Practitioners can feel confident that they have in this study useful designs of effective practice – wheels that go round if you like. They do not need to be subject to the third Welsh curse – the constant need to reinvent the wheel.
Although the report somewhat overstates it, we probably do need a balance between central imposition or prescription and local freedom to innovate, but it is likely this is best seen as something that should be achieved over time rather than at a point in time. In other words, central direction may be appropriate for an educational system like that of Wales currently which is far from good, with the freedom being introduced as it improves and as the system acquires the ability to make sensitive judgements.
As a picture of effective practice, and as a promoter of what should be vigorous debate, this is an important study.
