Combating Welsh and British ‘curses’ in education

Professor Reynolds and I are in agreement on a number of points:

• All the evidence indicates that Wales is seriously underperforming at a vital stage in pupils’ education, Key Stage 3.
• The lack of Wales-specific research on this 11-14 age group is a cause for serious concern. The IWA study is a crucial first step in building our knowledge base on what works in Wales.
• Given the “depressed and depressing” tone of the debate in the wake of Wales’ poor performance in the PISA tests, there is a need to acknowledge those instances where we are getting it right.

The report set out to provide, through in-depth case studies coupled with synthesis of the key features, a new level of information about good practice in Welsh schools. And Professor Reynolds concludes that it does just that:

“Practitioners can feel confident that they have in this study useful designs of effective practice.”

Whilst there is much on which we agree, Professor Reynolds presents a number of caveats about the research which are themselves open to question.

He first queries the fact that “the study is based upon school level data … rather than upon individual pupil data”. It is true that, by looking only at the whole-school averages, we lack the ability to determine the range of performance found within the school. Thus although we know that the school adds value to the student body as a whole, we cannot be certain how consistently that value is added. Is this a school whose pupils consistently perform close to the whole-school mean, or does the mean conceal a wide variation in attainment levels?

This notwithstanding, the data are pupil-matched, which means that the aggregated data do reflect the extent of each individual pupil’s progress. For the school as a whole to show value added, a substantial proportion of those individuals must have seen significant progression. Since the IWA has only limited resources, we had no choice but to make the compromise of working with school-level data, rather than the much higher volume of pupil-level.

Professor Reynolds’ second concern is that we cannot be certain that the five schools profiled are “Wales’ most successful schools”, not least because two of the schools did not exhibit success at Key Stage 3. To address the first point, the report does not claim to profile the “most successful” schools in the country – indeed, it’s doubtful that one could objectively define such a thing. Instead, it seeks to identify schools at different stages of development that exhibit good practice at Key Stage 3.

In this context, it is too simplistic to say that simply because a school is not adding statistically significant contextual value added at KS3 it is not ‘successful’. On a statistical note, for instance, the Welsh Government itself recognises that traditionally high-performing schools can find it hard to add value – the best they can hope is to perform as expected. And to take the Cwmtawe Community School example, the school produces excellent results at Key Stage 4, the culmination of a child’s compulsory education. In this case Key Stage 3 can be seen as part of a progression towards the achievement of GCSE success.

Professor Reynolds thirdly questions the use of Estyn judgements as the basis for analysis of the key elements of good practice. Crucially, however, these judgements were used only to identify general themes as a framework for discussions with the case study schools.

Findings from the inspection reports of thirty schools were amalgamated in the creation of an interview schedule, and thus the Estyn reports for the five case study schools were not directly used to inform assessment of the factors considered key to their success.

The fourth concern raised by Professor Reynolds is that the report “emphasises how in fact the schools are not really all that similar”. Certainly differences between the schools are acknowledged, but the report does not “emphasise” them. In fact, Professor Reynolds himself goes on to note that similarities “are repeatedly mentioned”. It is uncovering these similarities that prompted the research, and they therefore form the structure for the analysis of the schools’ approaches.

Nevertheless, it is important to note differences in how the key factors are implemented. This is simply to acknowledge that teachers do not (and should not be expected to) implement initiatives without regard to context. It should not detract from the importance of the common factors identified across the schools.

The significance of context is crucial. Professor Reynolds reproaches me for falling victim to the “Welsh curse” of putting teacher freedom above pupil outcomes. Yet we must equally avoid another very British curse, that of de-professionalising teaching through the imposition of one strategy after another (however well-meaning).

That said, there is undoubtedly a need in some Welsh schools for substantial external input as a means to improving both standards and consistency. As Making a Difference at Key Stage 3 concludes:

“Some schools are already operating to a high standard. They undertake monitoring and evaluation that is suitable to their particular circumstances, and should be given the freedom to continue doing so. Others will need to earn that degree of autonomy. Schools which are not yet deploying and acting upon self-evaluation procedures with sufficient rigour will require external input. This does not mean that they should be required to renounce all control over the monitoring of school and pupil progression. The emphasis must be on building internal capacity, to enable ongoing development and allow the school to earn autonomy in the medium to long term.”

Professor Reynolds and I share, therefore, the view that local freedom to innovate should be introduced only once schools have demonstrated the capacity to use it effectively. Where we differ is over whether the same strictures should be applied concurrently across the whole of the Welsh educational system. The gap in performance between our best and worst performing schools suggests not.

For those schools which require additional support, one critical question remains that only time will answer. Having intervened, will the government or its agents be willing and able to release control?