The IWA’s Dr Jack Watkins asks students and experts what lessons need to be learnt over the exam grading process furore.
Back in 2012, a scandal around the awarding of GCSE English Language saw widespread public anger.
Recriminations were passed back and forth between ministers, regulators and exam boards, leading to significant reforms in the hope that such scandals would be a thing of the past. Yet, through circumstances which nobody could have foreseen, 2020 has seen a row that makes 2012 look meager by comparison.
Reflecting on the events of the last few weeks, the biggest problem was probably not the outcome of AS, A Level and GCSE awarding but how we got there. Without the voices of learners coming to the fore in Scotland and the Scottish Government’s dramatic U-turn we might have been looking at a fairly different situation here in Wales.
Even after the initial decision was overturned, key decision makers, including the First Minister, claimed to stand by the model they developed for awarding A Levels in Wales.
There is an important lesson here about how data modelling is used in public life. Just as governments found with Covid-19, the data only takes you so far – beyond that, someone has to make a decision about what to prioritise. In this instance, exam boards, regulators and governments quickly realised that the public did not share their focus on reliability over time, or their concerns about grade inflation.
We all hope that the coming school year brings more stability and normality. But with the implementation of Curriculum for Wales, ‘normal’ actually means an ongoing process of change.
The experience of summer 2020 has raised important questions about teacher assessment, about the risks inherent in condensing a qualification into a high-stakes examination, and about how qualifications are viewed by universities, employers and the public.
In this article, we will hear several different perspectives that together address the complexity and challenges of this summer’s awarding and the lessons that we can learn for the future.
Sally Holland – Children’s Commissioner for Wales
When lockdown was announced on 13 March, it was clear that 2020 was going to be a year like no other.
When the decision to cancel exams was made in the same month, there was a focus on keeping stability in the system and consistency in the results from year to year. The events on A-Level results day revealed that this system used to award students with their grades had produced a series of individual injustices.
My office received impassioned pleas for support from students all over Wales whose grades had been downgraded.
“We will need… a robust centre-assessment moderation system in place, in case exams are disrupted again.”
I heard from Welsh speaking students from disadvantaged areas in rural Wales who missed out on medical school places. I heard from a disabled student who, with their family, spent months arranging suitable support and accommodation at a Russell Group university, only to face a rejection on results day. Head teachers told us of students who were downgraded by 3 grades when they had been assessed by teachers who were also experienced examiners.
It quickly became clear that providing students with their centre-assessed grades would correct many of these individual injustices, and, given the exceptional nature of 2020 having an outlier year in the results graph is a price worth paying.
While awarding these grades will help a lot of students move onto their next step, there will be some who feel that their school or college has not graded them fairly. It is a well-established expectation throughout our public services that we have an accessible complaints and appeals system.
I have consistently raised the need for an accessible and robust appeals process for learners with Qualifications Wales since the decision to cancel exams was made and I continue to do so. We will need this, and a robust centre-assessment moderation system in place, in case exams are disrupted again.
While 2020 has been exceptional , the events that unfolded highlighted some age-old issues regarding assessment within our education system. A significant proportion of young people in Wales are awarded GCSE grades that support neither their self-esteem or confidence, nor enable them to progress in education. With the age of compulsory schooling ending at sixteen, some of these young people do not receive any further opportunities to develop their learning.
This exceptional summer should be used for us to re-think some of the elements that were assumed as given before this period.
Dr Jennifer May Hampton – WISERD Education Data Lab
This year’s results season has been tumultuous, with last-minute changes to the way in which A level, GCSE and BTEC qualifications have been awarded. The dissatisfaction and controversies surrounding the determination of grades this year has highlighted a number of issues, not least the balance to be struck between fairness to individuals and fairness across the education system.
Pupils that have received results this summer have been living through unprecedented disruption to their education and it is right that they are not unduly disadvantaged by the circumstances that have meant that they could not prove their ability in the usual manner.
However, this also needs to be balanced by an awarding of grades across the different qualifications and across the years that allows for comparable achievement.
Before the last-minute changes allowing centre assessment grades to be awarded, the Welsh standardisation process appeared to be amongst the fairest in the UK, particularly for A levels. This was in no small part due to the differences in assessment practices across the different nations and the retention of AS levels as part of the A level, allowing the prior achievement of individuals in a given subject to be included in calculation of their awarded grade.
“What’s striking is that both the centre assessment grades and the standardisation processes have the potential to be particularly detrimental to high-achievers from disadvantaged backgrounds.”
Teachers will also have used non-examination performance in estimating centre assessment grades. That these were higher than those predicted by the standardisation process does not mean that the former were incorrect but rather that they were optimistic; teachers will have predicted grades based on performance ‘on a good day’.
That some pupils’ potential progression can be based on whether they are having a bad day or not, speaks more to the over-reliance on examinations than it does the validity of centre assessment grades.
The task that those who are involved with the awarding of qualifications – government, policy makers, regulators, exam boards – were charged with was both fraught and unenviable. The criticism of what has happened, in the lead up to, during and after, both results days has been extensive.
What’s striking is that both the centre assessment grades and the standardisation processes have the potential to be particularly detrimental to high-achievers from disadvantaged backgrounds. This, along with work on the various educational trajectories often overlooked by the focus on A levels, is an area of interest and investigation being undertaken by Wales Institute for Social and Economic Research and Data: Education Data Lab.
The issues touched upon in this piece are also explored in two blogs, available at the WISERD website: https://wiserd.ac.uk/news/level-results-day-2020 & https://wiserd.ac.uk/news/gcse-results-day-2020
Manon Clarke – Welsh Youth Parliament Member for Cardiff West
GCSEs are a rite of passage. They can be a source of worry for months on end for many of us, yet once they finish our minds focus on the next hurdle. This year was unlike any other. Doubt about whether we would sit exams, combined with the need to maintain revision, caused justifiable stress.
Understandably, these circumstances caused some people’s mental health to deteriorate. Why should an atypical year expect typical grades?
Nobody knew the pandemic was coming. Nobody could have predicted the challenges that students, teachers and policy makers would have to face. Given the situation, there was no perfect solution.
Having considered the fairest options, Welsh Government and Qualifications Wales decided to instigate an algorithm for the purpose of moderation. However, the algorithm hadn’t taught and known us over several years and it couldn’t take into account the hours of revision some of us had done.
Ultimately, I agree with Qualifications Wales’ and Welsh Government’s decision to return to the Centre Assessed Grades (CAGs). It was the right choice, evidenced by the relief of so many pupils and teachers.
Nevertheless, Welsh Government made this decision too late; some A-level and BTEC students are unfortunately experiencing the fallout now. Policy makers may come to regret that this episode will be fresh in the minds of the next generation of young voters.
Innovative. Informed. Independent.
Your support can help us make Wales better.
Iestyn Davies, CEO, Colegau Cymru
The year 2020 and the term ‘unprecedented’ will forever be synonymous. Nobody could have foreseen the extent of disruption brought about by the pandemic across schools, colleges and universities. However, the furore around results and the clamour for appeals was unfortunately very much predictable.
It was clear to us that no system to award results in the absence of examinations would be perfect. By calculating grades for learners as of mid-March 2020, results would inevitably be higher as learners who are normally unable to complete their courses and leave after this point would remain in the system.
Likewise, no learner had the opportunity to under or, for that matter, overperform in exam assessment. Add to that the perennial challenge of moderation and standardisation, it was clear as early as the spring, that the summer results season would present many significant challenges. What matters now is that we look to learn from the experiences of summer 2020.
“Currently, many learners studying BTECs still do not know their results.”
The Minister for Education’s announcement that there will be an independent review of the events surrounding this year’s cancellation of exams is welcomed. There are several areas that need to be covered.
These include the development of the moderation algorithms and any testing against Centre Assessment Grades (CAG) by Qualifications Wales (QW) and WJEC, as well as the process for ultimately deciding not to use this moderation tool. The role of QW and the balance between maintaining a UK-led (but in truth an England-led) approach versus a Wales-led solution also needs further exploration.
The lack of parity between academic and vocational learners must be highlighted. Swifter and more definite resolution was achieved for academic learners while those studying vocational qualifications faced further weeks of uncertainty, not just about the process for awarding results, but also in terms of receiving grades. Currently, many learners studying BTECs still do not know their results.
Assessments in 2021 need to be adapted considering potential for ongoing disruption. It is important we plan now. Government, the regulators and the awarding bodies need to work closely with Welsh Government and WJEC, but importantly, with learners too.
There are longer-term questions to address, such as the role of assessed coursework in all qualifications. All this needs to be done in a way that maximises the benefit of a made-in-Wales solution and acknowledges England is already divergent to Wales in key policy areas.
Whilst hindsight is indeed a wonderful gift, foresight is a necessary requirement. Despite all the change and uncertainty one thing is predictable: trying to identify and impose a solution unilaterally will fail and it will fail the most vulnerable learners the most.
Jonathan Powell – Welsh Youth Parliament Member for Wrexham
The young people of Wales were subjected to a discriminatory, prejudiced and detrimental system that prioritised statistical algorithms, expediency & the reputation of a grading framework over individual students and their future aspirations.
The method of standardisation prescribed to rectify inconsistencies across schools and colleges in truth broadened the discrepancies and enlarged the inequality. Students future prospects were hurled into jeopardy as a consequence of a calculated and consciously constructed postcode lottery disguised through the portentousness of the phrase “robustness.”
Students were assured at the outset of this pandemic that they would obtain fair grades that reflected their hard work. In reality, young people were penalised despite not sitting their exams by a scheme which caused chaos, confusion and for many catastrophe.
Notwithstanding the subsequent policy turnaround, initially, our educational professionals were ignored and their impartial and proficient judgments overlooked. Reliance upon the once fundamental Centre Assessment Grades declined and mathematical predictions became pre-eminent.
“We must now ask how did this happen and who will be held to account?”
Students were downgraded merely to satisfy previous boundaries and individuals across the nation consequently suffered. Previously predicted flaws and failures were ignored and neglected, resulting in belated changes both prior to and following results day.
This incompetency has meant that weaknesses which should have been rectified months ahead of results being published has led to numerous students losing their place at university. The uncertainty, anxiety and apprehension caused by the precarious handling of events has and will continue to cause widespread problems for this cohort of students.
A system formed and tailored with the intention of awarding students equitable grades in actual fact downgraded and improperly disadvantaged thousands around Wales. We must now ask how did this happen and who will be held to account?
All articles published on the welsh agenda are subject to IWA’s disclaimer.
