PISA 2018 Part 1 The Report and the Response
The reactions by media, politicians and educators to the PISA 2018 results published late 2019 are summarised and related to current education policy. Essays by Andreas Schleicher, Head of the PISA Program on a “Growth Mindset” and “Equity and Excellence” are summarised. The achievements of students in Estonia, Korea and Singapore are reviewed.
The publication late 2019 of the results of the PISA2018 exercise reignited all the usual commentary about whether the achievement levels of Australian school students were what they should be and Australian students should be amongst the best in the world and why had the results continued to decline despite all the extra money. Remedies were asserted that were not new and turned out to have little relationship to what the PISA reports said but were merely a restatement of persistent beliefs.
Go to Education Policy
Responses to PISA & the current policy agenda
PISA reports are detailed and comprehensive. And when the statistics in the PISA Report are considered it is clear that the commentators – most media and most politicians – have little understanding of statistics. Time after time evidence that they had actually read the reports was lacking; at best perhaps only a summary of some kind might have been read. The vast amounts of quality research on school education, and indeed learning as a whole, are not reflected either.
At the outset it should be noted that there are valid criticisms of the kind that should be levelled at all standardised testing. It is summative and therefore of little use in diagnosing the factors which have contributed to the results. In the case of PISA it is exceptionally difficult since the results are the outcomes of learning over all of the previous 18 or so years. Secondly the results are derived from a sample which means that how the sample is selected is important: to be reliable those selected should be the result of a random process. In the case of PISA it is known that in some countries there have been interventions of various kinds in some countries so that students known to be poorer performers are excluded. Additionally, as explained below, there are instances of uneven effort by the participants.
Since there are only three subject areas tested, PISA suffers the same problems as NAPLAN. Whilst the subjects are important and inform other subjects, having in mind the variations in learning styles and capabilities, caution should be exercised: many people succeed as adults in a variety of subjects and activities outside of those critically dependent on literacy, numeracy and science.
Federal Minister Dan Teehan, responding to the 2018 results, referred to them as a “wake up call”, said he would be seeking support from State Ministers for tracking of the achievements in tests of individual students and extra support for the best teachers so as to retain them in the profession. And he called for a “return to basics”, that is numeracy and literacy. He went on, “Australia should be a leader in school education. Our students should be ranked among the best in the world… We have a clear road map to implement the reforms that will improve student outcomes and we should be bold and decisive.”
The Business Council of Australia said educators needed to “lift their game” because students were exiting the school system lacking appropriate skills.
One media report said, “On Wednesday a chorus of concerned parents could be heard across the nation as the results from the triennial Programme for International Student Assessment (PISA) showed Australia recorded one of the sharpest falls in performance of any country.” The chorus came from a quarter whose children receive more support than any other but perform no better, a result that has not much to do with the children or the school.
The increased funding of schools was mentioned as if it had all been a waste. But Sue Thomson of ACER, the Australian Council for Educational Research, author of all the PISA reports for Australia, speaking in a panel on ABC RN Breakfast, cited research by the Grattan Institute: “after stripping out wage inflation and growth in student numbers, the claim is ill founded. There actually hasn’t been a huge increase in funding to schools, and actually 80 per cent of that funding has gone to advantaged schools rather than disadvantaged schools, so the funding is certainly something that needs to be looked at.” Many others have made the same point, including on many occasions, Trevor Cobbold (Save Our Schools Australia). Ministers time and again have asserted that money is not the issue, when in fact money does make a difference, especially for less advantaged students.
Educators and researchers drew quite different conclusions. Geoff Masters, chief executive of ACER and author of NSW’s curriculum review, argued that the PISA results show Australian 15-year-olds are lacking in their depth of thinking, rather than in their literacy and numeracy. Peter Goss, school education program director at the Grattan Institute, pointed out that arguing for more back-to-basics doesn’t fit the data. “We need strong foundations but we also need to help our young people stretch for the sky much better than we are doing.”
Finnish educator Pasi Sahlberg, now at the Gonski Institute at the University of NSW,
Finnish educator Pasi Sahlberg, now at the Gonski Institute at the University of NSW, very shortly after the release of the PISA results, responded to the question, should we be worried, by saying “panic is not the right way to react to PISA results here in Australia. Instead, we need to step back and take a good look at the bigger picture that the OECD’s study also offers”. He noted Australia was not the only country whose students’ results were in decline. “New Zealand, Canada (except in Alberta), Japan, Korea, the Netherlands and Finland are heading in the same direction. Actually, the 37 OECD countries, the wealthiest in the world, on average, are doing worse in PISA than previously.”
Sahlberg continued, “According to research done by the UNSW’s Gonski Institute for Education, three of five teachers in Australia believe there has been a clear decrease in students’ readiness to learn at school. Over 80 per cent of teachers think digital devices are a growing distraction to students. Almost all teachers reported that the number of children arriving at school tired has increased… [research revealed] a quarter of students aged 12 to 15 in Australia suffer from lack of good sleep. Most children have too little daily physical activity, have unhealthy eating habits, and have more mental health concerns than previously. No wonder, then, that they don’t learn as well.”
Rather than haphazard, simple solutions, PISA should be considered for the next steps, Sahlberg said. “Make equity in education the highest priority. PISA 2018 shows again that the most successful education systems are those that invest equally in equity and excellence. In Australia, this means, among some other things, funding schools based on their real educational needs.
“Give teaching, not testing, the highest priority. There are no successful education systems that use standardised testing as we do with NAPLAN, which pits schools against each other in a pointless contest. Australia needs a much smarter NAPLAN that gives teachers and schools a central role in deciding how children learn.
“We should focus systematically on student and teacher wellbeing. Research shows that a significant number of students today are distracted and not ready to learn the complex skills and knowledge required in schools today. Parents and schools together must find new ways to help young people to live and learn healthier and better.”
Sahlberg also pointed out that there was enough research and other well-grounded evidence allowing discovery of a new way forward to improve schools. The “American Statistical Association, for example, teachers account for about 1% to 14% of the variability in students’ test scores. The most important factors affecting student learning and quality of education are those found outside the school: students’ family background, where they live, their individual characteristics and the socio-economic make-up of the school they attend. Therefore, the majority of opportunities for improving quality of education systems are found in the society and its system-level conditions.”
Alan Reid pointed out that in the PISA the results from 79 participating countries and economies in 2018 Australia came equal 11th (with countries including NZ, UK and US) in reading (in 2015 it was 12th); equal 13th (with countries including US, UK and Germany) in science (in 2015 it was tenth); and equal 24th (same as the OECD average, and NZ and Russia) in maths (in 2015 it was 20th).” These are not quite the way media and political commentators saw the results!
Chris Bonnor, former school principal and author of a number of important reports, considered it might be worth putting a few ‘f’ words against a few of the responses: “fact, fiction, fable, fallacy, foible, fluff, fudge and so on.. The reporting and commentary around the current PISA panic, as in the past, largely avoids the deep-seated problems that are acting like an anchor on our progress, increasingly segmenting student enrolments.” He pointed to significant disparities between students from high and low SES backgrounds.
Trevor Cobbold, like Reid, identified difficulties with the samples of students sitting the tests: according to the OECD 3 in 4 Australian students do not try in PISA tests. Further he pointed out that “The PISA data also shows increasing student dissatisfaction with school which likely contributes to lack of effort on tests and is a factor, among others, behind Australia’s declining results: performance on PISA has no consequences for students as they don’t even get their individual results. The fact that Australia’s Year 12 results have improved significantly since the early 2000s raises further questions about the reliability of the PISA results… The report also shows large variation in student effort across countries. Around 80% of students in Germany, Denmark and Canada did not fully try compared to 60% in Japan and 46% in Korea.”
But, “However, the evidence that declining effort on PISA is a factor behind the declining results is only suggestive. An issue with the OECD data on student effort in PISA is that it is based on student self-reporting. There are well-known problems with self-reporting such as how truthful students are about their effort and the extent to which answers provided on subjective response scales can be compared across students and across countries.”
So much for whatever it is that PISA 2018 contributes to better policy on school education. When the 2015 results appeared, Emeritus Professor Dylan Wiliam of University College London argued that some interpretations were likely to be useless – a variety of interpretations could be made – and that what was needed to improve education in Wales was already known.
Shortly after the flurry of attention to the results, Minister Tehan met with his State and Territory counterparts in to adopt a new Alice Springs (Mparntwe) Declaration – based on the Declaration adopted in Melbourne in 2008. It committed them to action and expressed important goals, including in particular, promoting excellence and equity and confidence and creativity in young Australians to become successful lifelong learners. Another goal was “active and informed community members”.
Unfortunately, as several educators pointed out, the statement bore little or no relation to the actions they actually intended to take. For instance, Chris Bonnor branded the statements by participating ministers as fantasy: ”the decades-long focus on prescription, standardisation, testing and accountability seems to have done nothing to create … successful lifelong learners [or] active and informed community members”.. Quite the reverse: there is no shortage of indicators of student disengagement from learning, from school and from community”.
And worse. Aboriginal and Torres Strait Islander education is singled out from other marginalized groups as a central focus area needing its own commitment. But Melitta Hogarth of the Melbourne Graduate School of Education pointed out the Statement “contains nothing new or visionary” The words “are simply lifted from the Melbourne Declaration”:.
There were some agreements by ministers that could be considered positive such as the national plan to develop a suite of online tests to help teachers track the learning of individual students, which referenced the second Gonski Report. A review is to be undertaken of the national maths curriculum. That will do nothing!
Like the continual urging about the need for more students to study STEM subjects the review does not address the critical issue, in this case, the teaching of maths. What happened to the lessons which can be drawn from the success of celebrated secondary school teacher Eddie Woo, whose YouTube channel has been inundated with over 50 million views worldwide? People are alleged to be turned off by maths!
There won’t be more students studying STEM if the appallingly low recruitment of STEM graduates into jobs does not improve and the support of scientific research and institutions continues to receive the low priority it has ever since the mid-1990s! Even Boris Johnson’s principal advisor Dominic Cummings advocates substantial investment in pure research; the Australian government continues to look for applications of economic benefit.
Pisa 2018’s Special Topics
Andrew Schleicher expanded on two major issues in the PISA 2018 reports, “Fostering a growth mindset” and “Reconciling equity and excellence”. There was no mention of that in the media commentary.
Fostering a Growth Mindset[1]
Schleicher writes, “When comparing countries that score similarly in PISA, their income levels vary widely. History shows that countries with the determination to build a first-class education system can achieve this even in adverse economic circumstances, and their schools today will be their economy and society tomorrow. So it can be done.
“And it must be done. Without the right education, people will languish on the margins of society, countries will not be able to benefit from technological advances, and those advances will not translate into social progress. It will not be possible to develop fair and inclusive policies and engage all citizens if a lack of education prevents people from fully participating in society.
“When students struggle and teachers respond by lowering standards, teachers may imply that low achievement is the consequence of an inherent lack of ability. “Unlike effort, talent is seen as something that students have no control over, so students may be more likely to give up rather than try harder. According to some research, teachers also give more praise, more help and coaching, and lengthier answers to questions to those students whom they perceive as having greater ability. When teachers don’t believe that pupils can develop and extend themselves through hard work, they may feel guilty pressing students whom they perceive to be less capable of achieving at higher levels. This is also concerning because research shows that when a teacher gives a student an easier task and then praises that student excessively for completing it, the student may interpret the teacher’s behaviour as reflecting a belief that the student is less able.
“All of this is important because of all the judgements people make about themselves, the most influential is how capable they think they are of completing a task successfully. More generally, research shows that the belief that we are responsible for the results of our behaviour influences motivation, such that people are more likely to invest effort if they believe it will lead to the results they are trying to achieve.”
In one-third of the countries participating in PISA more than half the students considered intelligence was something about them that they couldn’t change much. However, students who disagreed or strongly disagreed with the statement “Your intelligence is something about you that you can’t change very much” scored 32 points higher in reading than students who agreed or strongly agreed, after accounting for the socio-economic profile of students and schools.
“Students who believe that their abilities and intelligence can be developed over time (those with a “growth mindset”) also expressed less fear of failure than students who believe their abilities and intelligence are “fixed”. .. the students with a growth mindset reported greater motivation to master tasks and self-efficacy, set more ambitious learning goals for themselves, attached greater importance to school, and were more likely to expect to complete a university degree.”
Students exposed to information about brain development gain through reading, class discussions and other activities tend to show greater belief about change and are less likely to blame failure to a lack of talent.
“Some students will require additional instruction time, others will not; some students will require different learning environments than others. Behind this thinking is the belief that all students can learn and succeed, and that the task of teachers is to design the learning environments, whether inside or outside the classroom, that help students realise their potential.”
Here is an especially important statement: “In many countries, it has taken time to move from a belief that only a few students can succeed to embracing the idea that all students can achieve at high levels. It takes a concerted, multifaceted programme of policy making and capacity building to attain that goal. But one of the patterns observed amongst the highest-performing countries is the gradual move from a system in which students were stratified into different types of secondary schools, with curricula demanding various levels of cognitive skills, to a system in which all students go to secondary schools with similarly demanding curricula.”[2]
Fear of failure can adversely affect well-being. Schleicher observes, “students who see themselves as more competitive scored higher in reading than those who do not, especially when they reported trying harder when in competition with others.” But, “some researchers argue that when co-operative and competitive behaviours are intertwined, as in interteam competitions, the performance and enjoyment of participants are even higher than in a purely cooperative or competitive environment.”
Equity and Excellence[3]
In considering reconciling equity and excellence, Schleicher concludes, “In sum, all countries have excellent students, but too few countries have enabled all of their students to excel and fulfil their potential to do so. The education that wise parents want for their children is what public policy should strive to achieve for all children. Achieving greater equity in education is not only a social justice imperative, it is also a way to use resources more efficiently, increase the supply of skills that fuel economic growth, and promote social cohesion. Not least, how we treat the most vulnerable students and citizens shows who we are as a society.”
Analyses show attracting the most qualified teachers can be a challenge. Disadvantaged schools are more likely to have less experienced and less qualified teachers and the students in those schools face a double disadvantage. But just paying teachers more to work in disadvantaged schools is not a solution: a holistic solution is needed, one in which teachers feel they are supported in their personal and professional lives, that they will be recognised and valued, when they take on additional challenges.
Many schools provide better support, sending the best teachers to work with students having the greatest difficulty (Singapore), transferring effective teachers (Japan), incentive payments and career structures to encourage high-performing teachers to schools with disadvantaged students and pairing high performing with low performing schools (Shanghai). “Policy makers can find it hard to allocate resources where the challenges are greatest and where those resources can have the biggest impact, often because poor children usually don’t have someone lobbying for them.” Further study of such schools is worthwhile.
High Performing Countries
It is absolutely vital that in all comparisons of countries reference be made also to studies other than PISA and that any cultural differences be kept in mind. Important commentary by such authors as Pasi Sahlberg, formerly of Finland and education historian Diane Ravitch as well as acknowledged pioneers in education and education philosophy be kept firmly in mind along with a clear view of the purposes of education.
There are three countries worth further consideration at this time because their students perform to a high level, Korea, Estonia and Singapore.
Korea
South Korea has long been a high performer. In 2018, according to the country profile, Korea’s students were amongst the highest performers in all three areas. But as the 2018 results reveal, the level of life satisfaction is one of the lowest among participating countries. Most of them have a clear idea about their future. They are amongst the most competitive, teachers are perceived as being very enthusiastic.
But there are problems. Students spend long hours at school and devote substantial time to extra study. Parents spend substantial money on after-school tuition. Competition is the “name of the game”. Students are formidable exam performers.
As Reeta Chakrabarti on BBC News in 2013 pointed out, “The huge investment in education has also resulted in an economy that’s grown at an astonishing rate since the end of the war with North Korea 60 years ago.
“South Korea has in two generations gone from mass illiteracy to being an economic powerhouse. Brands like Samsung and Hyundai, Daewoo and LG are internationally known. The country has built itself up through the sheer hard graft of its people.
“But it’s come at a big cost. The relentless pressure means Korea holds another much less enviable record, that of having the highest suicide rate of industrialised OECD countries.. success is built on an extraordinary work ethic that has delivered rich economic rewards, but that’s exacted a heavy price from its people and particularly its children. It’s a price the country is now gradually starting to weigh up.”
Estonia
Estonian students performed exceptionally in PISA 2018, their scores topping all non-Asian countries in all three areas of literacy. In terms of magnitude of increase in recent years, Estonia’s best performance was in reading; Ireland and Singapore students also gained substantially. Though Estonian students’ scores in math and science were higher than those of other non-Asian countries the gains over previous years were not spectacular.
Estonia has made high quality early years education a priority. “Almost every child in Estonia comes to Kindergarten from the age of three, or even earlier. Parents have to make a contribution, but it is capped as a proportion of the minimum wage. Kindergartens expect children to learn through play directed by teachers, with some more formal learning gradually introduced.” This helps level up attainment of all children, preparing them emotionally and physically ready to learn at school, so that there is a smaller gap by the time they are teenagers.
Whilst there are standardised tests at year 2, they are no longer used for league tables. Streaming students into different groups by subject or overall attainment is very rare in Estonia. There is an emphasis on extra study outside school.
“There is a national curriculum, but relatively few measures to hold schools to account through results. Teachers have a relatively high level of freedom to take risks in how they design lessons.”
School education is free and that includes textbooks, lunches and transportation.
“When Estonia regained independence from Russia, it looked to its neighbour Finland for education ideas.”
A majority (77%) of Estonian students exhibit a growth mindset [according to the PISA study] – “hey believe they are capable of improving their intelligence and are willing to put effort into their own development in order to secure a better future. This is the highest among the OECD countries: 70% of students plan to attain higher education.
The country profile PISA figures for Estonia show a slightly higher ration of teachers to students in socio-economically disadvantaged schools.
Estonia is a small country
As pointed out by Peter Goss and Mathew Cowgill of the Grattan Institute, “Economically illiterate arguments from our leaders don’t help.” Minister Teehan observed that the spend per student in Estonia was half that of Australia and therefore money is not the issue: he did not point out that wage levels in Estonia are half what they are in Australia!
Singapore
Last, it is worth reflecting on Singapore, a city state whose students have consistently been at or near the top of PISA results. The OECD report on “Strong Performers and Successful Reformers”[4] is particularly useful. A number of analyses of the system are available including one by the Grattan Institute[5] and another by University of Queensland‘s David Hogan.
A single education system was established under the administration of prime minister Lee Kuan Yew and schooling was varied to enable multiple pathways for students so as to reduce the drop-out rate and improve quality to produce a more technically qualified labour force. Singapore makes extensive use of international benchmarking as a way of continually improving the system: staff of the ministry, National Institute of Education (NIE) and schools all visit other systems to explore their practices.
The education system has changed from a knowledge-transmission education model to a one emphasising creativity and self-directed learning advanced through Ministry of Education policy directives, regular meetings and professional development opportunities. ‘Local councils provide families in need of financial help with support. Children who require special support in learning to read and in mathematics are provided with systematic intervention by teachers in small groups. Though preschools are privately funded government provides low-income students with financial support.
In recent years streaming has been replaced by subject matter banding, movement between streams has been made more flexible, and special attention is given to weaker performers.
Pasi Sahlberg has observed a high level of special coaching in Singapore which benefits more advantaged students.
CONCLUSIONS
Some worthwhile conclusions about what constitutes productive and effective teaching and learning can be drawn from PISA data. Unfortunately, media and politicians have failed to critically consider even the data they have had before them.
Continue to PISA 2018 Part 2: Education Policy In Australia And Its Failings or return to PISA 2018: Education Policies: Economic, Social and Organisational Influences
ENDNOTES
[1] PISA 2018 Insights and Interpretations p 37-39
[2] There is substantial relevance to the arguments about exceptional children and selective schools (see PISA 2018 Part 2)
[3] PISA 2018 Insights and Interpretations p 22-24.
[4] OECD (2010), Strong Performers and Successful Reformers in Education: Lessons from PISA for the United States http://dx.doi.org/10.1787/9789264096660-en
[5] Ben Jensen et al, 2012, Catching up: Learning from the best school systems in East Asia. Carlton: Grattan Institute. This is the report criticised by Alan Reid in Changing Australian Education for its unquestioning reliance on PISA data (see above).