Tests, League Tables and Accountability: A Democratic Right to Know?
Background
Public education in some countries – particularly the U.S – is asserted to be failing: despite substantial expenditures – in the U.S federal funding increased from $42.2 billion in 2001 to $54.4 billion in 2007 – there have been no improvements (it is said), indeed achievement has declined, at least relatively. In other countries there have been spectacular improvements and in some consistent high performance. Where the prevailing view is that education systems are failing the reason advanced is that standards have fallen and therefore proper accountability is needed.
These kinds of assertions, the demand for accountability and transparency, are common. So is the demand for quantitative data to achieve accountability. This is true of transport, hospitals, government services and activities generally. That quantitative data are not always indicators of the most critical aspects of the actions and performance of an enterprise, any more than they are of a person, is ignored in this. There is also the implicit assumption that the people responsible for the enterprise’s activities cannot be trusted.
In this essay assertions about the utility of tests and their application to enhance choice about the school students might attend are examined. The relationship of tests to teachers’ behaviour and performance is dealt with in a separate essay. Standardised tests are favoured most strongly by those who consider that neoclassical economics is a valid basis for decisions benefitting individuals and ultimately society. The drive is for accountability and transparency and publication of test scores is considered a right in a democratic society.
The essay explores the research and analysis of test scores and their use and the underlying justifications in the context of the many factors which influence educational achievement. The validity of the use of test scores aggregated across all classes in a school is also reviewed in relation to parental choice and other issues including the use of tests in successful school systems in various countries. The impact of tests on curricula, the influence of health and personal behaviour, the nature of learning in the classroom and the broad functions of education are considered and other factor which motivate individual performance are noted.
Expatriate Australian, US citizen and media magnate Rupert Murdoch, delivering the fourth of his Boyer lectures, ‘Fortune favours the smart’, said, “In particular, I would like to talk about why you hear so many business leaders talking about the problems with public education.” He reminded listeners, “The unvarnished truth is that in countries such as Australia, Britain, and particularly the United States, our public education systems are a disgrace. Despite spending more and more money, our children seem to be learning less and less—especially for those who are most vulnerable in our society.” He asserted that “if schools are to improve, and they must, then Australia needs in its schools a culture of performance and accountability”.
Murdoch’s views are shared by many of the political right in the U.S. and by other influential politicians and business people in the U.S., the U.K. and Australia. Some countries such as Sweden have adopted some of the responses to such assertions.
Accountability is demanded in the form of end of year standardised tests, the results of which are published. When the results of those tests influence future life of the student they are referred to as ‘high stakes’ tests. Making the test results available is claimed to be meeting the democratic rights of parents to know if their children are receiving the best possible education. The resulting league tables of schools are claimed to allow parents to more suitably determine which school to send their child to. Sometimes the results are published in a way which ties them to the individual teachers.
Where teacher performance is expressed as the achievement of the students in standardised tests, it is often proposed that those teachers whose students do well should be paid a bonus which, it is said, will motivate other teachers to perform better. Some commentators go so far as to suggest that teachers whose students consistently fail to improve their performance should be dismissed and that schools where students generally do not achieve improved performance should be closed. It is this combination of assertions that is the focus of debate. Encouraging good teachers to remain in the education system is of course an important goal.
Publishing the results for schools, which can be brought together in league tables, involve a number of assumptions and lead to various other outcomes. First, and at the most basic level, is the issue of tests themselves, how they are analysed and what kinds of conclusions can be drawn from those analyses, especially when the results for individual classes are aggregated for each school and at larger scales by state, province or country.
Secondly, assuming that there is some validity in the analyses of the test results, there is the issue of what factors contribute to the results such as teachers generally and the individual teacher for the particular year in which the test is conducted. Then there is the role played by other persons.
_______________
“Now, what I want is, Facts. Teach these boys and girls nothing but Facts. Facts alone are wanted in life. Plant nothing else, and root out everything else. You can only form the minds of reasoning animals upon Facts: nothing else will ever be of any service to them. This is the principle on which I bring up my own children, and this is the principle on which I bring up these children. Stick to Facts, sir!”
Thomas Gradgrind (Charles Dickens, Hard Times)
_______________
Students in some Scandinavian and European countries and some countries in South-east Asia consistently perform to a higher standard in the international tests PISA and TIMMS. The practices within the school systems in those countries are largely the same: they are different from the practices in the systems in other countries such as the U.S. In particular, standardised tests are not used in countries with successful education systems.
The Nature of Tests
Tests have been around for decades if not centuries. (In this essay, the term test is intended to refer to single end of year examinations in a few subjects such as math and spelling and reading used principally for the purposes of assessing the achievement of a class and/or school. It is not intended to refer to comprehensive examinations in a wide range of subjects at a national level such as Higher Schools Certificate only held close to or at the end of the last year of school the results of which are combined with other local assessments, or the National Assessment of Educational Progress (NAEP) tests, also in a variety of subjects including the Arts, which have been conducted for many years in the U.S.A., or the international PISA and TIMMS tests. However, any assessment based solely on externally conducted and evaluated examinations is but one assessment, even if it is standardised geographically and over time by competent professionals.(See later note on recent speech by US President Barrack Obama.)
In some countries their use has declined in more recent times. In subjects other than mathematics and similar subjects which involve the solution to problems, essays, in English and history for instance, which were once included in examinations, have been replaced by multiple choice questions as the time taken to mark essays has come to be considered as too great and the judgement unsatisfactorily subjective. The first casualty of testing therefore is written expression and the use of language.
Any system which involves applying a numerical score to a particular outcome or event involves grading. Once numerically scored outcomes for one person are compared with those of another there is the inevitable judgement of one being better than the other. In school systems where the results for classes of students are published, the data is an average or mean, and perhaps the median or mid-point of the total range. (The two together indicate the nature of the distribution of the scores, ie, the length of the distribution ‘tail’.) Equally important is a measure of the variation since the results for a set of samples can only be considered different if the probability of their coming from a different sample is significant. This is statistics 101, yet by and large it is ignored.
The focus on league tables ranks schools or classes or teachers, often without any regard, in commentary by media or politician, to the variation. In some cases, commentators even seem unaware that in a normal distribution – the usual situation of a sample – 50 per cent of the marks or scores will be below the mean: if not the median and the mean should be quoted. Some measure of the variation should be published and no distinction should be made between samples which are not significantly different.
When we look at the range of scores and the distribution of the scores for schools in any area we find substantial variation, not so much between schools but between classes within a school. Indeed within school variation is so great that there is substantial overlap between similar schools. This means that only the very best schools can be distinguished from the very worst. This is hardly going to make it easier for parents to decide whether they will send the kids to the school in the next suburb rather than the one down the road. The critical unit is not the school but the class.
Performance in the same class or even within the one school may be high in some subjects but not in others, performance may be good in one year but not good several years later. Performance is the result of the application of certain strategies and behaviours which may or may not be in place by the time the results are to hand – they are not leading indicators. And so on. Nor is the performance due mainly to the teacher in that year either, as we will see later.
The late Kenneth Rowe observed (‘Multilevel structural equation modeling with MLn/MLwiN & LISREL8.30: An integrated course (4th ed.) The 7th ACSPRI Winter Program in Social Research Methods and Research Technology, The University of Queensland. Camberwell, Vic: The Australian Council for Educational Research, 2000, p. 46), “In high stakes testing environments, educational practitioners are likely to distort their behaviour in order to meet the demands of the indicator, usually to the detriment of their real job… Measurement of learning outcomes using standardised achievement tests is inherently risky; test scores have low information value about the outlying processes “as well as the environmental and administrative frame conditions necessary to understand and appreciate the skills and efforts needed to fulfil a certain educational goal”… the majority of such tests assess skills in terms of generalised academic abilities and enduring cognitive ‘traits’ rather than specific learning outcomes arising from classroom instruction.”
Rowe concluded, “Above all, even when suitable adjustments for students’ intake characteristics and prior achievement have been taken into account, the resulting value-added estimates have too much uncertainty attached to them to provide reliable rankings”. “Australian politicians and senior bureaucrats currently advocating the publication of such performance information in the form of league tables”, Rowe observed, “are naively, and in typical fashion, stomping around in an uninformed epistemopathological fog’.
What can we say about league tables? The wish to have information about student performance is entirely understandable but opposition to it is not a matter of moral rectitude any more than a conspiracy of the Left, contrary to assertions of The Australian newspaper and the like-minded. The fact is that the statistical validity of the information is extremely limited. Therefore the value of the information is also extremely limited. The limited utility has nothing to do with unions or teachers not wanting to be evaluated.
Use of tables engenders competition. The belief is that competition improves the performance of players in the market. The belief is seldom borne out in reality. Competition is not a process leading to improvement of schools, as will be seen. There are valid reasons for opposition to the collection and dissemination of standardised test results and league tables. There is no evidence that competition leads to higher levels of educational achievement. Trevor Cobbold of Save Our Schools reviewed the latest evidence in early September 2014, referring in particular to the latest PISA in Focus brief. Instead of achieving improvement competition simply increases social segregation. This was shown years ago by New Zealand studies by Hugh Lauder & David Hughes ( in Trading in Futures Why Markets in education don’t Work, Open University Press, Buckingham, 1999).
Many factors contribute to test scores
Not only do statistics such as end of year standardised tests tell us next to nothing of any value because of purely statistical features. The outcomes of a year of teaching and learning and all the other features of schooling are a consequence of many teachers, home environment or environments, interaction with peers and experiences in many other situations. And the individual teacher in any one year can hardly claim entire credit or be held wholly accountable for the results of test performance in that year: other teachers and sometimes other schools have also contributed.
League tables are pounced on by tabloid media and many politicians, often in a nonsensical manner. In footnote ii of ‘Assessment, League Tables and School Effectiveness: Consider the Issues and ‘Let’s Get Real’! (Journal of Educational Enquiry, Vol. 1, No. 1, 2000), Kenneth J. Rowe notes the British media response on February 6 1996 to publication of the Chief Inspector of Schools Report on Quality of Education, The Independent headlining “Half of schools failing their pupils”, The Guardian, “Nearly half the schools in England are failing their pupils” whilst others ‘fulminated’: “Inspector condemns 25 years of weak and trendy teaching” (The Daily Telegraph); and “Black mark for half of primaries: Teaching is blamed for low standards” (The Times). In an article in The Guardian of 10 August 2010 titled ‘Primary pupils’ science knowledge at lowest level since 1999’ Jessica Shepherd, education correspondent, wrote “The proportion of pupils who achieved level five – higher than that expected of their age group and considered to be the level expected of a 12- to 14-year-old – has fallen from 43% last year to 28% this year. The confidence interval, indicating the reliability of the sample, was 95%” (my emphasis). What can we say of a reporter who deals with a statistical issue and yet demonstrates ignorance of statistics at the most basic level? The statement should have been the 95% confidence interval was … The Department’s website gives the 95% confidence interval as 27 to 30 per cent!)
Former Australian Education Minister (in the Hawke Government) John Dawkins made a national focus on student assessment and monitoring a key part of his proposals in Strengthening Australia’s Schools. Former Minister for Science and President of the Australian Labor Party Barry Jones (‘Dawkins changes a mistake, says Jones’, by Catriona Jackson, Canberra Times, 30 November 2002) once criticised the radical changes to Australian universities under former Dawkins reforms as the “greatest mistake of the Hawke/Keating years”..The Howard Government in its last years likewise demanded performance testing and publication (and a national curriculum) tied to funding to the States by the Commonwealth.
ANU economist Andrew Leigh (now member of the Australian House of Representatives and one time Assistant Minister) asserted (in ‘Schools need a report card too’, Australian Financial Review, December 20, 2007 and reprinted in On Line Opinion) “Boosting the performance of Australian schools is far from straightforward, but one sensible reform would be to begin reporting on the performance of individual schools, so that parents can better choose between their local schools. Such a reform would bring us into line with Britain and the United States, where policymakers across the board take the view that a school’s test scores are quintessentially public information.” One respondent to Leigh pointed out that the Australian Bureau of Statistics survey shows that with the exception of teenagers, older people have lower levels of literacy than younger people. Leigh’s assertion is unsupported by the evidence on competition and choice and is simply market economic rhetoric!
The Australian newspaper on 2 December 2008 trumpeted the call to arms, “Information about school performance must be public”. Amongst the assertions: “As the federal Government’s budget papers pointed out this year, OECD and other research shows that public reporting of student and school results has ”significant positive impacts on student performance”, even after accounting for demographic and socioeconomic backgrounds. This is one reason, but not the most important one, why the Government must honour Kevin Rudd’s commitment in August to make “individual school performance reporting a condition of the new national education agreement to come into effect from 1 January 2009.”
Onetime Minister for Education in the Howard Government, Dr Brendan Nelson, cautioned, “Opponents of the publication of school performance, league tables and comparisons other than on similar socioeconomic profiling, should ask of themselves whose interests they serve.” He observed, “What kids bring to school with them determines where they start. What happens at school determines how far they will get. For too many it is their only chance. The key is teaching…” But then he asserted, “For dysfunctional households, educationally, technologically and economically poor, the only way many will find out what’s going on at school will be electronic coverage of a published league table.” He called for more funds for “teacher training, accreditation, performance pay and teaching conditions in our poorest performing schools”.
Though the Rudd Government proposed publication of school test results it planned to only compare like with like. However, Mr Rudd, addressing the National Press Club on 27 August 2008 spent considerable time talking about the ‘Education Revolution’. “Our focus must be on the basics: ensuring that all of our children emerge from school able to read and write, with basic maths and science skills and the ability to enter the workforce, vocational training or university study… Greater transparency for the outcomes achieved by Australian schools is another area of reform.
“All Australian schools – public and private – need to do more to demonstrate the outcomes they achieve with the resources they receive from the broader community. This is not about creating an arbitrary public league table. It is about ensuring that all schools, all teachers and all students are focused on achieving the results we need as a nation and realising the potential also of each child…
“We should not tolerate underperformance… Where it is clear that individual schools are not up to the mark, we need to be prepared to invest money and effort to lift their performance. And where despite best efforts, these schools are not lifting their performance, the Commonwealth expects education authorities to take serious action – such as replacing the school principal, replacing senior staff, reorganising the school or even merging that school with other more effective schools.”
Not surprisingly, unions representing teachers responded strongly to these statements. The Queensland Teachers Union spoke of being appalled and of feeling betrayed. The Australian Education Union Victoria Principal Class members responded that it was difficult to maintain that positive morale in the face of ongoing, unsupported, negative public attacks.”
Most recently, in Australia, scores from tests in reading and mathematics universally administered to alternate years of schools across Australia have been incorporated into statistics which compare schools, asserted to be similar to each other in features including socio-economic background of the families from which the students come. Serious problems in almost every aspect of the resulting statistical tables have been identified in reviews of NAPLAN.
More recently, a large number of school districts in the U.S have started using what they call “value-added” test scores – the test score gain for the year – for students in each class in each school and related these to the individual teacher for the class, publishing data on the gains for each teacher. Of course those gains are averages and therefore must be accompanied by variances which overlap so that distinguishing each teacher is made more difficult.
None of these kinds of statistics tell us what it is that an organisation with good performance statistics does to achieve those results. Nor are the consequences of a focus on standardised test scores properly considered those advocating the use of tests. And those consequences are severe: they cut across a second feature of effective teaching and learning, the curriculum. Testing of a wide variety of subjects is considered to be unaffordable in terms of people and money. But other broad tests are used in most jurisdictions though comparability can be an issue.
The Contribution of Testing to Individual Performance and Future Consequences
There is a fundamental flaw in the underlying rationale for the tests. It is a flaw that is found in all initiatives based in neoclassical economics. The assumption is that competition is assumed to be a principal motivating force and that getting higher marks in a standardised testing regime is a significant driver in educational achievement at the individual level. What if that is not the principal motivation of most students?
The evidence is that the way to improvement is actually setting of personal goals and seeking to improve on individual past performance rather than achieve the goals achieved by someone else or some group of others. Arguments abound relative achievement and standardised tests and league tables comparing the performance of students in different schools, states and countries but comparison of individual progress over time is more significant. Moreover, failing tests is not in fact a condemnation of a desultory future.
Professor Andrew Martin of Sydney University’s Education and Social Work Faculty (and an internationally recognised scholar in the field) has studied the performance of students who set high personal goals. As he points out (in ‘Personal bests (PBs): A proposed multidimensional model and empirical analysis’, British Journal of Educational Psychology, 76, 803-825, 2006) sportspersons, such as Australian Olympic swimmer and gold medallist Ian Thorpe, frequently set high personal goals rather than relying simply on beating the other participants in a competition. Personal best (PB) has also used been used as an indicator of leadership behaviour. “PBs hold implications for motivation and achievement in terms of their facilitating effects for self-efficacy, persistence, educational participation, enjoyment of school, and task interest and engagement.” Martin also points out that PBs as for the most part specific goals or outcomes and therefore lead to higher levels of performance, seemingly by reducing the ambiguity about what is to be achieved.
From a study of attitudes to goals and self improvement concerning both schoolwork and homework (involving students from years 7 through 12 from various Sydney schools) Martin concluded that PBs were likely “an important means to achieve their potential at school. PBs have the capacity to carry weight with many young people who routinely see PBs lauded by elite athletes they so often hold in high esteem. Schools incorporating PBs into their reporting regime can harness the intuitive appeal of PBs [and] … provide their students with genuine experiences of success and give them a reason to dig deep on each new challenge.” “Students are most likely to reach PBs on tasks/goals that are (1) specific, (2) challenging, (3) competitively self-referenced, and (4) focused on self-improvement.”
A later study (Andrew J. Martin and Gregory Arief D. Liem, ‘Academic personal bests (PBs), engagement, and achievement: A cross-lagged panel analysis’, Learning and Individual Differences 20 (3), p 265-270, 2010) examined the relative salience of prior academic personal bests in predicting subsequent engagement and achievement compared with the relative salience of prior engagement and achievement in predicting subsequent PBs. PB goals were found to be more salient in predicting engagement and achievement than previous and engagement and achievement were in predicting PB goals. In other words, competitive position relative to other students was less motivating than individually set goals. The study also re-affirmed the importance of PB oriented goals in academic processes and development In a media interview (Anna Patty, ‘To aim for the top of the class, aim for a personal best’, Sydney Morning Herald May 27, 2010) Martin said, “We accept competition is a reality of life, so students will have to perform and achieve under competitive conditions, but an effective way of handling those competitive conditions is to compete with yourself and your previous best, more than competing with others.”
The marginalising of achievement against personal bests is as unfortunate and ill-informed as branding the experiences of children at pre-school, children’s museums and science centres as play but not learning. Pursuing personal bests is greatly encouraged by formative evaluation, the use of tests to measure progress by feeding the results into the analysis of behaviour.
Academic under-achievement is not the end of the road for most students: a very large majority of ‘low performers’ at age 15 years go on to make a successful transition into full-time work or study (or a combination of these). Sue Thomson & Kylie Hillman of ACER’s 2010 study, ‘Against the odds …’ research found that “… students who recognise the value of mathematics for their future success are more likely to achieve this success, and that includes being happy with many aspects of their personal lives as well as their future and career”
It also found that a positive school experience not only impacts on students’ lives while they are at school but appears to continue to influence them once they have left. That experience can positive experience regardless of achievement level “if an appropriate balance is maintained between the pursuit of personal goals and individual development”. That balance generally involved being fully engaged in education, employment or a combination of these and being happy with their situation, enjoying being at school, enjoying learning and feeling safe and secure. Fostering social and emotional development of young people, as well as their academic development, is an important aim of education.
The implication for testing of the research about ‘personal bests’, motivation and the future of students who rate as ‘low performers’ at school is that testing is often not reliable as an indicators and that simply focusing on reading and math misses the very important relationship between a later successful life and factors such as overall happiness and personal development.
The focus on relationship between teacher performance and student achievement has used “value-added” statistics which measure the difference between the test scores of students in subsequent years. Schools Chancellors in some states have fired teachers based on evaluations from principals and other educators and value-added scores when they were available as David Leonhardt (in New York Times Magazine September 1, 2010) found. The use of such statistics has provoked varied reactions, those from teachers generally negative. Secretary of Education Arne Duncan calls the release of the statistics “long overdue”.
Value-added Tests
In late August 2010, the Los Angeles Times published “value-added” ratings of some 6,000 Los Angeles elementary school teachers and 470 elementary schools. The Times reported, “Third-, fourth- and fifth-grade teachers who taught at least 60 students from the 2002-03 through 2008-09 academic years were evaluated in the Times analysis… [but that] whilst most Los Angeles Unified School District’s elementary schools were included, data for charter schools that do not report directly to the district were not available.” (emphasis added)
The Times explained, “A teacher’s value-added rating is based on his or her students’ progress on the California Standards Tests for English and math. The difference between a student’s expected growth and actual performance is the “value” a teacher added or subtracted during the year. A school’s value-added rating is based on the performance of all students tested there during that period.
“Small differences in rankings are not statistically significant, particularly for those rated near the average. In some cases, recent gains made by teachers and schools may not be reflected in their ratings. Although value-added measures do not capture everything that goes into making a good teacher or school, The Times decided to make the ratings available because they bear on the performance of public employees who provide an important service, and in the belief that parents and the public have a right to the information.”
The Times also explained in response to the question, Do value-added scores tell you everything you need to know about a teacher or school? “Not at all. Even advocates of the method say it should count for half or less of a teacher’s overall evaluation. In reviewing a teacher’s performance, administrators may want to consider their classroom observations, the quality of students’ classroom work, instructors’ abilities in subjects other than English and math and many other factors. Similarly, parents looking for a school for their child may also want to consider factors such as the school’s API score, course and extracurricular offerings, and their own impressions of the teachers and campus.”
It should be noted that data was confined to test scores in English and math. No account was taken of previous teachers’ contribution to the student’s performance. All the data was already available; the analyses were done by an economist from Rand Corporation.
Professor John Hattie, as already mentioned, has strongly criticised these tests as taking little away from the serious problems of reliability. A panel of economists and educators convened by the Economic Policy Institute in Washington DC, in reviewing the technical evidence for the use of value-added statistics, found that there were a number of problems. They concluded, “although standardized test scores of students are one piece of information for school leaders to use to make judgments about teacher effectiveness, such scores should be only a part of an overall comprehensive evaluation. Some states are now considering plans that would give as much as 50% of the weight in teacher evaluation and compensation decisions to scores on existing tests of basic skills in math and reading. Based on the evidence, we consider this unwise. Any sound evaluation will necessarily involve a balancing of many factors that provide a more accurate view of what teachers in fact do in the classroom and how that contributes to student learning.”
The tests themselves do not deal with the social skills crucial to early learning, the scores can vary from year to year for the same teacher and, in the case of teachers dealing with already high performing students, fail the teacher in that indefinite improvement is obviously impossible. Moreover, as many including Trevor Cobbold, have pointed out, the system can be ‘gamed’: poorly performing students excluded from the tests
With all the fuss about tests, one important issue has hardly surfaced. In his important paper on improving teacher quality Kenneth Rowe (‘The Importance of Teacher Quality as a Key Determinant of Students’ Experiences and Outcomes of Schooling’, ACER Research Conference 2003, p 10) noted that boys exhibit greater externalizing behaviour problems in the classroom and at home, many demonstrate poor achievement progress in literacy and boys make up the vast majority of children identified as “at risk” in the early years of schooling because of poor progress in literacy. But, Rowe notes, boys have a higher prevalence of auditory processing problems: after the age of four boys have less ability to process auditory “streams” of speech than girls. Therefore it is not reasonable to expect that boys will be able to absorb teaching materials as readily as girls.
Conclusions
General community attitudes affect the commitment of students and others to educational achievement. An example is mathematics. As mathematics expert Professor Celia Hoyles observed in her paper ‘Rich collaborations: The case of mathematics’ (at the Graduate School of Education National Curriculum Symposium, University of Melbourne 25 – 27 February 2010), England (as an example) is one of the few countries where it is fashionable to be ignorant about mathematics! In such an environment it is hardly reasonable to blame teachers for poor math results in exams.
Tests are a way of measuring performance, of assessing productivity. But productivity is influenced by much of the way we live. It is clear that hours of sleep influence performance. (According to the 2009 OECD Social Indicators survey the French spend over two hours a day eating, more than anyone else in the OECD and nearly double the time spent by Americans and Canadians. They also sleep more than anyone else, on average 8.5 hours per night.)
The belief that test scores are reliable indicators of school performance is a fantasy. The key performance unit is the class. Variation in test scores within each school is such that only the very best schools can be distinguished from the very worst and many factors contribute to student performance, not least teachers in previous years, even teachers in other schools. The emphasis on tests drives schools to “teach to the test” and thus narrows the curriculum, with no lasting benefit to the student. Test scores are an unreliable indicator of student future achievement.
There is such a concern with productivity that in countries such as the US vacation is limited and working hours are long, almost a return to the earliest days of the Industrial Revolution. People arrive at work early and stay late, unspoken sanctions directed at those who do not. In industries where shift work is common, the hours are long. Meals are eaten on the run, toilet breaks are restricted, lunch is eaten at the desk, tea breaks are eliminated. And hours of sleep are reduced, a habit acquired at school as students stay up late swotting for exams or attend extra coaching sessions.
But research shows that longer hours of sleep lead to better brain function, as ABC Science News reported recently. Researchers at Harvard Medical School taught people how to locate a specific shape somewhere in a computer-generated maze. They made one group of participants have a nap before trying the exercise again. A second group of people were kept awake. A third group were allowed to sleep, but were woken up several times to check whether they were dreaming. People who reported dreaming about the task actually performed better than those who didn’t dream about the task and even better than those who didn’t sleep at all.
In the same report it was stated, “Dr Colin Sullivan, a Professor of Medicine at the University of Sydney who specialises in sleep, says there is now a large body of evidence showing sleep has an important role not just in learning, but in creativity and mood as well. “Seven and a half and eight hours sleep is fairly normal but we do also know that probably most people in the modern day get less probably by about an hour than what they need,” he says. Sullivan says sending everyone to bed an hour early may be the key to boosting the creativity and productivity of the nation.”
Demands for testing, couched in the notion of accountability and transparency, of value for money and of productivity, are not peculiar to schools. Testing has parallels with the search for performance indicators for nonprofit organisations such as charities, the arts and sectors such as health and publicly funded R & D organisations. They are grounded in the managerialist mantra that believes that merely publishing the results of some indicator of performance, one which is numerical, will encourage those who might otherwise not seek to do their best to do better. Its basis is a focus on ends as if no other way of judging achievement and progress is possible. It has marginalised subjects which cannot be related directly to a productive economy.
School teachers should be accountable for the outcome of actions they take but it is not reasonable to have them be accountable for outcomes which they are not responsible for. Accountability requires a meaningful link between effort and results, not accountability for outcomes to which there is no meaningful link. Individual teachers make a contribution in any one year to a student’s achievement but past teachers have also contributed. So have parents. Other factors also contribute. Parents make a significant contribution to the educational attainment of their children before the child even arrives at school and then by supporting the child and their learning in the home and providing an environment conducive to learning. There would not seem to be a strident call for them to be accountable.
There is a further issue of importance to schools as to all organisations, even small ones. That is the matter of external control. Centralised control is based on the proposition that people generally can’t be trusted and that only those at the top of the hierarchy have the knowledge and experience to make the right decisions. Almost all State schools are controlled by a central bureaucracy at government level. This is something which supposedly neoclassical policy was to rid society of but in fact it has achieved the opposite. It is axiomatic that involving all schools in annual tests for students and merit pay mean greater intervention by people and agencies outside the school itself; where the government’s education agency is involved pressure can nevertheless be exerted by non education agencies outside government.
By compiling and publishing league tables which rank schools by the average score of their students, advocates claim that parents will be able to make better choices as to which school their children should go to. And therefore that their child will receive a better education. In New Zealand, as shown in the study by Hugh Lauder & David Hughes (Trading in Futures Why Markets in education don’t Work, Open University Press, Buckingham, 1999), the provision of greater choice or ‘marketisation’ flowing from publication of such league tables, led to no significant improvement of student performance but only to polarization of schools by social class and ethnicity.
The problem is not just that standardised testing, league tables, the false opportunity of choice (available only to those who both can afford to exercise choice and wish to take their child outside their local community to be educated) is destructive of a quality education and the linking of test results to individual teacher performance is irrelevant to any group of employees, not to mention a highly significant group of professionals whose performance influences the future of every society in almost every area.
The fact is the statistics are of the most limited utility: they don’t reliably distinguish good schools from bad schools and are poor indicators of teacher effectiveness, certainly less reliable than properly skilled and committed leaders applying supportive performance management. Worse, giving tests the prominence they have in these regimes leads to attempts to ‘game’ the system and behave in a corrupt manner by cheating, through schools excluding poorly performing children from testing and so on and so forth. Trevor Cobbold of Save Our Schools has drawn attention to this on numerous occasions and also highlighted the fact that even doctors have a strong incentive to “play the numbers” to look good. One way is to “cherry pick” patients.
Accountability is everything. A group of economists including Nobel prizewinner in economics, Professor James Heckman of the University of Chicago (John Cawley, James J. Heckman and Edward Vytlacil, ‘On Policies To Reward The Value Added By Educators’, The Review of Economics and Statistics, 81(4), p 720-727, 1999) observed some years ago, “One current educational reform seeks to reward the “value added” by teachers and schools based on the average change in pupil test scores over time. … ][However] the average gain in test scores is an inadequate measure of school performance and current value-added methodology may misdirect school resources.”
The demand for testing and everything that goes with it is another aspect of neoclassical economics and managerialism. It assumes that centralised control can significantly influence the outcomes of individual enterprises and groups of people. It does not. It is not a matter of accident, indolence or simply a desire to manage risk that Sir Richard Branson, one of the world’s leading entrepreneurs, has arranged that each of the companies in the Virgin group is semi-autonomous. Rather than control, Branson strives for genuine commitment and trust.
Our education debate is seriously out of kilter with the real world! The world needs creativity! As Sir Ken Robinson, expert on education and creativity points out (for instance, in an interviewed with Kerry O’Brien on ABC TV 16/06/2009: ‘Education systems too narrow’), “young children starting elementary school this year will be retiring around 2070: no-one has the faintest idea what the world will look like in 2015 let alone 2070… education clearly has to cover some common ground for everybody.
“You know, we all need to learn to read and write and so on. But there’s much more than that… when politicians talk about tightening their belts and raising standards, they always focus on these top, apparently, subjects of maths and science and languages. Well, they’re very important. But so are music and dance and art and poetry and all the things that the arts teach, and humanities and history, and all of those things which speak to the nature of what it is to be a human being and to be able to make your way in the world.”
A school experience narrowed by a tested focus on “literacy” and “numeracy”, which have been acknowledged as skills rather than curriculum issues anyway, do not advance creativity or lead to a lived life! Or to a better community or society in any sense whatsoever, individually or collectively. The use of tests narrows the curriculum and compromises the efforts of teachers.
League tables do not discriminate meaningfully and inasmuch as they encourage competition between schools they have a negative, not a positive, effect on educational achievement. Value-added scores, which measure gains over a particular time and therefore supposedly indicate the actual contribution during that time, do not give all the information needed to make judgements and choices because of the influence of other factors.
Though it is claimed that test results help parents choose the best school for their child, choice is limited by other factors and schools are chosen for factors other than simple scores in end of year tests. Teaching to the test narrows the curriculum and diminishes students’ preparedness to enter later life and work. Gains in test scores misdirect resource allocation.
Linda Darling-Hammond, visiting Australia in May 2011, has consistently criticised the use of NAPLAN tests: ”We have learnt about the potential negative effects of very narrow tests, particularly when they are put in a high-stakes context”. Those schools whose students achieved high results in standardised tests were rewarded financially and so were teachers of the particular successful teachers which led, according to Darling-Hammond, to many of the best teachers abandoning schools in the disadvantaged areas. “We have seen growing student exclusion to get the scores up. Schools either prevent students from taking the test or encourage them to leave school. Schools that have choices about who to admit will not admit low-achieving students because they will bring their scores down.”
Comments from such experts did not sway those committed to an alternative view such as Miranda Devine writing in the Daily Telegraph in 2011, “When it comes to Naplan tests, you just have to ignore the propaganda from the self-serving Teachers Federation, which has always pitted itself against transparency, choice and excellence in education.”Teaching to a test is not necessarily bad, according to this view, considering that the tests focus on core skills of literacy and numeracy! The knowledge that a broad curriculum is a feature of the most successful school systems is not within the comprehension of some! “And now that school principals are held responsible for NAPLAN results, it’s about time they were given the tools to improve their students’ scores – the freedom to hire and fire teachers.” That there is no evidence to support that view is not apparently relevant either!