Best Teaching Part 1: How teachers make a difference – John Hattie
Professor John Hattie, a New Zealander who has worked in Australia, North America, New Zealand and is now at the University of Melbourne, completed an extensive review of the literature – more than half a million studies – on teaching and learning. His conclusion is unequivocal! [1] “We should focus on the greatest source of variance that can make the difference – the teacher. We need to ensure that this greatest influence is optimised to have powerful and sensationally positive effects, but they must be exceptional effects. We need to direct attention at higher quality teaching, and higher expectations that students can meet appropriate challenges – and these occur once the classroom door is closed and not by reorganising which or how many students are behind those doors…” This essay deals mainly with Hattie’s research.
Everyone, almost, agrees that teachers are the key element in the education of children in school. As the McKinsey reports[2]observe, the only way to improve student outcomes is to improve the quality of classroom teaching across an entire system. The best-performing systems around the world go to great lengths to ensure that all their teachers are well qualified and well prepared in the subjects they teach and have access to high-quality, ongoing professional learning opportunities. High quality performance data, teachers and school accountability, appropriate financing and organisation structure and pedagogy models are required for schools to move from ‘fair’ to ‘good’. and further advances – from good to great – require teaching and school leadership and appropriate career structures like those in medicine and law. Last, in achieving excellence the focus is on peer-based learning, system-wide interaction, innovation and experimentation.
Several essays explore effective teaching and summarise some of the most important research on effective teaching and highlight some case studies. In this first one I deal with a very important meta-analysis of education outcomes; a subsequent one will summarise a particularly interesting study of what goes on in the classroom.
What does the effective teacher do that makes the most difference and what other factors might be relevant? A few decades ago, the simplest received explanation was that teachers who were content experts were most likely to do the best job. Few now believe that because the evidence doesn’t support it. That doesn’t mean that content knowledge is unimportant, just that it isn’t enough. Knowledge of superior teaching instruction is vital and that is not simply a matter of more experience. Cooperation between teachers is also very important.
Teachers Matter
The OECD Report Teachers Matter points out, “Student learning is influenced by many factors, including: students’ skills, expectations, motivation and behaviour; family resources, attitudes and support; peer group skills, attitudes and behaviour; school organisation, resources and climate; curriculum structure and content; and teacher skills, knowledge, attitudes and practices. Schools and classrooms are complex, dynamic environments, and identifying the effects of these varied factors, and how they influence and relate with each other – for different types of students and different types of learning — has been, and continues to be, a major focus of educational research.[3]
“… Improving the efficiency and equity of schooling depends, in large measure, on ensuring that competent people want to work as teachers, that their teaching is of high quality, and that all students have access to high quality teaching”
The largest source of variation in student learning is attributable to differences in what students bring to school – their abilities and attitudes, and family and community background … While “teacher quality” is the single most important school variable influencing student achievement , qualifications, experience and tests of academic ability don’t capture many of the important aspects of teacher quality. Other reports make the same point. [4] Selecting teachers on the basis of their education and qualifications alone is unlikely to be appropriate. Again, this is true of recruitment generally, something that is very commonly ignored.
Professor John Hattie, recently of Auckland University and now Director, Melbourne Education Research Institute in the University of Melbourne Graduate School of Education, is renowned for his detailed survey of hundreds of research papers on the what factors contribute to superior outcomes for the student. His meta-analysis studies reveal that after what the child brings to the school experience – what he or she has learned in the first few years of life – the teacher contributes more than any other factor to student achievement. But it is the expert teacher, not simply the experienced teacher, who makes the great difference! Expert teachers “engage students in learning and develop the student’s self-esteem as learners”. And it is certain things that the teacher does. Effective instruction and certain components of feedback, dialogue between the teacher and the student, contribute substantially.
Exemplary teaching means challenging interaction between teacher and student together with formative evaluation, that is use of frequent feedback to improve performance against already determined (and high) standards (and previous performance). That is what those reports, inquiries and studies which compare the education systems and practices of various countries make clear. And the better education systems and practices are most likely found where teachers are respected within the community.
This first essay on the subject of best teaching deals mainly with these studies and their conclusions. The relationship of class size to student achievement is a frequently mentioned issue in debate about education. One study in particular carefully reviews the evidence for that.
Some Initial Observations
The studies of teaching and learning and what makes them effective is vast, especially in the US. The focus in teacher training now is much more on pedagogy, on the nature of the teaching process. There are a number of recent reports which consider practices in many countries.
The considerable research tells us that the quality of the instruction makes the difference and more than that, that it is the interaction between the teacher and student, most especially the feedback the student gets, which is essential. The effective teacher not only monitors the student’s performance, the ability to show they have understood what has been learned. Effective teachers also provide a challenging curriculum, not one brought to the level which it is thought all students might easily manage. As in most organisations, cooperation between teachers makes a significant difference as each one helps others. And two other things matter, the respect people in the community give the teacher – indeed the relationships between the school and the parent community particularly – and the leadership provided by the principal of the school.
Whilst there is an understandable demand for some kind of indicator of achievement, mere performance by school students remembering what it is they learned and being able to apply that to answering questions in a test hardly resembles living a life or working in an organisation. An alternative way of assessing best practice, for that is what we actually want to find out about, is to look at the processes which occur in the organisations that perform well according to whatever criteria we have chosen.
No set of measures are sufficient by themselves in the long run for the formulation of best practice. And in the case of schools we need to understand what it is that we actually want the student to do in later life as a result of whatever it is they have achieved at school. Training and knowledge are important but not sufficient any more than are the social skills alone.
We have reliable national and international tests from students in a large number of countries and information about schools and teachers. Some countries, notably the US, have huge runs of statistics on all kinds of matters from test scores to teachers’ qualifications to indicators for disadvantage, geographic location and other demographics in relation to such things as student gender and age. But rather than more measures, we need to look at the practices of the teachers and the relations between schools and governments and communities at the best performing ‘benchmark’ schools.
The numerous reports make clear that better schooling means ensuring competent people are recruited to the teaching profession and that motivation to teach is of fundamental concern as is ongoing professional development and effective leadership by the principal and senior teachers in the school. A focus on efficiency by, for instance, merging schools or classes defeats attempts to bring greater effectiveness to the task. In these many ways a school resembles other organisations and the factors which contribute to an effective teaching profession are much the same as those that lead to an effective organisation in any enterprise.
Media has tended to focus on the trivia of conflict between the various players in the debates. Parents are portrayed as being concerned that the school their children are attending is probably not up to scratch but that if they had some performance indicators they would be able to make better decisions.[5] Meanwhile the business community complains that their recruits are insufficiently prepared for work implying thereby that the education system is failing. Some companies have established their own tertiary education courses in frustration at what they perceive as inadequate performance by the existing system. Every organisation has a responsibility to train its staff and superior staff development is one of the hallmarks of superior organisations. Even computers do not arrive at work ready on day 1 to perform what is required of them. Inasmuch as employers pay their more qualified staff more it makes sense to invest in their training, a value-adding exercise!
Research on effective teaching all have very similar conclusions about attention to the student, use of tests and other assessments to improve teaching strategies (not judge schools), cooperation amongst students in the classroom, feedback to the student on their performance and setting of high standards of achievement along with the belief that all students can succeed.
John Hattie: The teacher, not the school, makes the difference
Professor Hattie recognises six major sources of variance in student’s educational achievement: the student, home, schools, principals, peers and teachers. Students themselves account for about 50% of the variance. “It is what students bring to the table that predicts achievement more than any other variable.”
The influences of home, schools and peers each account for between 5 and 10% of the variance. The influence of principals is, according to Hattie, already accounted for in the school effect. (We should also remember that ‘home’ has already been influential in the early childhood situation and therefore is relevant to what the student brings to the classroom experience.) As to peers, Hattie observes, “It does not matter too much who you go to school with, and when students are taken from one school and put in another the influence of peers is minimal (of course, there are exceptions, but they do not make the norm).”
Hattie identifies 14 important influences, each with effect sizes greater than 0.4, the average effect size of all influences.[6] These include feedback, instructional quality, direct instruction, remediation/feedback, challenge of goals, peer tutoring, mastery learning, homework, teacher style and questioning. These are all sourced to the teacher! The first two influences – feedback and instructional quality – like students’ prior cognitive ability – have effect sizes of more than 1; indeed feedback has an effect size of 1.13 according to Hattie’s research. An effect size of 1 is equivalent to about a year of schooling at primary or elementary level.
Hattie also explored the difference that expert, not just experienced, teachers make. With a colleague, the literature on distinctions between expert and experienced teachers was “reviewed and the findings sent to pre-eminent researchers and expert teachers in the field for comment, changes and input”.
Five major dimensions of expert teachers were identified as a result of the consideration of the initial research findings: ability to identify essential representations of their subject, guide learning through classroom interactions, monitor learning and provide feedback, attend to affective attributes and influence student outcomes. Expert teachers adopt a problem solving stance to their work, have high respect for students and are passionate about teaching and learning. They “engage students in learning and develop in their students’ self-regulation, involvement in mastery learning, enhanced self-efficacy, and self-esteem as learners. [They] aim for more than achievement goals and to motivate their students to master rather than perform, they enhance students’ self-concept and self-efficacy about learning, they set appropriate challenging tasks, and they aim for both surface and deep outcomes.”[7]
Formative evaluation
I learned early in my enhanced interest in education (listening to a talk by Professor Paul Black of Kings College London) that student learning was most significantly advanced by formative evaluation, ongoing interaction between teacher and student in together reviewing performance and achievement. On the other hand, summative evaluations, test results, count for relatively little. It is just like the wider world of which the organisation and the family unit are parts. We are all much more likely to respond positively when those with whom we work and live take notice of and value our contribution, when what we are saying and doing is responded to, when we are encouraged to believe we can go on to greater things. And when we are trusted.[8] Threats cause resentment and generate avoidance behaviour. Waiting until you are about to leave school to find out if you have made the grade is not very helpful, not only because it is rather late: it is certainly too late to suggest remedial action.
With colleagues including Dylan Wiliam, Black explored the influence of formative assessment through literature analysis and experimentation. Learning gains, as measured by improvements in test scores of pupils involved in the relevant innovation compared with a typical group of pupils in the same test, confirm the value of formative evaluation. Acknowledging that learning is driven by what teachers and pupils do in classrooms Wiliam and Black note however, “In terms of systems engineering, present policies in the U.S. and in many other countries seem to treat the classroom as a black box.”
Though admitting that both summative and formative assessment have essential roles in encouraging a wide range of teaching and learning, the Report All Our Futures: Creativity, Culture and Education, by a Committee chaired by Sir Ken Robinson[9] noted “trying to use summative forms of assessment for all purposes, the education system downgrades the importance of formative assessment.” The Report refers to a study by Paul Black and Dylan William which reviewed 250 separate studies of the link between assessment and learning. Initiatives designed to enhance the effectiveness of the way assessment is used in the classroom to promote learning was found able to raise student achievement by the equivalent of between one or two grades for an individual – and, for England as a whole, it was estimated they would have raised the country’s position in the then most recent – the third – TIMSS (Trends in International Mathematics and Science Study) from the middle of the 41 countries involved to the top five. The gain for lower-achieving students was considered likely to be even more substantial.
Recently, with Helen Timperley (also of Auckland University), John Hattie has explored the influence that feedback has.[10] Noting that it is one of the most powerful influences on learning and achievement, Hattie and Timperley point out that the impact of feedback can be positive or negative depending on the type of feedback and the way it is given. “Feedback is conceptualized as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding. A teacher or parent can provide corrective information, a peer can provide an alternative strategy, a book can provide information to clarify ideas, a parent can provide encouragement, and a learner can look up the answer to evaluate the correctness of a response. Feedback thus is a “consequence” of performance.” Paul Black and Dylan William concluded, “the provision of challenging assignments and extensive feedback lead to greater student engagement and higher achievement”.
A synthesis of meta-analyses demonstrates that “the most effective forms of feedback provide cues or reinforcement to learners; are in the form of video-, audio-, or computer-assisted instructional feedback; and/or relate to goals. Programmed instruction, praise, punishment, and extrinsic rewards were found to be the least effective for enhancing achievement. Indeed, it is doubtful whether rewards should be thought of as feedback at all.” In the model developed by Hattie and Timperley, feedback, to be effective, must specify the goals (‘where am I going?’), progress toward the goal (‘how am I going?’) and activities to improve progress (where to next?’). Written comments are more effective than grades, as shown by Black and Wiliam who demonstrated that classroom testing “encourages superficial and rote learning, concentrating on recall of isolated details, usually items of knowledge which pupils soon forget . . . teachers do not generally review the assessment questions that they use and do not discuss them critically with peers, so there is little reflection on what is being assessed”.
Obviously goals are more effective when students share a commitment to attaining them: that commitment needs to be ”nurtured and built” and parents and teachers and peers are some of the ‘agents’ that can influence that process. Feedback on progress cannot always be positive and therefore the feedback may not be welcome; tests and assessments are but one way of dealing with the issue and are not always effective. Feedback can provide information which opens up further possibilities for learning.
Feedback about the task is more effective when it leads to better ways of processing and understanding the material. Feedback about the process is most beneficial when it helps provide cues about how to go about finding suitable places and strategies to complete the task and leads to more challenging tasks and goals. Feedback directed at the personal level such as praise is rarely effective. Citing Paul Black and Dylan Wiliam, the point is made that “when feedback draws attention to the self, students try to avoid the risks involved in tackling challenging assignments, to minimize effort, and have a high fear of failure to minimize the risk to the self”.
Feedback is not the answer but rather one answer: when combined with effective instruction in classrooms, Hattie and Timperley point out, it can be very powerful in enhancing learning. “Students construct their worlds of learning and classrooms, and it is a major argument of this article that it is crucial for teachers to understand and appreciate that providing feedback is only a part of the equation… Feedback can only be built on something … it is what happens second”.
John Hattie’s research draws on many sources, amongst them the studies of Adrienne Alton-Lee and Graham Nuthall at the University of Canterbury. Alton-Lee was at one time a Professor of Teacher Education before becoming a Senior Policy Analyst in the Medium Term Strategy Policy Division of the New Zealand Ministry of Education. She conducted classroom research and later used a “best evidence synthesis approach” to strengthen the evidence-base for policy and practice. That is all in the following essay.
[1] John Hattie, ‘Teachers Make a Difference: What is the research evidence?’, a paper for Australian Council for Educational Research Annual Conference 2003, delivered at the2003 conference of ACER which focused on “Building Teacher Quality: What does the research tell us?”. The conference brought together key researchers, policy makers, school administrators and teachers. Critical questions reviewed included teacher evaluation and teacher effectiveness, professional standards and relationship with attracting, preparing and retaining quality teachers, relationship between teacher remuneration and teacher performance. Amongst the distinguished plenary speakers was Professor Linda Darling-Hammond from Stanford University (later advisor on education to Barrack Obama in his 2009 U.S. Presidential campaign). Kenneth Rowe from ACER, John Hattie from the University of Auckland and Adrienne Alton-Lee from the New Zealand Department of Education spoke about research on teacher quality in concurrent sessions! What happened next?
[2] Michael Barber & Mona Mourshed, How the world’s best-performing school systems come out on top, McKinsey & Company, London, 2007; Mona Mourshed, Chinezi Chijioke & Michael Barber, 2010, How the World’s most improved schools systems keep getting better, McKinsey & Company, London.
[3] Teachers Matter: Attracting, Developing and Retaining Effective Teachers, OECD,Paris 2005.
[4] Aidan Mulkeen, Tony Read & Amanda Buchan, ‘Providing Effective and Equitable Opportunities to Learn’, Chapter 7 in Adriaan M. Verspoor with the SEIA Team, ‘At the Crossroads – Choices for Secondary Education in Sub-Saharan Africa, World Bank Africa Human Development Series, 2007, DRAFT
[5] In May 2010 in New South Wales (as reported by Anna Patty in ‘Streamline teacher sackings, say parents’, Sydney Morning Herald May 25, 2010), the NSW Federation of Parents and Citizens Associations asserted that the process of removing ineffective teachers should be quicker and more succinct. The state government backed away from an earlier decision to give principals the autonomy to hire and fire teachers, in response to pressure from the NSW Teachers Federation. The Education Minister, Verity Firth, said every teacher deserved ”due process”.
[6] ‘Effect size’ is a way of quantifying the size of the difference between two groups: it is the standardised mean difference between the two groups, in other words, the difference between the means of two groups divided by the standard deviation. (The ‘standard deviation’ is a measure of the spread of a set of values in a sample.) One feature of an effect size is that it can be directly converted into statements about the overlap between the two samples in terms of a comparison of percentiles. An effect size is exactly equivalent to a ‘Z-score’ of a standard Normal distribution. For example, an effect size of 0.8 means that the score of the average person in the experimental group is 0.8 standard deviations above the average person in the control group, and hence exceeds the scores of 79% of the control group. The paper by Robert Coe (School of Education, University of Durham), ‘It’s the Effect Size, Stupid: What effect size is and they it is important’ presented at the Annual Conference of the British Educational Research Association, University of Exeter, England, 12-14 September 2002, is very useful.
[7] Hattie defines surface learning as being more about the content (knowing the ideas, and doing what is needed to gain a passing grade), and deep learning as more about understanding (relating and extending ideas, and an intention to understand and impose meaning). “The claim is that experts are more successful at both types of learning, whereas both experienced and expert teachers are similar in terms of surface learning.”
[8] In the paper ‘Bad For Practice: A Critique Of The Transaction Cost Theory’ (Academy of Management Review21/1, p13-48) Sumantra Ghoshal and Peter Moran criticise the trend to substantial oversight based on the proposition that managers cannot be trusted: the result is loss of motivation and of trustworthiness.
[9] National Advisory Committee on Creative and Cultural Education, All Our Futures: Creativity, Culture and Education, Report to the Secretary of State for Education and Employment & the Secretary of State for Culture, Media and Sport, May 1999
[10] ‘The Power of Feedback’, Review of Educational Research 77 (1), p 81-112, 2007; “Feedback is information provided by an agent (e.g., teacher, peer, book, parent, experience) regarding aspects of one’s performance or understanding. It occurs typically after instruction that seeks to provide knowledge and skills or to develop particular attitudes.”