Evidence to Action
Legislation for ongoing improvement

Tim McMahon, Project Manager, Schools Planning and Reporting Project, Ministry of Education

Paper presented at National Assessment Regional Seminars, April 2000

New Zealand has a good education system. Our teachers have done a very good job. Those New Zealand-educated youngsters lucky enough to be able to do their 'OE' in Europe have traditionally been able to find a ready market for their skills.

But the world is changing. We cannot, and must not, rest on our laurels.

Back to top

Knowledge expansion

The expectations that those of us who have chosen to work in the education sector have of youngsters must match society's expectations for succeeding generations. But those expectations are not, and cannot be, static.

People who went to school, as I did, in the 1950s and early 60s, will have no trouble recognising the differences and more extreme demands that life places on young people at the beginning of the 21st century. There is equally no doubt that the world will be even more complex, diverse, fast-paced and demanding when today's students finish with their initial foray into education. When they leave school to participate in the adult world, today's students will need more, new and different knowledge, skills and social resilience than is the case today, or was ever the case previously.

This reflection suggests that if young people do not leave schooling better prepared than we were, our society will not go forward. It will go back.

Indeed the rate of expansion of knowledge, and the ever-increasing array of skills required to deal with that knowledge, suggest that every generation of school leavers needs to leave school with a different and more complex set of skills and understandings than the previous generation. Our expectations for student outcomes must be for continually better than before.

Continuous improvement is the name of the game.

Back to top

Increasing demographic diversity

In New Zealand we have a number of reasons to be conscious of the needs of the next generation. One of these is demographic. Pākeha will become a minority in the population some time around 2040. The majority will comprise people of Māori, Pasifika, or Asian heritage.

We need to think about this in the light of achievement evidence.

The majority of New Zealand students have consistently scored in the top quartile on international tests of reading and literacy. A significant minority, nevertheless, perform very poorly.

Māori and Pasifika students are dramatically over represented among those who under-perform.

The same is true of international studies in mathematics.

The demographic data shows that around half of the working-age population and parents of 2040 will be Māori and Pasifika people. A large proportion of those people are in school now.

This fact underlines the importance of improved education outcomes for all students. This fact alone justifies society's expectation that under-achievement among Māori, and Pasifika students must be eliminated. The expectation that schools will actively and deliberately seek to improve Māori and Pasifika education outcomes is not a liberal nicety, but an urgent necessity.

The requirement to contribute to increasingly improved outcomes does not rest with the profession alone. It requires education policy makers to be sharper. It will require us to seek every possible advantage from value-adding technologies. It will require increasing engagement of communities themselves.

With all that in mind, education policy is currently focused on a number of key areas:

• making achievement expectations much more explicit so they can be discussed and debated;
• establishing strong foundations in early learning;
• increasing participation at all levels;
• continuing to improve the quality of teaching at all levels;
• ensuring that schools and early childhood centres provide environments that support students and promote achievement for all;
• engaging families and communities closely in education.

The latter point is particularly important in government education policy. We know that successful students are very often motivated by the support and expectations of family and whānau. A number of recent interventions and improvement strategies have been successful because they sought to engage communities more closely in the education of their children. We see building partnerships between formal education settings and families and communities as particularly important.

Hence our work with iwi.

And hence our concentration on improving information flows between schools and communities.

Back to top

Using evidence to plan for improvement

The new legislation institutionalises the processes known as school development and self-review by formalising the requirement for schools to develop a strategic improvement plan.

Many schools undertake self-review as a matter of course. Many are involved in school development programmes. Self-review, school development, and strategic planning (in the education context) are different names for the same process. All involve working out what steps need to be made to achieve improvement.

The major thrust of the National Assessment Strategy that we, the teaching, research, and policy sectors, have brought together into current education policy, is that assessment evidence should be used to inform the next steps in our enterprise. Assessment data should inform formative practice.

The converse of this notion underpins the ideas of self-review, school development, strategic thinking or strategic planning.

Any changes we wish to make ought to be informed by good data or evidence.

Only collect assessment data you can use, and make use of the assessment data you collect!

The production of strategic plans has a bad press in some circles. Strategic planning is regarded by many as yet another management fad that has nothing to do with the real world. In many organisations, not only schools, the strategic plan is a neatly presented document that is prepared for them before "we get on with the real work". At best, the preparation of a strategic plan in those organisations has created an opportunity for some creative work. At worst it has been seen as an intolerable interruption and an unnecessary, meaningless workload addition.

Education is a complex enough job without extra unnecessary compliance burdens. The Ministry of Education is not interested in requiring anyone in schools to go through hoops to produce documentation that is of no use to anyone.

Strategic planning is not about merely "producing a strategic plan". It's about data collection, decision making, and deliberate implementation. Documentation of the process will eventually provide evidence and checks, but if a "plan" bears no resemblance to intention or reality it isn't worth doing.

Mt Roskill Grammar School, Onehunga High School and Western Heights Primary School are examples of schools that have recently produced remarkable improvements in student outcomes by having identified (through comprehensive assessment) that poor literacy was holding back student achievement. The schools introduced different programmes, that were nevertheless:

• prompted by data that showed improvement was needed;
• management-led;
• school-wide;
• involved literacy learning strategies based on well-researched good practice.

The planning of these schools was evidence driven, focused and deliberate, and involved interventions that research suggested had a high probability of success. In short, their planning was strategic.

Back to top

External reference points

Good strategic planning in the education context must be driven by good information about student performance. The strategic thinking that would seek to lift a school's student achievement from its current level to some improved position asks three fundamental questions:

In thinking about assessing student progress, it is one thing to ask the question: Has the student made progress? What is the evidence that (s)he has understood this or that idea, or can perform this or that skill?

But the harder question that goes to the heart of school improvement, is: How well should (or could) the students be doing?

The idea of seeking external reference points for assessment is the key that opens up our expectations as educators. It is a truism that students perform up or down to the expectations we have of them. The complexity and diversity of the society into which they are bound, demands that student outcomes must not be limited by ill-informed expectations.

The need to inform expectations underpins the national assessment strategy. The standards and moderated resources of the NCEA (National Certificate of Educational Achievement), the asTTle (Assessment Tools for Teaching and Learning) development, and the curriculum exemplars project will provide schools with the means to conduct good externally referenced assessments. By looking at school-wide data from those assessments, principals and senior managers will be able to determine in what classes and in what curriculum areas students or student groups are doing well and where they could be expected to do better.

An area that's sometimes overlooked when thinking about the need to be evidence-driven is the intervention strategy. If data shows that the current programme is not working, it's unlikely that "more of the same" will achieve better results.

In deciding what steps to take to achieve their goals, effective schools consult up-to-date research (or involve someone who has). The schools mentioned above that made significant gains in literacy outcomes, all chose to make changes to their programmes and to refocus their classroom practice on strategies that research information had shown to have been successful elsewhere.

The Ministry's Literacy Leadership project is a rich source of such research information.

Research tells us that the factor which makes the most difference to how well a student learns at school is the quality of the interaction between the student and her/his teacher (Hattie, 1999). Any other intervention that we consider must be geared at maintaining and improving the quality of that relationship.

Any change made at the governance or management level that does not influence the way teachers work in classrooms, will have minimal impact on student outcomes.

Successful strategic planning in education has committed leadership, involves all or the majority of staff sharing the commitment, and has each staff member taking on an amount of change that they can control. Success in small steps breeds a culture of continuous improvement.

For this reason, many of the most obvious examples of successful strategic planning in schools can be seen in innovations designed to improve literacy or numeracy or information technology skills. These are relatively focused areas; they are important and can therefore gain commitment to strategies that involve the whole staff.

Back to top

The new legislation

Recent changes made to the National Administration Guidelines, and the new changes to the Education Act, seek to redress the apparent invisibility of learning and achievement in the dialogue between the Government and schools. Learning and achievement are what schools are about. Administration, infrastructure, personnel performance and the like, are means to this end. They are not the end in themselves. If they fail to influence the quality of the student-teacher relationship, they will not improve outcomes.

The new legislation firmly places planning for, and reporting on, improved student achievement at the top of schools' agenda.

The Government has changed the Education Act to legislate that schools' charters indicate how the schools will plan for the ongoing improvements that society requires. A school's charter will have a section in which the board describes its long term (strategic) goals for student achievement, and an annually updated part in which the school will commit itself to annual targets for improved achievement: steps towards the strategic goals.

The revised Act has a requirement that, in its annual report to the community, the school's board will provide its analysis of the variance between the outcomes it has achieved and what, in its planning, it sought to achieve. Variance analysis is just another name for the self-review that leads to continuous improvement.

Copies of the annually updated part of the charter and the annual reports are to be lodged each year with the Ministry of Education.

The legislation change is not, as some have suggested, because the Government wants to retreat from self-managing schools, but, on the contrary, to strengthen self-management practice.

It is important to recognise that self-management itself is a public policy mechanism that has been put in place because the Government knows it's the best way to respond to student and community diversity. However, self-managing schools are not independent. They are still part of the network of state agencies established and funded because of the requirements of our society. Decisions taken by self-managing schools should be decisions that will contribute to society's expectations for its school system.

It is not inconsistent then, within the concept of self-management, for the Government to require the school sector to account for outcomes that society holds important. Improved literacy and numeracy, and improved outcomes for Māori and Pacific students are such outcomes.

Back to top

Target setting and reporting

There are roughly 700,000 students in the school system. Each is unique. Each has her/his own needs, unique experience, and stage of development.

That's why the quality of the relationship that a teacher can forge with each student is so important in what the child learns.

Similarly the more than 2,700 schools in New Zealand are all different. They serve different communities.

One size does NOT fit all.

It's because of this diversity that the Government has decided that schools should, using their own choice of assessment and evaluation instruments, choose for themselves the areas of greatest priority need when setting annual improvement targets. As a general rule, a school seeking continuous improvement will set as priorities the things they acknowledge they're weakest at.

It should go without saying that schools need to acknowledge they're a key part of society's strategy for the future. Accordingly, society's broader goals for education (the national priorities set out in the NAGs) should be strongly influential in this decision making. If a school's assessment programme has identified an across-the-board relative weakness in the arts, and chronic under-achievement in literacy for the Pasifika minority of the student population, it should not be difficult for the school to prioritise actions aimed at improving the literacy outcomes.

Back to top

What information does the Ministry expect to receive?

The legislation asks that schools lodge with the Ministry a copy of their charter, and their plans, as well as a copy of their annual report.

The premise that one should collect only information that is useful applies as much to the Ministry as to teachers.

The Ministry has thought long and hard about how to get useful information from schools' targets without creating perverse incentives for schools, or compromising schools' freedom to choose the difficulty of their targets or the measures or assessment tools they will use.

The Government's need for data about the effectiveness of public interventions mirrors the schools' need for data almost exactly:

• both need to know what it is that students need to learn: the schools to decide what to teach; the Ministry to advise on curriculum policy;
• both need evidence to review the effectiveness of programmes and policies and determine whether changes are needed;
• both need to identify which students need additional support;
• both schools and the Government are required to account; schools to the local community, the Government to the taxpaying public via Parliament.

The Ministry is preparing to receive and to store electronically the text of the charter. We're preparing to receive the full text of schools' annual reports. We're also going to ask schools to provide three pieces of metadata about each target:

• what curriculum area is targeted;
• what year level of students are targeted;
• what particular group of students is targeted.

It's planned to make the selection of this metadata from a drop-down list provided either on the "thinking template" CD-ROM (see Support tools), or on the web page through which schools will upload their plans and reports.

We believe the metadata, and the detail in the text that schools choose to supply, will provide very rich information for us about the outcomes that schools are prioritising for improvement and the student populations being targeted. That information will be very helpful in reviewing curriculum and support policies. It will enable us to fulfil our obligation of reporting to Parliament about the effectiveness of our intervention and support strategies.

The development of the Ministry's new Management Information and Resourcing database (MIR), creates an opportunity for collection and analysis of the information in plans and reports in a way that would scarcely have been manageable if plans and reports had to be interrogated entirely by a person reading one at a time.

MIR is being designed to enable schools to go into the system and look at all of the information the Ministry has about the school, as well as to pull off statistical data against which to benchmark themselves.

It is important to point out again that the Government has decided that schools should, using their own choice of assessment and evaluation instruments, choose for themselves the areas of greatest priority need when setting annual improvement targets.

In the draft thinking template, the developers suggest that schools should identify the information that has led them to their target setting. The suggestion is that the school could declare not only its target but also the data that led it to choose the target.

Back to top

Quality outcomes, not compliance

The Ministry knows that there has been a tendency in the recent past to confuse accountability with compliance. Requiring compliance with form-filling does not guarantee accountability.

It's for this reason that we do not propose to provide a tight "template" within which all schools write their plans and reports. By their nature, templates are constraining and they encourage the "filling in for them" mindset. The Government wishes to recreate an environment where the emphasis in information exchange is on quality educational outcomes rather than compliance.

We recognise that trust is a significant issue. Some schools say they don't trust the Ministry. Some parents say they don't trust schools. We need to break these attitudes down.

All of the Government education agencies – schools, the research community, the Ministry and the Education Review Office (ERO) – are part of the same societal mission to improve achievement outcomes, and eliminate under-achievement. We have different but complementary roles in that enterprise. It is very difficult to work together constructively in an atmosphere of suspicion.

Part of the policy environment the Government is trying to establish is one of high-trust professionalism. It is trying to get everyone in education to be more explicit about the outcomes we seek for all young people. It is trying to shift the emphasis from compliance to open debate and discussion about quality educational outcomes.

The decision to design MIR to be accessible to the owners of the information is, like the regulation requiring schools to disclose their plans and progress, based on the premise that candour and integrity is necessary to break down suspicion and mistrust.

Back to top

Consistent strategy

The Government is taking a strategic approach to supporting this focus on improvement of student achievement.

The functions of the ERO have recently been scrutinised by a Parliamentary select committee. As a result of that review the ERO has adopted a new review model known as the 'assess and assist' model.

ERO's reviews are now concentrated on student achievement and factors that directly affect learning. In particular, ERO reviews will concentrate on the quality of schools' self review: that is, on the data the school has used in deciding priorities for improvement, the strategies it has put in place, and the quality of the school's monitoring of the effectiveness of its interventions.

In Budget 2001 the Government made available several million dollars over the next three years for principal training. The training modules for new principal development are very much built around a strategic, data-driven approach to school development. The same approach is being taken to ongoing principal development.

Work is currently being planned for training boards of trustees. The Ministry is working closely with NZSTA and management advisers to ensure that the board training also focuses on a strategic approach to improvement of achievement outcomes. Board training will continue to emphasise the boundary between governance and management, but it will stress that investigation of outcome data and negotiation of achievement targets is an appropriate governance role.

Back to top

Support tools

To assist schools in implementing the new legislation, the Ministry set up a development working group of primary and secondary principals to produce a planning tool. The tool takes the form of a "thinking template" designed to assist schools in carrying out the planning process, deciding their achievement priorities, setting strategic and annual targets, and appropriate reporting frameworks. The tool is not a model strategic plan for schools to copy. It will help schools understand the school development process and to mainstream data-driven self-review and continuous improvement.

On the advice of the working group, the planning tool has been designed to be used flexibly. Schools that are very familiar with school development or strategic planning would use it differently from those that were facing the exercise for the first time.

The draft tool has been trialled by about 120 schools. We are meeting with all of the those schools over the next month to determine in what ways the tool was helpful and how it may be made more helpful. We are on track to distribute to all schools a final version of the tool by the end of July.

Back to top

Conclusion

Effective school development is achievement evidence-driven: it builds on strengths and seeks to continually develop practice so student outcomes are improved.

The new legislation requiring school charters to focus on student outcomes and to discuss openly their improvement plans and progress, reinforces the Government's policy objectives of:

• building capability in the education profession;
• encouraging all involved with education to be more explicit in their expectations;
• promoting professionalism and trust;
• promoting dissemination of effective practice;
• eliminating under achievement.

The requirements of the strategic planning legislation build on, and seek to institutionalise, effective practice in evidence-based school development.

All of our students must be equipped to live and learn in a world that is increasingly complex and diverse.

Reference: Hattie, J. (1999). Influences on student learning. Inaugural professorial lecture, University of Auckland.

Back to top

Assessment tools
Assessment tools search
asTTle
e-asTTle
Exemplars
Self/Peer assessment