The Development of Exemplars in
New Zealand
Background and Rationale

Mary Chamberlain
Ministry of Education
March 2001

Abstract

Exemplars are nationally moderated examples of student work that are annotated to illustrate learning, achievement, and quality in relation to the levels described in national curriculum statements.

This paper provides a brief background to and rationale for the development of exemplars of student work in New Zealand. Drawing on recent research, it sites the development of exemplars as an educational strategy that will contribute to effective curriculum, pedagogical, and assessment practices and provides a rationale that explains how the development of exemplars will support the Government's strategy to raise achievement and reduce educational inequities.

The proposed development process, teacher professional development programmes, and the expected use of exemplars are also discussed.

Why develop exemplars?

The Government's aim is to reduce inequities in students' achievement and to raise achievement levels for all students. It is expected that the development of exemplars will help to both raise achievement and improve educational outcomes for underachievers.

New Zealand's official curriculum has been developed as an outcomes-based model in seven essential learning areas. Each essential learning area is organised according to eight curriculum levels in a national curriculum statement. The achievement objectives in each curriculum statement help set the direction for learning through descriptions of the broad outcomes that students are expected to achieve.

Teachers' judgments about progress and achievement are guided by their understanding of the long-term educational outcomes signalled in achievement objectives. However, it is difficult to express progress and quality in words alone. Words are needed to describe criteria and examples are needed to demonstrate quality.

Achievement objectives alone are insufficient to provide the clarity of focus needed to signal learning progressions that assist teachers to provide focused formative feedback to students. They are also insufficient to help teachers make consistent and comparable judgments about students' quality of performance for reporting purposes.

Exemplars have considerable practical value because they provide concrete referents for verbal descriptions and valuable keys into complex evaluative frameworks (Sadler, 1987).

Background

The development of exemplars was initially proposed in the Government's Assessment Green Paper (Ministry of Education, 1998). This paper proposed that exemplars would be developed in both English and Māori at levels 1 to 5 of the curriculum in each of the essential learning areas. It was proposed that exemplars would provide clear examples of expected levels of student achievement in relation to the achievement objectives in national curriculum statements.

Responses to the Assessment Green Paper showed considerable support for the development of exemplars. However, there were concerns that the exemplars of students' work could potentially become a de facto curriculum if teachers treated the achievement objectives demonstrated in the exemplar material as the most important ones. Some respondents thought that although "reinventing the wheel" was time-consuming, locally developed initiatives meant that teachers had "ownership" of the outcomes, and also gained new professional skills and knowledge during the development process. There was therefore some concern that national exemplars could be used to take the place of locally developed exemplars. Issues were also raised concerning the advisability of illustrating low achievement as the Green Paper stated that "the exemplars could also illustrate a range of student performance, from high-achieving to low-achieving students, providing realistic expectations of what students should achieve".

The Assessment White Paper (Ministry of Education, 1999) announced that exemplars would be developed in both English and Māori at levels 1 to 5 of the curriculum in each of the essential learning areas. It announced that exemplars would be real pieces of work produced by students that illustrate or exemplify features that teachers, parents, and students could point to as meeting achievement objectives.

In August 2000 the Minister of Education, Hon Trevor Mallard, announced that the development of exemplars of students' work in both English and Māori at levels 1 to 5 of the curriculum would commence as part of a broader national assessment strategy on the grounds that exemplars would:

• help teachers to focus on important aspects or indicators of a student's achievement;
  
  
• ensure a more consistent interpretation of what constitutes achievement in relation to achievement objectives;
  
  
• model effective practice; and
  
  
• describe the next developmental steps for the student whose achievement was shown in the exemplar.

How will exemplars help to enhance learning and achievement?

It is expected that exemplars will:

• make the desired outcomes of learning more explicit by highlighting critical features of students' work and signalling important things to watch for, collect information about, and act on to support growth in learning;
  
  
• provide a basis for discussion about important qualities, aspects, or indicators of learning with students, teachers, and parents;
  
  
• provide reference points that will support teachers' professional judgments about the quality of their students' work.

It is also expected that teachers will be able to use exemplars with students to:

• draw students' attention to critical features of the work;
  
  
• help set specific and challenging learning goals;
  
  
• provide focused feedback to students that supports sustained learning conversations; and to
  
  
• help students understand what features to look for when peer and self-assessing.

While exemplars will help signal the direction for learning, their effective use in supporting learning will depend on the interaction between teacher, learner, lesson content, and the wider learning environment.

How will exemplars support effective curriculum educational practices?

Curriculum is the sum of what happens in a school, it includes the experiences, interactions, teaching materials, and the environment in which students learn. The New Zealand Curriculum Framework (Ministry of Education, 1993) and supporting curriculum statements outline New Zealand's official curriculum. Teachers plan activities that form the implemented curriculum, the experiences, and opportunities offered to students. Teachers and students remake the curriculum through their interactions. What students actually achieve, the attained curriculum, is a result of the interactions that occur between teacher and learner, learner and subject matter, and the broader socio-cultural context.

Freebody et al. (1996) found that for many students classroom experiences were not closely related to clear expectations or shared goals. This made it difficult for students to identify what was required of them, and those students whose cultural or economic background was different from that of the teacher found teaching confusing or inaccessible. Exemplars will provide clear expectations that will discourage deficit explanations of student performance based on cultural and social class stereotypes (Hill et al., 1998) and will support teachers to focus on the next learning steps for each student regardless of their background.

Teacher expectation about what their students are likely to achieve is powerful. Exemplars will provide a vision of what is possible. They will be developed to capture what it is known children can do, as well as what is considered desirable, and they will be used to help shape teacher expectations about effective learning and meaningful criteria.

There is consistent evidence that clear expectations that focus on educationally significant learning and high but attainable standards raise achievement (Hattie,1999). To achieve this clarity, teachers need to develop shared understandings about what is meant by achievement, what progress means and what makes quality work. "The area of progression is one of the least understood areas, yet this is crucial for designing curriculum and enhancing student achievement" (Carr, McGee, Jones et al., 2000, p. 60).

It is proposed that exemplars be linked to curriculum levels using a "best fit" approach after progressions have been developed. Most things in the real world do not have crisp boundaries, for example, there is not always a clear distinction between yellow and orange. However, if they are given an example, most people will be able to say if something is more yellow than orange. They will be able to say, on balance, in which category something belongs (Sadler, 1987), or in other words, make a judgment about best fit.

The best fit approach is supported by deriving the criteria for assessment from the descriptions of achievement objectives ..." (Ministry of Education, 1997).

The challenge will be to ensure that exemplars show progressions as possible pathways, show that quality is expressed in many ways, and that clear indicators, expectations, and consistency in teacher judgment don't lead to conformity in terms of one approach or getting to one right outcome or answer.

It is not intended to develop exemplars for each achievement objective or learning outcome. Teachers and schools will be encouraged to continue to use or to develop their own exemplars to supplement those produced nationally. This will reinforce the idea that quality can be expressed in a wide variety of ways, rather than seeing exemplars as models to be copied.

How will exemplars support effective pedagogical educational practices?

Pedagogy refers to the interaction that occurs between students and teachers in teaching-learning situations and the theories about teaching, learning, and environment that inform teaching practice. It is the interaction between teacher and learner that has the most impact on learning outcomes.

Some themes emerging from pedagogical research that could be considered during the development of exemplars include those related to student ownership and control of learning, goal setting and planning, self regulated learning and metacognition, motivation and scaffolding. Authentic learning that relates to students' prior knowledge and the desirability of designing effective learning environments that are relevant to the students' world are also important themes.

While there is no recipe for pedagogy and no one strategy that will work in every learning situation, Newmann (1996) found that high levels of what he termed "authentic pedagogy", specifically intellectual engagement and connectedness, enhanced students' learning outcomes for both mainstream and equity target groups.

Authentic pedagogy as defined by Newmann (1996) involves the following key elements of classroom teaching and learning:

• higher order thinking;
  
  
• depth of knowledge and understanding;
  
  
• sustained and substantive classroom conversation; and
  
  
• the relevance of the lesson to the real world.

The Australian School Reform Longitudinal Study (SRLS) built on Newmann's work with the University of Wisconsin's Centre on the Organisation of Restructuring of Schools (CORS) and identified a further 16 coding categories which they called "Productive Pedagogy" (University of Queensland, 1999a, 1999b). The researchers say that a focus on these strategies improved student outcomes for diverse student populations including under achievers and the gifted and academically talented.

During the process of developing the exemplars the possibility of highlighting significant interactions between teacher and learner as part of the annotations that accompany exemplars will be explored. The ways exemplars can be used to influence pedagogy will be a focus of the trial phase of developing the exemplars and this will inform the professional development programmes that will follow the publication of exemplars.

How will exemplars support effective assessment educational practices?

Assessment information provides feedback to improve teaching and learning. It allows teachers, schools, and systems to report on what students have achieved at a certain point in time and it provides accountability information that assures parents, boards of trustees, and the public about the quality of education.

Assessment for improving teaching and learning

In 1988 Terry Crooks published a review of more than 240 studies on the impact of assessment on students' classroom learning and achievement. He found that in order to improve learning students need:

• to be offered clear learning outcomes;
  
  
• specific, constructive, and regular feedback;
  
  
• a strong sense of involvement in the assessment process; and
  
  
• the opportunity to set and achieve specific learning goals.

Paul Black and Dylan Wiliam from Kings College in London built on this study and reviewed 250 major studies published between 1988 and 1998 on the links between assessment and learning (Black & Wiliam, 1998). They found that in order to improve learning students need:

• effective feedback (not just feedback that serves social and managerial purposes);
  
  
• to be actively involved in their own learning and assessment;
  
  
• teachers who adjust their teaching to take account of the results of assessment (teachers need to know about progressions and students' learning needs);
  
  
• teachers who ensure that assessment practices impact positively on students' motivation.

The Black and Wiliam (1998) study concludes that a focus on the above aspects of formative assessment results in achievement gains for all students and higher gains for underachievers.

Exemplars will be able to be used to inform evaluations that are ongoing, formative, pluralistic, qualitative, in context – and shared with the learner.

Assessment for reporting

Parents and communities have a right to quality information about their children and their schools. Exemplars will support the development of a manageable qualitative approach to reporting to cover a wider range of important learnings and balance narrower quantitative measures.

When teachers make qualitative judgments intuitively without reference points or guidelines a number of problems may occur that lead to this information being less useful for reporting purposes.

Problems may include:

• unreliability because of discrepancies in judgments between teachers;
  
  
• inconsistencies in teacher judgment over time;
  
  
• a teacher's personal view of the student clouding their judgment;
  
  
• the carry over of positive or negative judgments from one assessment to the next;
  
  
• a general tendency towards leniency or severity;
  
  
• the influence of extraneous factors, for example, neatness or handwriting.

The development of exemplars will increase teachers' ability to make reliable and valid qualitative judgments that may be used for reporting purposes.

Accountability

It is important that schools take responsibility for the quality of the education they provide and that they are able to demonstrate the quality to others (parents, community, government) who have the right to know what has been achieved and how. Those outside schools cannot control the quality of education within them. If this is attempted it is likely to lead to "poor working morale and to a sterile game where the evaluation criteria may be reached, but both essential information and possibilities for development are lost" (Kauko & Kauppi, 2000).

Exemplars may be used to help set, and assess the achievement of, clear and challenging goals for individual students and groups of students at classroom and school levels thus leading to accountability with insight which increases the likelihood of responsive reflective action (Johnston, 1989).

The development process

The process for developing exemplars will be an iterative one and may be changed during development and trialling.

Teacher involvement

Unless educational change is owned by teachers and generated interactively between centralised authorities and local schools, it cannot make a substantive difference in generating changed pedagogical practices (Fullan, 1993).

Teachers are more likely to come to new understandings and try out new possibilities if they are engaged in the development process from the outset. It is proposed that teachers will be involved in gathering and analysing collections of students' work, and in in-school and between school discussions and activities that make implicit knowledge about criteria, indicators, and progressions explicit.

The process for developing exemplars development process will help increase teacher professionalism by providing teachers with opportunities to share their understanding of curriculum, and by providing opportunities to build on teachers' knowledge of what progress looks like in each curriculum area. Focused online discussion forums about exemplars will also contribute to the development of professional learning communities.

Clusters across transition points

There is increasing evidence that transferring schools for any reason leads to a dip in learning for many students. After reviewing the research on transition and transfer from the United Kingdom and North America, Galton et al. (1999), estimated that 2 out of 5 students failed to make progress during the year immediately following the change of schools. The authors noted that contributing factors may be related to teaching methods and the interpretation and depth of treatment of curriculum. For these reasons the development of exemplars will involve cluster meetings set across transitions to encourage continuity through sharing examples of activities and expectations.

Developing a framework for what will be exemplified

Exemplars can't cover the whole curriculum and every conceivable outcome, but they can provide a framework of reference points to clarify expectations and guide teaching, learning, and assessment practices.

Teachers, expert teams, and advisory groups will be involved in deciding what will be exemplified in each curriculum area. Decisions will be mapped against the curriculum. Existing resources and materials will also be investigated. For example, National Education Monitoring Project (NEMP) material will be examined to consider what students at years 4 and 8 actually do. Some of the NEMP examples may be able to be used in the process of developing exemplars. Similar links may also be made to the Assessment Resource Banks in English, mathematics, and science.

It will be important to ensure that exemplars reflect significant learning outcomes with integrity and that important qualities of learning are not ignored. Issues to be considered relate to the range and balance of exemplars including the extent to which exemplars:

• are transdisciplinary and include essential skills, attitudes, and values;
  
  
• reflect what students can do and also what is considered desirable; and
  
  
• reflect a wide variety of cultural contexts and many ways of knowing in their content and contexts.

Gathering examples of student work

Exemplar developers in each curriculum area will work with teachers in a representative sample of schools to develop exemplars of student work. Teachers will discuss indicators of progress and the meaning of quality work at each level. With the support of exemplar developers, teachers will add commentary to selected exemplars.

Selected exemplars will be submitted to cluster and regional meetings for moderation and a subset will then be forwarded for national moderation.

Trialling

Developers will provide feedback about the results of the moderation process to original development schools, discuss ways in which teachers would like to use exemplars, and trial some of these. Developers will record processes for possible use in exemplar support material.

Teachers in schools that have not been involved in the exemplar development will make judgments about the level of the work shown in exemplars and their judgments will then be compared with the results from moderation processes. Teachers in these schools will also trial the use of exemplars.

Research

It is important to ensure that exemplars will be used in ways that make a positive difference to student achievement and that they will not have unintended negative effects. Research will accompany the development of exemplars and help shape it. Key questions include the following, which are grouped by categories.

Process

• What issues have been identified by development teams?
  
  
• How is a similar "look and feel" maintained amongst the exemplars?
  
  
• How do national exemplars build upon local or school-developed exemplars?
  
  
• How understandable and usable are the exemplars, for instance, indicators, advice.
  
  
• What impact has development had on schools and teachers involved?
  
  
• Are project teams' approaches and rationale similar?

Use

• Who is using exemplars and in what ways?
  
  
• Are exemplars making a difference in ways schools judge student learning and achievement? How do exemplars help?
  
  
• What impact are exemplars having on teaching practice – those involved, those receiving exemplars?
  
  
• Has the use of exemplars made a difference to the feedback students receive about their learning and achievement?
  
  
• Is there a risk that exemplars become a de facto curriculum?
  
  
• Do exemplars create an additional workload for teachers?
  
  
• What impact have national exemplars had on schools who have previously developed their own, and are schools supplementing the national exemplars with their own ones?
  
  
• What kinds of feedback were provided by the trial schools – what issues have they raised about the use of exemplars?
  
  
• Are the exemplars being used to help with reporting to parents, and if so, how?

Implementation

• What issues need to be considered and addressed before the development moves towards full implementation?
  
  
• What issues for the design, production, and dissemination of the exemplar resources need to be addressed?
  
  
• What issues about teacher support and professional development need to be addressed?

The use of exemplars

Investment in exemplars will not on its own make any difference to student outcomes. The difference will happen when motivated and skilled teachers use the exemplars as one of a range of strategies and tools.

It is expected that teacher professional development programmes to support the use of exemplars will contribute to the development of teacher professionalism. Effective use of exemplars is likely to require professional development programmes that:

• help teachers use exemplars to set specific challenging goals for students;
  
  
• assist teachers to integrate the use of exemplars with effective pedagogical practices and other assessment tools;
  
  
• use exemplars as a focus for discussions that make the implicit knowledge held by expert teachers more explicit;
  
  
• draw on exemplars to increase teachers' confidence and curriculum knowledge;
  
  
• support teachers to use exemplars to report to and provide more focused feedback to children, parents, and whānau;
  
  
• help teachers to use exemplars to modify teaching programmes; and
  
  
• use exemplars to contribute to teachers' and schools' capacity to engage in reflection, critique, and renewal.

It will be important to use the development of exemplars not only to create expectations and help teachers make better judgments, but to help them use evidence from exemplars to support learning and change practice. It's what teachers ultimately do with exemplars that matters most.

Flexibilty

Students develop at different rates and their aspirations, interests, and learning needs are at the heart of planning for teaching and learning. While exemplars will further clarify possible directions, expectations, and what learning means, students and teachers will be able to negotiate the pace, direction, and outcomes for learning. Teachers will also be encouraged to expand and increase the relevance of exemplars by developing their own exemplars that expand on the contexts in the national exemplars.

Key questions

Some key questions to ask about the exemplars as they are developed include:

• Can teachers see under the surface of exemplars and recognise the intellectual substance they represent?
  
  
• Do they reflect important ideas or arrays of ideas that require sophisticated thinking?
  
  
• Are the ideas in the exemplars explicit to teachers, do they encourage teachers to reflect on their teaching?
  
  
• Do exemplars capture difficult to assess dimensions to learning?
  
  
• Do exemplars represent subject or discipline knowledge, for example, do science exemplars reflect science knowledge?

Challenges and tensions

There are many challenges inherent in the way we develop exemplars and make judgments or take action using them. There is a need to recognise the complexity of contextual factors in developing shared meanings and values.

Tensions include the tension between increased demands for comparability of assessment with increased flexibility for delivery of school curriculum and between clearer expectations for clarity and clearer expectations for control, conformity, and standardisation.

It will be important to keep a balance between a focus on achievement and a focus on learning. Students not only need to be thinking about improving performance but they also need to be intrinsically motivated to learn, to wonder about how things work, to be inquisitive, and to ask questions.

Conclusion

As Beeby once noted, knowledge may be doubling every few years, but wisdom certainly isn't. Exemplars have the potential to contribute to teachers' and students' educational wisdom through the provision of learning oriented signposts. They have the potential to contribute to effective curriculum, pedagogical, and assessment practices that clarify and improve learning outcomes and reduce inequities. The challenge will be for learning communities of teachers, developers, researchers, and policy makers to work together to ensure that this potential is realised.

Appendix: Categories of Productive Pedagogy

The asterixed * items are those that both the CORS study and the SIRLS study suggest are key to changed student outcomes.

The SIRLS study suggests that different combinations of all of these strategies are effective for different students.

References

Black, P., & Wiliam. D., (1998). Inside the black box. London: Kings College London.

Carr, M., McGee, C., Jones, A., Mckinley, E., Bell, B., Barr, H., Simpson, T. (2000). Strategic research literature review, The effects of curricula and assessment on pedagogical approaches and on educational outcomes. Wellington: Ministry of Education.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58(4), 438–481.

Freebody, P., Ludwig, C., & Gunn, S. (1996). Everyday literacy practices in and out of school in low socioeconomic urban communities. Brisbane: Griffith University.

Fullan, M. (1993). Change forces: Probing the depths of educational reform. New York: Falmer Press.

Galton, M., Gray, J., & Ruddick, J. (1999). The impact of school transition and transfer on pupils' attitudes to learning and their progress. London, Department for Education and Employment, Research Report RR131.

Hill, S., Comber, B., Louden, B., Reid, J., & Rivalland, J. (1998). 100 children go to school: Connections and disconnections in literacy experience prior to school and in the first year of school (Vols 1–3). Canberra: Department for Education, Employment, Training and Youth Affairs.

Johnston, P. (1989). Constructive evaluation and the improvement of teaching and learning. Teachers College Record, 90 (4, Summer), 509–528.

Kaukao, H., & Kauppi, A. (2000). Evaluation in Today's Education: From control to empowerment. Lifelong Learning in Europe, (2), 68–75.

Ministry of Education. (1993). The New Zealand curriculum framework. Wellington: Learning Media.

Ministry of Education. (1997). Planning and assessment in English. Wellington: Learning Media.

Ministry of Education. (1998). Assessment Green Paper. Wellington: Learning Media.

Ministry of Education. (1999). Assessment White Paper. Wellington: Learning Media.

Newmann, F. (1996). Authentic achievement: Restructuring schools for intellectual quality. San Francisco: Josey Bass.

Sadler, R., (1987). Specifying and promulgating achievement standards, Oxford Review of Education, 13, 191–209.

University of Queensland. (1999a). School Reform Longitudinal Study literature review. St Lucia, QL. (Author).

University of Queensland.(1999b). School Reform Longitudinal Study (SRLS) Theoretical rationale for the development of productive pedagogies. St Lucia, QL. (Author).

Back to top

Assessment tools
Assessment tools search
asTTle
e-asTTle
Exemplars
Self/Peer assessment