TKI - asTTle: Assessment Tools for Teaching & Learning: About asTTle

West Eyreton School, Rangiora

AtoL school story 2005

At a seminar in Christchurch in 2002 West Eyreton school principal, Jillian Gallagher, heard John Hattie (Head of University of Auckland School of Education) talk about assessment and how the asTTle CD-ROM tool could help teachers create and analyse tests in reading, writing and mathematics. His remarks on feedback, quality input and sharing learning criteria with students really interested her.

Later that year, Jillian again heard Professor Hattie talk about the asTTle trial. She saw the potential to shift the teaching practice at her school and make a difference to learning and teaching.

West Eyreton, a decile 8 school with a roll of 152 students in North-Canterbury, was already using a range of assessment tools to identify student progress and achievement in literacy and numeracy, including Progressive Achievement Tests (PATs), teachers' monitoring and testing, running records and class descriptions. Jillian felt PATs were not a true measure of the New Zealand curriculum achievement levels.

"We were teaching and assessing to the achievement levels but administering a standardised test which was not based on what we were teaching. The PAT could not be translated into classroom practice whereas asTTle directly linked the assessment results to each student's next steps for learning and provided a bank of excellent resources to use in the classroom."

The school also used School Entry Assessment (SEA) with new entrant five-year olds and 6-year Net at the end of the first year at school. The levels of the curriculum document were the main benchmarks. In reading, Probe and PM Benchmark Running Record tests were the best available tools to ascertain progress and learning needs. In maths, pre- and post-tests were used.

Teachers of senior school students found it difficult to plan literacy programmes, especially reading and writing, as there was no structured literacy programme compared with the programmes in the junior school. There were no senior school diagnostic 'tests' with clearly defined steps for teachers to use.

Jillian said," I realised that teaching at this level you had to continually try a variety of programmes, strategies and resources to find what would meet the needs of your students. asTTle with its New Zealand means-based assessment and next step learning gave focus and constructive guidelines for senior students' programmes".

Recent changes in assessment practice and professional development

Historical – the asTTle trial

In 2002, West Eyreton teachers decided to become part of the Ministry of Education's asTTle trial project. They wanted to explore the use of asTTle because it promised benefits for teaching and learning, and provided a huge range of applications for students, teachers, parents and the board of trustees.

asTTle supports teaching and learning in the following ways:

learning criteria can be identified then shared with students and parents
it gives feedback through a range of useful information which is visual and clear
it shows teachers, parents and students the next steps for learning
it supports teachers and gives them direction / resources for the next steps for learning.

Assessment to Learn director, Steve Edwards (School of Professional Development, Christchurch College of Education), led the school on the asTTle trial and supported teachers by responding to their questions and explaining interpretations and misunderstandings.

Next step learning

Students' reading and writing were tested to gauge a broad understanding of the asTTle tool, its possibilities and how it could improve the quality of learning for senior students. Teachers entered the data and analysed their own results.

Once teachers knew exactly where their students were, they used asTTle data to guide their future decisions. Teachers were reassured, knowing their teaching was focused on the students' learning needs. They had clear descriptions of what students could do, should be able to do, and needed to learn next, as well as specific information about weak areas and strengths. National means showed how students were performing at their expected age levels. All this valuable data informed teachers' decisions for next-step learning and could be used to explain what was happening in classrooms to students and parents.

As a result of participating on the asTTle trial the school embraced asTTle and results are now used to inform teaching and learning as well as to outline and report achievement to the board of trustees.

asTTle informs teaching and learning

"asTTle is an extremely supportive tool for teachers to focus their teaching on the needs of their students," says Jillian. "It uses the formative teaching strategies well".

"asTTle is also a good barometer," continues Jillian. "I enter pre-test results onto an Excel spreadsheet, and can quickly see trouble spots and identify students with learning needs. The post-test results for each child are entered alongside the original results and progress measured and reported to students and teachers".

A year later, the school was accepted onto the Assess to Learn (AtoL) Contract. In 2003–2004, they developed a new English programme for levels 1–4, based on the English exemplars and incorporating the asTTle strands. Jillian notes, "Our English programme aligns learning and teaching to the New Zealand curriculum document, exemplars and asTTle". (Refer to Appendix 3).

Tracking and reporting achievement – mapping student needs

While on the AtoL programme the school developed a cumulative record form for recording the asTTle results which shows the expected curriculum levels at each year of schooling. The record form is designed with a shaded curriculum line that provides teachers with a clear idea of where each child should be achieving against the New Zealand curriculum levels. This record form is a cumulative grid which shows the expected achievement at each year level. As the cumulative record identifies children who are achieving above or below the expected curriculum level, teachers can also use it to select children for the GATE (Gifted and Talented Education) programme and learning support programmes. (Refer to Appendix 1)

All asTTle information is collated to build a picture of learning needs and strengths. asTTle reading results are triangulated against Running Record and teachers' assessment information. This process led to the school making an important decision that has changed their use of Running Records and Burt Tests for analysing reading achievement. As from 2005, the school's methods for identifying reading targets are now based on the following information:

Three years ago, the curriculum review information for reading was based on Running Record results. This showed that around 94% of year 3–8 students were reading at or above their chronological age but now with the introduction of assessment tools such as asTTle teachers know that these students although reading very well are not always achieving at the Expected Curriculum Level (ECL) in the deep thinking skills such as the critical thinking and processing information strands.
Through close scrutiny of the teachers' records and comparing with asTTle results, teachers find that asTTle and the exemplars are giving them a more complete picture of reading achievement than that of the Running Record information. Running Records have traditionally measured children's ability to read and comprehend conventional fiction and non-fiction texts but asTTle is testing children's ability to interpret information in a wider range of genres. It shows students' weaknesses and strengths in their ability to fully comprehend information in such text types as recipes, dictionary pages and online information. In the past teachers measured reading achievement through Running Record tests and Burt word tests, but with asTTle they feel they have a deeper understanding of students' attainment.
To assist teachers to identify a clearer picture of student achievement, they designed all of their school record forms to show the Expected Curriculum Level by inserting a Plunket line (a line on a cumulative record that shows the expected level of achievement for year of schooling. This line is shaded on Appendix 1, asTTle reading results West Eyreton School cumulative record form to show our school's benchmark of where students should be achieving).
There are three focus areas for the schoolwide reading cumulative record sheet: knowledge, understanding and inference. The school believes that these strands are reliable indicators of children's reading ability.

How did teachers select the targets? What data did they collect?

In week 4 of term one the teachers administered the asTTle reading to year 3–8 students. The teachers then analysed the various data displays in a number of ways:

They looked at:

The asTTle Console report for an overall picture of achievement. This showed that year 7 and 8 students were doing very well at surface features (Running Records show these), they were slightly above the New Zealand mean in each of the three reading strands and the reading scale was above the New Zealand mean. These students had a reasonably positive reading attitude and the deep thinking area was just above the New Zealand mean.
Teachers then entered the results of the asTTle onto the school's cumulative record sheet. (Refer to Appendix 2)
Remember, 94% of students read at or above their chronological age and the asTTle showed that the year 7 and 8 class were achieving at or just above the New Zealand for deep thinking skills but where were they on the Expected Curriculum Level sheet?

What asTTle identified was that many of the year 7 and 8 students were reading well below the Expected Curriculum Level in the area of critical thinking skills although their general achievement was around the New Zealand mean.

(In fact, the only students the Running Records showed reading BLA for chronological age are those in the yellow highlighted areas.)

So, what did all this mean? This is what took the most time – analysing and understanding what the results were showing. Staff meetings were held and all teachers brought along their 2004 end of year reading assessment summaries (including Running Record information) and they discussed and analysed them together.

Next all this information was collated and an analysis of reading achievement was developed across the school and again presented to the teachers. They discussed this information at a staff meeting and identified which year group and areas to target for the year. They decided it was to be a particular year group, not a whole school literacy target.

The data presented in Appendix 2 informed the teachers that:

Year 4, 5 and 6 students were achieving well in the critical thinking area and in understanding and knowledge areas as well (these were not the year levels targeted in reading).
However, teachers identified that a significant number of our year 7 and 8 students were not achieving at the expected curriculum levels in the critical thinking area.
53 % year 8 and 43% year 7 were achieving below the Expected Curriculum Level in inference.
However, the Running Records had only identified two students in year 7 and 8 reading below their C.A. – asTTle had identified 17 students achieving below ECL and this is because the Running Records give only some of the picture.
Schools are charged with delivering the New Zealand curriculum but in reading this extends to a much broader range of texts than previously focused on. Analysis of asTTle reading tests showed us that critical thinking skills were low. This was due in part to students' weaker skills in interpreting the full range of text types.
AsTTle alerted the teachers to this anomaly in reading achievement.
This forms the basis of the approach to analysing reading at the school.

Developing the action plan? (Refer to Appendix 4.)

To find the specific needs of the target group of eight year 7 and 8 students (red highlighted students) the asTTle results were re-entered into the asTTle data base to determine these students' exact areas of specific learning difficulty. Teachers then worked through this plan with the year 6–8 teachers and these learning needs were entered onto their class reading plans.
The last stage was to develop the action plan for raising the school targets. Parts of this plan were also applicable for use with the class targets. Before the action plan was completed, teachers also sought advice from the Resource Teacher Literacy teacher about students' needs and built this advice into the action plan.
Regardless of whether it was a class or school target, each teacher fed this information onto their school reading plans to develop a focused programme for achieving their target areas.
As a result of whole staff meeting discussions all teachers in the school had identified class targets to work at in their reading programmes this year.

2005 and beyond:

Now, towards the end of 2005, because teachers understand what the asTTle information is telling them, they use Running Record testing and BURT testing as part of the triangulated assessment process. Year 4 students who are achieving at or above the ECL on asTTle reading tests may not need a Running Record or Burt test but those students achieving below the ECL on asTTle, or on the teachers' assessment records are still administered both.

The school is currently working with Garry Taylor (ATOL adviser Christchurch College of Education) to develop an Excel spreadsheet to analyse the shifts in achievement using the asTTle data.

"The first spreadsheets were originally designed to measure sublevel shifts as a result of entering entry and exit point data. asTTle scores would have been easier to work with as they are just a number, but they are not as meaningful as having curriculum related sublevels. Exported .csv files from asTTle are placed in the spreadsheet in entry and exit point areas. By using a numerical equivalent and the Vlookup function the sublevels are then able to determine that, for example, a student progressing from level 2A to 3P has had a two sublevel shift.

The second spreadsheet was designed after discussions of the analysis that Jillian Gallagher did with the entry point asTTle data. The information gained was invaluable as it measured each student against expected curriculum level performances. Through using the same .csv export data and the concatenation function the spreadsheet was automatically able to determine whether a student was working below, within or above their expected level of performance. This was based on the grids the teachers at West Eyreton had developed. For example, a year 4 student performing at level 3A will be identified as working 'above' the school's expected level of performance. A similar sheet has also been developed for the strategy stages in numeracy."(Garry Taylor)

Keys to asTTle success

Jillian works to engage her colleagues in a collaborative process to ensure they all understand the work of the school.

"I believe that the job of raising targets is not only the class teachers' responsibility – it is a whole school responsibility."

The following points summarise what she has learnt as an asTTle leader:

Adequate time needs to be allocated to the process for identifying the school targets to ensure total buy-in from all teachers.
Team leaders need to be prepared to lend support to teachers in the planning stage.
Targets need to be clear and specific – they need to be SMART = Specific, Measurable, Attainable, Realistic, Time Driven.
Ongoing professional development in the area of raising the targets needs to be provided.
Team leaders need to tap into the wealth of experience within the school.
It is important to celebrate each stage of the journey.
Staff meetings need to be facilitated where teachers can discuss how it is going, what is working, not working; high points and low points; areas requiring help; input from others etc.

Enthusiastic teachers

West Eyreton School teachers are enthusiastic about asTTle. It has proven to be an effective tool, supporting their teaching and building deeper knowledge of what needs to be done to improve learning.

Appendix 1 Cumulative record form for the asTTle results

AsTTle Reading Level

Expected working level by

Knowledge

Understanding

Inferences

Yr 9 &10

Year 9

Yr 8&9

Year 8

Year 7

Yr 6& 7

Year 6

Year 5

Yr 4 & 5

Year 4

Below 2B

Yr at school

calendar year

Mark level achieved with a cross.
Shaded area depicts expected level

Appendix 2

AsTTle Reading Level	Expected working level by	Knowledge					Understanding					Inferences
5A	Yr 9 & 10
5P	Year 9
5B	Year 9															2
4A	Yr 8 & 9				1	2					1
4P	Year 8			1		2					1
4B	Year 7		1		1 2	3			1	2	5			2	2 1	4
3A	Yr 6 & 7		2		1	3			1	1	2			1	1	1
3P	Year 6			5	1			1	1	5	1		1 1	2	5	3
3B	Year 5	4	3	5	5	6	3	3	4	5	4	5	5	2	6	1 2
2A	Yr 4 & 5	4	3	1	2		4	5	4	3	3	2	3	3
2P	Year 4	7	4	4	3	1	6	5	3	1		9	4	2 1	1	2
2B	Year 4	7	1				13					15		1
Below 2B		9 1	1				2 3	1	2			2	1	2
	Yr at school	4	5	6	7	8	4	5	6	7	8	4	5	6	7	8
	calendar year

Appendix 3: An example of the reading planning sheet

West Eyreton School – Reading programme Level 3

Achievement objectives	Specific learning outcomes. (Assessment skills are written in bold) Students will:	Contexts for learning
Personal: Select and read independently, for enjoyment and information, different contemporary and historical texts, integrating reading processes with ease.	Choose books appropriate to their reading level and interest. Set own purpose for reading. Know texts are constructed for different audiences. Visit and use school and community libraries. Select and read independently a range of texts. (incl NZ material) Read often and with concentration Comprehend at or above chronological age level using skills of: retelling, inferring, predicting, asking and answering questions	Reading to, with and by students: Includes: library books (school library, National Library etc) magazines/ newspapers school journals articles email/internet, computer programmes word study programmes language skills lessons writing skills lessons reading a variety of texts to students, shared reading guided reading, guided silent reading reciprocal reading SSR – sustained silent reading, building on imagery through drama play reading poems/songs re-reading and retelling author studies genre studies topic studies links with writing programme reading across the curriculum
Close: Discuss language, meanings, and ideas in a range of texts, relating their understanding to personal experiences and other texts
Exploring language Identify, discuss, and use the conventions, structures, and language features of different texts, and discuss how they relate to a topic AsTTle: 'Connections' includes skills from all reading processes AsTTle: 'Surface features' 'Knowledge'	Use a range of strategies automatically when constructing meaning from text. For example, through its structure, organisation, language features & specific language Use table of contents, index, and glossary confidently Read aloud to an audience with clear diction and reflecting personal interpretation Talk about books and authors, and deliver reports and summaries Know the difference between different types of texts – diaries, poems, narratives, directions, advertisements Understand and follow logical sequences in instructions or explanations Discuss the different language features used to engage the reader Show awareness of the variety of sentence type and structure as per 'West Eyreton School Punctuation, Usage and Grammatical Skills' Understand use of colon, semi-colon, dash, bullet points, hyphen, brackets, paragraph Show understanding of the parts of speech as appropriate. See 'West Eyreton School Punctuation, Usage and Grammatical Skills' Show understanding of word knowledge as appropriate. See 'West Eyreton School Punctuation, Usage and Grammatical Skills'
Thinking critically Discuss and convey meanings in written texts, exploring relevant experiences and other points of view asTTle: 'Inference', 'Understanding'	Comment appropriately, using own interpretations, on aspects of fiction – plot, characters, sequence of events. Read beyond the lines, making assumptions based on personal experiences and prior knowledge. Discuss topical items in newspapers and magazines, and relate these to own understanding and experience. Use evidence from texts to support a point of view. Make critical comparisons with other texts read, & compare styles & forms. Offer opinions about material read.
Processing information Gather, select, record, interpret and present coherent, structured information from a variety of sources, using different technologies and explaining the processes used. asTTle: 'Finding information'	Identify and retrieve information from texts Use the school library independently, confidently, and appropriately. Use information-gathering skills eg. Dewey decimal system, computer catalogue, internet, CD-ROM Access and use relevant information from a range of sources. Skim-read for information or when selecting a book Skim-read, make notes, and summarise information Adjust reading strategies for different texts and purposes. Formulate a focus study question and provide a researched answe Make effective use of non-fiction texts eg. Use index, key words, contents page, map key, paragraphs, glossary, headings, diagrams Use a database

Appendix 4

2005 Literacy targets action plan

What needs to be done:		When by?	Who by?	Budget / Cost?
Action plan:
1.	Conducted asTTle reading testing in Week 4 term one.	March 2005	Year 6–8 teachers.
2.	The results of the asTTle testing were compared against the November Running Records. These showed that the year 7 & 8 students were reading at or above their chronological age.	March 2005	principal / teachers
3.	asTTle showed that although they were achieving at or slightly above the New Zealand mean in critical thinking skills that this was well below the expected curriculum level for year 8s (inference & understanding.)	March 2005	principal / teachers
4.	Year 7 students were also achieving below the asTTle expected curriculum level for critical thinking. (inference, understanding)
5.	The year 7 & 8 target group's asTTle results were entered onto a separate database and reports taken to differentiate their specific learning needs in each strand.	April 2005	principal
6.	Staff met to discuss asTTle results – which strands to focus on as understanding, inference and knowledge all need focus.	April 2005	principal / teachers
7.	Resource teacher of Literacy (RT Lit.) conducted testing of target students to determine Reading needs.	March – April 2005	RTLiteracy.
8.	Meeting held with the RT Lit.) to discuss the target group's teaching needs – plan developed = reading mileage needed.	April 2005	Year 6–8 teachers Principal / DP
9.	Meeting to develop action plan using asTTle pathways learning report – use of What Next section; journals; ARBs;	April 2005	Year 6–8 teachers / principal	Staff Mtg – all staff as well
10.	Meeting with NPM reading rep to look at reading resources to purchase to teach the targets – Journeys into Literacy and Bloom's Taxonomy.	April 2005	Year 6–8 teachers / principal
11.	NPM rep checked book rooms at the school to determine purchase / school buying to meet targets.	April 2005	Year 6–8 teachers
12.	Use the asTTle Learning Pathways report to focus on understanding and Inference learning outcomes in class programmes.	T2 Wk 1	Year 6–8 teachers
13.	AsTTle Interim reading inference/ understanding testing end of each term.	Week 8 T2.	Year 6–8 teachers
14.	Formative teaching strategies to be a vital component of teaching all classes.	T2 Wk 1 onwards	All classes	Staff Meetings
15.	Readers' clubs to be set up in all senior classes to develop reading mileage and focus on knowledge area of asTTle.	T2 Wk 1 onwards.	Rooms 3, 4, 5 & 6	National Library books Ashley Resource centre
16.	Reading timetables to be re-visited to ensure that adequate time is available in senior classes for reading instruction.	T2 Wk 1	Principal
17.	Observation of experienced teachers taking instructional reading lessons to be arranged. Room 5 & 6 teachers observed teaching. RT Lit. to observe.	T2 Wk 2, 2 wkly intervals	Principal to release year 6–8 teachers
18.	Attend Stephen Graham 'Comprehension' seminar.	26 May 4–6 pm	Rooms 3, 4, 5 & 6	Relief teacher, if out of school visits necessary.
19.	N.P.M. rep. to attend staff meeting on 31 May to develop senior reading resource room with staff.	31 May 3.15 – 4.30 pm	As above	$60.00 x 4 = $240.00
20.	Conduct asTTle testing at regular periods to gauge effectiveness of action plan.	Towards end of each term.	Year 6–8 teachers
21.	Staff meetings – discuss, feedback progress	T2 Wk 1 on	All teachers	Staff Meetings

TKI global navigation

asTTle: Assessment Tools for Teaching & Learning: About asTTle local navigation

asTTle: Assessment Tools for Teaching & Learning – He Pūnaha Aromatawi mō te Whakaako me te Ako