TKI - asTTle: Assessment Tools for Teaching & Learning

e-asTTle is a web-based version of asTTle. asTTle was first developed by Auckland UniServices in 2000 as a CD-ROM package that enables teachers to create and analyse literacy and numeracy tests for years 510 and curriculum levels 26. Since then, improved versions have been developed and launched (we now have version 4) each more user-friendly and comprehensive than its predecessor and each version taking teachers' and principals' suggestions for improvements into account. e-asTTle is part of this ongoing improvement and development.

2. Who is e-asTTle for?

e-asTTle is primarily for teachers of students in years 510 and curriculum levels 26. Many schools using asTTle have also found it to be a great tool for planning purposes, for helping students to understand their progress, and for involving parents in discussions about how well their children are doing.

3. What parts of the curriculum does e-asTTle assess?

e-asTTle will continue to assess reading, writing, maths, pangarau, panui and tuhituhi for years 510 and curriculum levels 26. Another area will be added for assessment possibly some aspects of key competencies.

4. Why is e-asTTle ground-breaking?

e-asTTle allows teachers and school leaders to set tests that are aligned to the curriculum when they want and at the level they want; and to measure student progress over time.
e-asTTle uses a programming method (linear programming heuristics) to create the 'best possible' test while also meeting the requirements specified by the teacher.
e-asTTle has developed a new approach to developing and classifying test items so that every test is tailored to the specific needs of the classroom.
e-asTTle has made advances in standard-setting methodologies. It gives teachers a realistic picture of how well each student, class or school is doing compared to the national average and the curriculum requirements (including curriculum levels). It allows comparisons to other groups such as gender, ethnicity, English as a second language, or 'schools like mine'.
e-asTTle provides rich interpretations and specific feedback that relate to student performance (rather than simply providing a score). It identifies areas of student weakness and strength that may otherwise go unnoticed.
e-asTTle presents the results in visual ways making it easier for teachers to discuss performance and the steps they're taking with students, parents, and boards of trustees.
e-asTTle supports teachers by giving them direction and access to extensive and relevant Internet resources (What Next on the Te Kete Ipurangi website) for raising student achievement more efficiently.
e-asTTle is the first bilingual (English and Te Reo) assessment tool to be developed in New Zealand.

Features of e-asTTle

1. Apart from being web-based how will e-asTTle differ to asTTle?

Some of the features being scoped include enabling:

students to complete tests online
students to be assessed in a new test area
teachers to create shorter (10 minute) or longer (60 minute) tests depending on its purpose
teachers to choose whether e-asTTle selects the difficulty of the assessment based on information collected as the student completes the assessment (this is also referred to as computer-adaptive testing)
teachers to choose whether closed-ended questions are marked automatically
teachers to mark open-ended questions online with access to correct examples
teachers and students to access a wider range of reports and analysis
schools to have a seamless integration between e-asTTle and their student management system.

2. How did you decide which features to include?

We asked teachers and schools for feedback about asTTle and the improvements they'd like. The new features are all based on their suggestions. We have met with schools to gain face-to-face feedback and encouraged teachers to provide feedback when using our asTTle website.

3. Is it true that the computer will mark the tests? If so, what about subjects or questions without 'yes and no' answers.

One option that is being considered is to allow teachers to choose the proportion of closed and open questions. The more closed questions the teacher selects the more marking that the computer is able to do. Responses to the open-ended questions will be presented in a way that makes them simpler to mark.

4. Will e-asTTle be available in Te Reo as well as English?

Yes.

5. Some claim that asTTle is hard and time consuming to use. How will e-asTTle be better?

e-asTTle will be easier and less time consuming to use:

As it's web-based, schools won't have load a new programme onto their computer systems
Students will be able to complete most tests online so teachers will have to less data entry
Teachers may spend less time marking if they choose to have closed questions (eg, yes/no, multiple choice, matching questions) marked automatically.

6. I've heard that e-asTTle is more than an assessment tool. Is that true – what else can it do?

e-asTTle provides teachers and principals with information that can be used to inform learning programmes and to apply teaching practice that maximises individual student learning.

How to prepare for e-asTTle

1. What steps should we take to prepare for e-asTTle?

Install asTTle V4 and get used to using that assessment tool. Set up processes within your school such as booking computers to set up and run assessments. Update your student management system to an accredited system.

2. When will e-asTTle be introduced?

It's expected that e-asTTle will be progressively introduced over the next three to four years. The project will proceed in three phases with pilot testing completed during the first phase. The three phases are expected to be as follows:

Reading assessment available on e-asTTle.
Maths, pangarau, panui assessments added and available on e-asTTle.
Writing, tuhituhi, new test area (either science or key competencies) available on e-asTTle.

3. How can our school take part in the pilot?

Twenty-seven schools are currently involved in the reading pilot. Information on further pilots, and how to take part, will be available on this site and in the Education Gazette.

4. What technology do we need to run e-asTTle?

You will need a computer with an internet connection capable of running an up-to-date browser (Internet Explorer 6, Firefox, Safari). You will need a screen capable of running a resolution of 1024x768 and you will need an up-to-date version of Macromedia Flash player installed. These are all things that are recommended for school computers and are not exclusive to e-asTTle.

5. Will we have to get broadband?

Having broadband will make using e-asTTle easier. As all New Zealand schools have the capability to connect to broadband, it is strongly advisable to do so connecting to broadband ensures your students have access to the growing opportunities that the Internet has to offer.

6. How much will it cost schools to use e-asTTle?

Access to e-asTTle is free. So it will cost no more than the cost of broadband or other internet connections that your school already has.

7. Where can we find out about best practice use of the internet to safeguard the privacy of our students?

You can get information about using the internet safely from NetSafe a programme offered by New Zealand's Internet Safety Group (ISG). This independent non-profit organisation is focused on 'cybersafety' education for all New Zealanders. It includes a section specifically for schools. Find out more at www.netsafe.org.nz or call 0508 638 723.

8. How is professional development going to be provided with e-asTTle?

We will provide online tutorials as we have with asTTle V4. While feedback on these tutorials was positive, we will continue to make further improvements so that they're even easier to use.

Using e-asTTle

1. What assessments does e-asTTle replace?

None. e-asTTle supplements other assessments such as PAT tests and STAR.

2. How does e-asTTle cater for children with visual or motor impairments who are unable to use computers?

A teacher of a visually impaired student could use asTTle in the same way that they use any test with that student (i.e. oral responses). Students with motor impairments, who may require adaptive technology (i.e. some type of switch) to use a computer, will use asTTle just like other students would.

3. How do you recommend we use e-asTTle with parents?

The reports that asTTle provides lets parents see clearly what their child can do and where there are gaps in their learning. Some schools are using information from asTTle V4 now with students and parents in three-way interviews.

Check out the case studies at www.tki.org.nz/r/asttle/about/case_e.php on the asTTle website to find out more about how schools are using asTTle.

4. How will e-asTTle keep pace with the changing curriculum?

As e-asTTle will be web-based we will be able to update it at any time, so that the data and information is always current.

5. How do you know it will work on technology in the future when technology is changing so fast?

e-asTTle will be run in a single hosted environment so if there are unforeseen technology changes in the future there will only be one installation of asTTle to alter or upgrade without any disruption to the school users.

Having said this there is a lot of thought going into e-asTTle to cover the known technical directions for the near future.

Troubleshooting

Screen Behaviour

1. PC - sometimes when I tab through an asTTle page, the tab disappears, or goes up to the address bar.

If you keep hitting TAB then you should eventually reach the field you are after. If you accidentally TAB passed the field, you can use SHIFT-TAB to return to it. Otherwise, you can click in the field using your mouse.

2. MAC - I can't TAB through the screen. This definitely used to work when I was marking a Test in vn4.

In general, Mac's do not handle tabbing through a website on a browser very well, whether the site is e-asTTle, yahoo.com or any other site. If you are a MAC user, you should use your mouse to click on fields and buttons.

3. Mac Panther - the homepage looks a bit funny Sometimes the MAC Panther doesn't interpret the style-sheet correctly, but all the links off the page should still behave normally.

To reset the style-sheet there are a couple of things you can do:

Refresh the screen
Clear the cache
Close the browser and log back in

Logging In

1. I'm getting the message "Invalid username/password." when I try to log into asTTle.

Make sure you do not have Caps Lock on. If you are certain you are entering the correct username and password, then you will need to approach your schools asTTle administrator. He/she will need to use the appropriate channels to determine if the user account is set up correctly.

2. A student cannot log into asTTle. She is getting the message "User doesn't exist in e-asTTle database".

A number of things could have happened:

The student has not been added to asTTle (your asTTle Administrator will need to follow the appropriate procedures to determine if the student is in user account system and may have to add the student manually.)
The student is in asTTle, but the NSN ID is incorrect. Your asTTle Administrator will need to follow the appropriate procedures

Sitting Tests

1. I've created an Onscreen test but I want to administer it as a paper test - can I do this?

Yes - but only if the test is still in a Pending state. You will need to go back into the e-asTTle application, select Edit Test Settings and change the delivery method. Please note that it is unlikely you will get the same set of questions.

A test MUST be administered using the delivery method specified during the test creation process.

2. How long before the assignment's Due Date can a student begin to sit a test?

A student can start sitting a test right up until the Due Date/time. So, if the Due Date/time is 01/04/2008 14:00, the student could start the test at 01/04/2008 13:59. The student will then have the total test time available to sit the test, i.e. he/she wont have to stop 1 minute later, at 14:00.

Viewing Tests

1. MAC G4 Notebook - I can't view the reading tests I've created, or any reports

Check if you are able to view pdfs from any other websites. If you can't, then check that your browser pop-up blocker is not enabled - you may need help from your asTTle administrator to do this.

Creating Tests

1. Why is it that when I make a new Comprehensive test it always looks exactly the same?

There is only 1 Year 7 and 1 Year 9 Comprehensive test. So, if you create more than 1 comprehensive test for the same Year level, they will all look the same.

2. Why can't I pick more curriculum strands for the test I am creating?

The number of strands that can be selected is directly related to duration for the test. To be precise:

if the test duration is 12 - 23 mins, the user can select up to 1 strand
if the test duration is 24 - 35 mins, the user can select up to 2 strands
if the test duration is 36 - 47 mins, the user can select up to 3 strands
if the test duration is 48 - 59 mins, the user can select up to 4 strands
if the test duration is 60 mins, the user can select up to 5 strands

3. Why do I need to set the number of Closed Questions when creating an onscreen test?

By default the proportion of closed questions is set to "Some". If you are happy with this, then you need not adjust the slider. The option is provided for onscreen tests only as a way of minimizing the amount of marking time needed. The more closed questions, the more questions the computer can mark automatically.

4. Why is the number of closed questions always "Some" when I create a Paper test?

Paper tests follow the traditional asTTle test make up which includes a proportion of both open and closed questions.

5. When I add up the number of questions in the test summary list, there seem to be more than are in the test. Why is this?

All items were classified by teachers according to the New Zealand Curriculum. Some items measure more than one objective or curriculum strand, so they are counted for each strand they are connected to. For example, if one item was related to two different strands, it would be included in the count for both of these strands. This gives a total higher than the number of items in the test, but each item counts only once for the total score.

6. Why do the difficulty totals on the Summary page not match the number of questions?

In order to improve the accuracy of measurement for pupils with extreme scores, asTTle may include very easy items (Below 2B) or very hard items (Above 6A) in a test designed to assess curriculum level 2 or 6 respectively. These items are not counted in the difficulty total and so the tally may sum up to less than the number of items in the test.

7. Why do I seem to get the same test no matter how often I go through the test selection?

To avoid getting the same test questions you need to:

Accept the test and the asTTle application will know that those questions have been used. Otherwise you will get the same questions until they have been used.
Change your requirements. If you ask for exactly the same thing, even if you have accepted previous tests, you may get similar items simply because there are not enough reading passages or questions that ask what you want.

Check carefully before assuming the test is the same. The reading passage may be the same but the questions themselves may be different.

8. How long is an e-asTTle test?

Total length of an asTTle test can be determined by the Total Test Time. For an onscreen, time to answer attitude questions and review time is included. It does not include time to answer practice questions. For paper tests, only the test duration is included and therefore, teachers will need to allow time for students to answer attitude and practice questions.

Assigning Tests

1. I've selected the school that I want to assign a test to, but the students in the school are not displayed.

When assigning a test, you must select the School and then a Group within the school. Once you select the Group, the students within that Group will be listed and you can select one, many or all of the students to assign the test to.

2. A student I want to assign a test to is not listed.

You need to make sure that the student is in the Group you have selected. You can check what students are included in the Groups you have made by going to the Manage Student Left Menu.

3. How does the Date Available work when you are assigning a test?

If you want to make the assignment available to the students straight away, you can click the "Now" tickbox next to the Date Available. If you want to set the Date Available to be in the future, you can change it in the date and time fields, but make sure the "Now" tickbox is not ticked.

4. I keep getting a message telling me that the Date Available I have specified is in the past, but I haven't changed the Date Available and it looks like it is a few minutes in the future.

The system's current date and time default into this field when you first enter the screen, but as it takes some time to pick the students you want to assign the test to, the time is likely to lapse. Also, it is the asTTle system's date that is being checked and this may not exactly match the date/time on your computer. If you want the test to be available immediately, click the "Now" tickbox next to the Date Available.

5. Can I delete an assignment?

Yes - you can delete an assignment if its Date Available hasn't passed. If its Date Available has passed, then the assignment is available to students and can no longer be deleted.

Marking Tests

1. What exactly do I enter when I am marking an assignment the responses of the students, or do I need to score them first?

Tests delivered on Paper
All the multi-choice questions require a response entered in the Response column. e-asTTle marks the multiple choice questions for you; all you need to do is select the response (a letter) from the dropdown box corresponding to the bubble the student chose. All the open-ended questions require a mark entered in the Mark column. You will need to mark the open-ended questions, using the Marking Guide. Select Help for further information.

Tests delivered Onscreen

Closed Questions
All the closed questions (where students select a response) will automatically be marked by the system. The teacher cannot change the marks or responses entered by the student when he/she sat the test online.

Open Questions

Open questions (where students have to construct a response) need to be marked by the teacher.
Select "Help" for further information.

2. When entering responses or marks, how do I know what are the valid characters that can be entered?

In e-asTTle you can only enter responses and marks from the list of possibilities in the dropdown box which is tailored to each question. There is also a column which lists all the Valid Entries that you can enter.

3. How do I score the attitude questions?

The most unhappy face is first in order, so it gets marked as 1. The most smiley face is fourth, so it gets marked as 4. This means that 1 is a very negative attitude and 4 is a very positive attitude.

4. What do I enter if a student selects two or more responses for multiple-choice questions?

Multiple-choice questions require students to select only one answer. A student who selects more than one response on his/her paper test does not know the correct answer. Enter this response as a dash (-). The asTTle software will treat the question as wrong.

5. How can you record data for the same test given twice to the same students?

If you want to know how students are going in the same learning areas, create a new test in asTTle with the same selection criteria (level and strand) and administer it to them. For all tests created in e-asTTle, these details can be found on the test summary page. Once you have entered their data you can compare the old and new reports and use the Progress report to judge their growth over the timespan.

6. Is it a good idea to re-use the same test with a class, at the start and the end of the year, say, if I want to compare the before and after performance of my class?

This is not necessary within asTTle as all the items are on the same scale. You should create a similar test to use as your post-test (appropriately adjusted Level based on the expected progression. The danger with using the same test is that the students remember the questions.

7. Somebody else has made a test with the same name as my one. How can I tell which one is mine?

Users are allowed to make tests with the same name, however, you can usually see the name of the Owner of the test in any table which lists the test, and so can identify your one. You could also edit your test name in the View Existing Test page, to make it more identifiable.

8. Why can't I mark my assignment?

There are a number of reasons why an assignment may be unavailable for marking:

Paper Tests:

The Date Available hasn't passed yet.

Onscreen Tests:

All closed questions (there is no need to mark this test as the system will mark all the questions for you).
The assignment's Date Due hasn't passed (onscreen tests cannot be marked until after the Date Due has passed, to ensure that there is no one sitting the test at the time)
No students sat the test onscreen (if no students sat the test, then you cannot enter in any marks).

9. If there are a lot of assignments for the same test, how can I tell which assignment is the one that I created?

When you create an assignment, you should print out the Assignments Report (from the Assignment page) and note the Date Available/Date Due and Group that was assigned. This will assist you with finding your particular assignment. If you haven't printed an Assignment Report, then go into the assignment you think is yours and check that the list of students included is as expected.

Interpreting Reports

1. How do I know when there is a meaningful difference between two scores?

The standard error of measurement is about 15asTTle scale points in each subject. Any point difference of more than 15 points between two students, two classes, or two performances over time for the same student, is statistically significant. Teachers should interpret this as reflecting a change in that individuals or groups performance. Changes of less than 15 points are not so meaningful and not so much dependence should be placed on such close-to-random differences.

2. I want to know the range of scores for individual students, and can't see how to get this from the Learning Pathways reports.

You can get this information from the Tabular reports.

3. Where do I get the individual attitude scores?

The overall attitude score is shown on the Tabular report. This is the mean from the six attitude questions. Remember that 1 indicates low attitude whereas 4 indicates high attitude. If you want to see more information you will have to go into the marking screen and view what responses where entered or look at the students' individual test forms.

4. What does the red circle on the Individual Learning Pathways report mean?

The circle represents one standard error of measurement around the students score. This is like the margin of error reported in political polls two out of three times the students true score will lie somewhere between the top and the bottom of the red circle.

5. How do I get a Console Report on a subset of students in my class?

You will have to create a special group for those students. Within asTTle you can go to Manage Student from the Left Menu and create a new group (class), including those students of interest in the group. Students can be in multiple groups in e-asTTle.

6. Why do the Learning Pathways Reports repeat items and achievement objectives in different quadrants?

Items can measure more than one objective, and objectives are measured by a number of items of varying difficulty so they can appear in different quadrants. This is determined by comparing the difficulty of the items with the ability of the student, and whether the questions were answered correctly or not.

7. If two students get the same number of questions correct, why are the scores different?

It is possible to get different scores. While getting the same number of questions right gives the same total raw score, a student's asTTle score is determined by the relative difficulty of the questions they get correct. So, a student who does poorly on a hard question may get a higher asTTle score than one who does well on an easy question. A student who gets the harder questions right on the same test as another student will get a different overall score: in this way, different scores will be generated for each student.

8. How come my high scoring students have no strengths, while the low scoring students have lots of strengths listed in their ILPs?

The position of objectives into the four quadrants is based on the relative position of the items difficulty to the students individual overall ability and whether they were answered correctly or not. Thus, it is not possible to have strengths if all the items answered correctly were easier than ones overall ability. To find a students strengths the test must have items in it harder than the students overall ability you should give such a student a harder test next time. On the other hand, a weak student who gets some hard items correct, will have strengths. Remember, the ILP report is a diagnostic analysis of an individual it is not intended for comparisons between pupils. You should use the Console Report for that purpose.

9. Are the data in the Tabular Report sortable?

Yes. The Tabular Report is now created as a .csv file, which can be opened with a spreadsheet application, e.g. MS Excel, and the spreadsheet applications standard sorting capability can be used.

10. What does the aRs scales mean?

aRs = asTTle Reading Scale
This is the Reading specific asTTle scale scores, which measure students' performance. Scores range between 100-3000 points, with the national mean of Year 57 students set at 1500. A weighted score for each item, which takes into account the varying degrees of difficulty of items, is converted to the asTTle subject scale. This allows for more dependable variations between students' scores, and comparison of a student's past and present performance.

11. How can all students in a year level within a school be compared to the national mean?

Because all items are calibrated to a common scale, it is possible to compare performances in a subject regardless of which tests have been completed. The overall asTTle scale score for each subject compares any student to the national sample of year 412 students on which the materials were standardised. So all you need to do is collect students' asTTle scale scores and compare them.

12. What do the boxes on the Console Report mean?

The boxes are commonly known as Box and Whisker plots. They enable you to get a richer picture of how your students are doing compared to the national group by taking into account the way in which scores are spread, and not just the middle.

The bottom of the box represents the 25th percentile, and the top of the box is the 75th percentile, which means that half of students are within the box. The line in the middle of the box is the median (50% of your students are below this point, 50% are above). The ends of the whiskers represent the maximum and minimum score.

Remember, the blue box shows the NZ means based on the sample tested by asTTle it is possible for your students to score higher or lower than the students in asTTles sample.

TKI global navigation

asTTle: Assessment Tools for Teaching & Learning local navigation

asTTle: Assessment Tools for Teaching & Learning – He Pūnaha Aromatawi mō te Whakaako me te Ako

e-asTTle FAQ

The basics

Features of e-asTTle

How to prepare for e-asTTle

Using e-asTTle

Troubleshooting

Screen Behaviour

Logging In

Sitting Tests

Viewing Tests

Creating Tests

Assigning Tests

Marking Tests

Interpreting Reports

The basics

Features of e-asTTle

How to prepare for e-asTTle

Using e-asTTle

Troubleshooting

Screen Behaviour

Logging In

Sitting Tests

Viewing Tests

Creating Tests

Assigning Tests

Marking Tests

Interpreting Reports