Nick Sherry (Tasmania, Australian Labor Party, Minister Assisting the Minister for Tourism) Share this | Link to this | Hansard source

I present three government responses to committee reports as listed on today's Order of Business. In accordance with the usual practice, I seek leave to have the documents incorporated in Hansard.

Leave granted.

The reports read as follows—

AUSTRALIAN GOVERNMENT RESPONSE TO THE SENATE EDUCATION, EMPLOYMENT AND WORKPLACE RELATIONS REFERENCES COMMITTEE

REPORT ON THE ADMINISTRATION AND REPORTING OF NAPLAN TESTING

AUGUST 2011

The National Assessment Program – Literacy and Numeracy (NAPLAN) is part of the Ministerial Council for Education, Early Childhood Development and Youth Affairs (MCEECDYA) National Assessment Program (NAP), an ongoing program of assessments to monitor student performance against key performance measures. The Australian Curriculum, Assessment and Reporting Authority (ACARA) manages the national and technical aspects of NAPLAN test development, assessment and reporting on behalf of MCEECDYA. State and territory Test Administration Authorities are responsible for the administration and delivery of NAPLAN tests in their jurisdiction.

The Committee's recommendations are directed to ACARA and MCEECDYA. As ACARA's work is directed by MCEECDYA, all recommendations need to be considered by MCEECDYA which may then direct changes to ACARA's work plan or Charter as required.

The Australian Government has consulted with ACARA and state and territory governments in developing its response to the report, and their comments have been incorporated in this response.

It should be noted that work is already being progressed in a range of areas related to Recommendations 3, 4, 5, 7, 8, 10, 11 and 12. Since the establishment of the Inquiry a working party of stakeholders has met and recommended to ACARA a number of changes to the My School website. Many of the concerns raised in the submissions to the Inquiry have been addressed in My School 2.0 by broadening the range of information provided and increasing levels of user choice.

The Australian Government response to each recommendation in the report is provided below.

Recommendation 1

The committee majority recommends that ACARA and MCEECDYA explore and report publicly on ways in which to use below-average NAPLAN test results as a trigger for immediate assistance aimed at helping individual schools and students perform at appropriate levels.

Australian Government Response

The Australian Government notes this recommendation.

Decisions on the provision of assistance at school and student level are generally made at state and territory level. Through the National Education Agreement and the National Partnerships, announced by MCEECDYA in 2008, the Australian Government provides funding to states and territories so that government and non-government schools can deliver an education that provides all young people with the skills to participate actively in our society and support students to achieve their potential.

In May 2008 the Council of Australian Governments agreed that students who have not achieved the national minimum standards for literacy and numeracy “need and will receive focussed intervention and support to help them achieve the skills they require to continue in schooling”1.

All state and territory government and non-government education authorities in Australia have developed means to evaluate and assess system and school performance. Student outcomes, including student level NAPLAN results, already inform state and territory school planning and improvement practices. At the school level, NAPLAN provides a wealth of information to support teachers to identify and analyse areas for improvement as well as strengths in student performance.

School level data as reported on the My School website provide valuable information that enables governments to identify and respond to areas of need. State and territory governments have access to these data within the same year that students sit the tests. This information can be used to target funding to where it is most needed.

Support for schools not performing at appropriate levels is also provided through improved teaching methods and planning that do not require additional monetary investment.

————

1 MCEETYA Media Release 12 May 2008

The Australian Government has delivered unprecedented levels of investment in Australian schools, more than doubling the level of funding provided to schools in the last funding period. In total the Australian Government has committed to provide a record $64.9 billion for schools from 2009 to 2012.

This includes additional funding under the three Smarter Schools National Partnerships: the National Partnership for Teacher Quality - $550 million over five years (2008-2009 to 2011-2013); the National Partnership for Low Socio-Economic Status School Communities - $1.5 billion over seven years (2008-2009 to 2014-2015); and the National Partnership for Literacy and Numeracy - $540 million over four years (2008-2009 to 2011-2012). The notional allocation of funding to each state and territory under the National Partnership for Literacy and Numeracy is based on each State's share of students at or below the minimum standard in Reading and Numeracy for Years 3, 5 and 7 based on the 2008 NAPLAN results.

In addition to this funding, the Commonwealth provided $11 million to states and territories to support 110 schools that were identified from My School 1.0 data as having substantially below average student outcomes in literacy and numeracy compared with the national average and similar school average.

Recommendation 2

The committee majority recommends that ACARA assess and report publicly on the potential benefits of moving to a system that reports the median rather than the mean school performance.

Australian Government Response

The Australian Government agrees in part with this recommendation.

The Australian Government will raise the issue with MCEECDYA and recommend that ACARA investigate the feasibility of showing the median in addition to the mean in its reporting in school performance.

Mean and median both provide an idea of where the middle of a set of scores lies. The mean averages all scores of students in a school and as a consequence includes extreme scores that are either very high or very low. On the other hand the median score is the middle score when all scores are placed in numerical order.

All states and territories consider that the mean is the better measure for the statistical analysis of school data as it enables:

Reporting NAPLAN results for 2008-2010 has used the mean as the expression of score averages. The use of the mean in the analysis of NAPLAN data has enabled the above statistical analyses to be used with NAPLAN results from 2008, 2009 and 2010, including comparisons of data between years. Continuing to use the mean means it will be possible for these analyses and comparisons to be made into the future. It is not possible, however, to do the same statistical analysis using the median so comparisons across years will not be possible and a new baseline year for reporting will need to be established.

The My School website provides information on the distribution of student performances within a school and so reveals whether the distribution is skewed in a way that would make the median substantially different from the mean.

States and territories provide data to schools in a number of ways. Some, including Tasmania and Victoria, use the median in reporting school level performance to complement their analysis of NAPLAN data for schools. This additional information is helpful for schools with small student populations.

Recommendation 3

The committee majority recommends that MCEECDYA and relevant jurisdictional test administration authorities look at and report publicly on ways to ensure that children with disabilities are not discriminated against and denied the right to participate in national testing.

Australian Government Response

The Australian Government agrees with this recommendation.

All students are encouraged to participate in NAPLAN. The National Protocols for Test Administration state that students with disabilities should be given the opportunity to participate in testing should their parent/caregiver prefer that they do so. This is consistent with the Disability Standards for Education, which set out the rights of students with disability and the obligations of school authorities in relation to education under the Disability Discrimination Act 1992. The Protocols do allow students with significant intellectual and/or functional disabilities to be exempted from sitting the tests if they are unable to access the tests within the guidelines for accommodations.

In the past, interpretations of provisions in the Protocols varied across jurisdictions. ACARA has revised the Protocols for the 2011 NAPLAN tests to provide clarity about positive expectations for participation in NAPLAN and accommodations to facilitate access to the tests and to ensure consistency of application of the provisions in the Protocols.

My School reports the participation of students in NAPLAN at the school level compared with the national average. In My School 2.0, there is stronger reporting with categories of exempt, absent and withdrawn students reported separately.

ACARA will be publicly reporting levels of participation in testing to make this more transparent at a school level. These data are already published at state, territory and national levels.

There are a number of accommodations already made for students with disabilities that range from separate supervision and rest breaks to use of assistive technology for students who would have them as part of their normal classroom support. ACARA will report to MCEECDYA on the type and number of accommodations.

ACARA is already looking into further ways in which the tests can be made more accessible to students with disabilities.

Recommendation 4

The committee majority recommends that ACARA analyse and report publicly on how NAPLAN tests are serving different groups of Language Background Other Than English (LBOTE) students.

Australian Government Response

The Australian Government agrees with this recommendation.

ACARA currently publishes information annually about the performance of children by LBOTE status in the NAPLAN National Report, and will publish data on the percentage of LBOTE students in each school profile on My School 2.0.

At present all students are encouraged to sit NAPLAN tests, though students from a non English speaking background who have arrived in Australia within a year of the test may be exempted. ACARA also advises schools in the Protocols that literacy

should not be a barrier to the numeracy tests. ACARA is already looking into ways to improve accessibility to the tests for LBOTE and Indigenous students.

The current definition used for LBOTE is very broad, and does not identify the group of LBOTE students who are educationally disadvantaged. As a result, data show that LBOTE students perform as well or better than non-LBOTE students.

In My School 2.0, account has been taken of the presence of students from a language background other than English when establishing comparison groups of schools that serve students from similar socio-educational backgrounds. To do this, it is inappropriate to use a simple measure of the proportion of students from language backgrounds other than English since some students in this category are not socio-educationally disadvantaged. ACARA uses an additional LBOTE measure in calculating the influence of family background on student results, specifically the proportion of LBOTE students at a school whose parents also report low education levels.

ACARA is also looking at enhancing the definition of LBOTE used for NAPLAN data collection to enable reporting of more useful information to support LBOTE students who are educationally disadvantaged.

Recommendation 5

The committee majority recommends that ACARA investigate and report to MCEECDYA on enhancing NAPLAN to support the diagnostic needs of higher and lower student achievers.

Australian Government Response

The Australian Government agrees with this recommendation.

The discrimination of the NAPLAN tests at the higher and lower levels of student achievement is supported, and under direction from MCEECDYA, ACARA is investigating alternative test delivery mechanisms that might facilitate this. ACARA will also seek ways to decrease the time taken to provide feedback to schools.

The Australian Government made a commitment to provide teachers with a diagnostic tool that will enable them to identify and support the individual learning needs of their students at any time. Feedback provided through the online service will link teachers to resources that are targeted to the particular needs of the students.

To make NAPLAN strongly diagnostic at the higher and lower end could require a longer test. There are limits on the length of the tests in regard to what students can reasonably be asked to do, particularly for year 3 students. The provision of tests with a greater diagnostic capacity at the higher and lower ends could be achieved by moving away from pen and paper testing to adaptive online testing.

An element of the Australian Government commitment to online diagnostic tools is the move to online delivery of the annual National Assessment Program sample assessments. This would enable future trials of adaptive testing for NAPLAN.

The Australian Government will ask ACARA to report on the feasibility of enhancing NAPLAN to provide improved diagnostic capability for the students achieving in the highest and lowest bands.

Recommendation 6

The committee majority recommends that ACARA and MCEECDYA expand NAPLAN to include annual testing from years 3 to 10 in order to more accurately track student performance and give parents, teachers and policymakers a far better understanding of how students, teachers and schools, are progressing.

Australian Government Response

The Australian Government does not agree with this recommendation.

Further large-scale cohort testing is not the best option for giving parents and teachers better information. The next step will be to provide teachers with better diagnostic tools to address the needs of individual students. The government has committed to developing a national online assessment and learning bank for students, parents and teachers to provide a sophisticated diagnostic assessment of each student's strengths and learning needs.

National testing is agreed by COAG and forms part of the National Education Agreement. The costs for test development are shared by the Australian Government and state and territory governments (using the MCEECDYA formula for cost sharing) but states and territories carry the cost of test delivery. The cost of test delivery is currently around $48 million for states and territories.

Providing parents with meaningful reports more frequently would require tests that measure student improvement more precisely than the current test program. ACARA has trialled “off level” testing (in which students undertake NAPLAN assessments from a higher or lower level than their year level, eg a year 5 student sitting the year 7 test) and will be asked to trial online testing, which in the future may be able to provide this more precise measure of student improvement.

A decision to expand testing would have significant cost implications both at a national level and for state and territory governments with responsibility for test delivery. The NAPLAN scale was constructed for a testing regime where students in years 3, 5, 7 and 9 are tested. If the test program was altered and students in all years from years 3 to 10 were tested, the scale would need to be reconstructed or a new scale developed that would be sensitive enough to measure student improvement in yearly increments. If the tests were to be held for all year levels the scale would need to be recalibrated to measure the smaller increments of student progress. This would be a significant task requiring additional funding. It would also have a significant impact at a school level, due to the impact on teaching time. Consideration should also be given to any possible negative consequences of increased testing.

NAPLAN is only one form of student assessment, although an important one. Schools use additional assessments to provide a more detailed picture of student performance and parents receive information about their child's school performance from the school every year.

In addition to classroom assessments, each year a sample of students participate in the National Assessment Program sample assessments and, while individual students reports are not provided, schools where classes participate get reports of their performance.

Recommendation 7

The committee majority recommends that MCEECDYA explore ways for state and territory test administration authorities to more strongly enforce security protocols.

Australian Government Response

The Australian Government agrees with this recommendation.

National Protocols for Test Administration have been agreed by all governments so that all students around Australia sit the tests under common conditions. Test Administration Manuals are provided to all schools and teachers supervising the tests.

MCEECDYA had already asked ACARA to review the National Protocols for Test Administration before the 2011 tests to ensure that they are providing clear and

consistent advice to test administrators and principals. This work has been completed.

For the first time, the Protocols include a Code of Conduct which outlines expected behaviour and processes, with a view to strengthening security requirements. Also at the request of MCEECDYA, ACARA will report publicly for the first time in early 2011 on allegations and substantiations of cheating and security breaches.

It is important to note that compliance with security protocols is the responsibility of state and territory testing authorities. The revised Protocols will enable tests to be administered in a more consistent manner.

It is noted that the Test Administration Authorities do not have a mandate or authority to investigate incidents or enforce penalties for breaches of test protocols in all jurisdictions and sectors, and that there are a broad range of legal and industrial frameworks that determine the way in which investigations are made. ACARA is working with government and non-government education authorities in developing a nationally consistent approach to handling test incidents.

Recommendation 8

The committee majority recommends that ACARA prioritises the improvement of the method used to develop like school comparisons and commits to the introduction of a method based on student-level SES data for all schools prior to the reporting of 2011 NAPLAN test results.

Australian Government Response

The Australian Government agrees with this recommendation.

Consistent with this recommendation, MCEECDYA has asked that ACARA collect and use student level data for all schools when reporting on NAPLAN 2011. ACARA is working with schools and school authorities to collect the additional data required to implement this method for 2011.

For 2010, the ICSEA (Index of Community Socio-Educational Advantage) which is used to reflect the level of socio-educational advantage in a school for My School 2.0 uses direct student level data where available and includes the addition of a language background other than English factor. For 2010 it is anticipated that the ICSEA calculation will be based on direct student data (parent education and occupation information) for 76 per cent of schools, representing 92 per cent of students nationally.

The improved ICSEA now being used is the result of ACARA's analysis and advice to ministers. It builds on the existing ICSEA, and uses better data that have since become available. The new approach draws more extensively from data collected directly from parents. The two ICSEA measures are, overall, very close (Their correlation is 0.9). The new ICSEA, however, is an even better predictor of school-level NAPLAN performance. The new ICSEA explains 67% of the variance in school performance on NAPLAN; the initial ICSEA explained 59%.

Recommendation 9

The committee majority recommends that ACARA and MCEECDYA examine and publicly report on ways to mitigate the harm caused by simplistic and often distorted information published in newspaper league tables.

Australian Government Response

The Australian Government notes this recommendation.

The Australian Government does not support the media using data obtained from the My School website to publish simplistic league tables. The Australian Government also does not support the use of legislation to restrict website users from publishing these data.

Simplistic league tables fail to take into consideration the context within which a school operates and thus are likely to provide unfair, misleading and invalid comparisons between school performances.

The My School website provides the public with a means for making valid school comparisons and is the only source of genuinely nationally comparable information for all Australian schools.

The My School website reports performance comparisons of statistically similar schools only.

At the request of MCEECDYA, ACARA has strengthened legal and technical protections of the data published on My School 2.0 and will continue to actively advocate against league tables based on school performance data.

The My School 2.0 website has new logon requirements and terms and conditions to protect the integrity of the data and to help prevent misuse of data.

Ministers have also agreed that ACARA will be supported to closely manage the information it provides to prevent individual students from being identified and to

promote the meaningful use of data by third parties. ACARA will work with the media to explain the information published, advise on how to properly interpret it, and will take steps to counter any inaccurate use of the information including, if necessary, responding publicly with correctly interpreted data.

Recommendation 10

The committee majority recommends that ACARA identify, analyse and report publicly on possible means of strengthening the relationship between NAPLAN tests and the wider curriculum. The committee majority reserves its support for any alignment between the tests and the new national curriculum until the quality of, and community support for, the curriculum become clearer.

Australian Government Response

The Australian Government notes this recommendation.

It is Australian Government policy to review the NAPLAN assessment framework to provide alignment with the new national curriculum. This has been agreed by all states and territories.

The current NAPLAN tests align well with current state and territory curricula taught in schools, and will be reviewed to align with the Australian Curriculum as it is progressively implemented.

The NAPLAN tests are based on curricula that all teachers throughout Australia are required to cover and reflect the essential elements that should be taught at each year level. The NAPLAN test items are currently developed using the National Statements of Learning. The tests in future years will be informed by the Australian Curriculum. At the December 2010 MCEECDYA meeting, Ministers agreed to publish the content for Foundation to Year 10 English, mathematics, science and history, as the nation's first Australian Curriculum.

ACARA will consult with senior education officials and provide recommendations on measures to ensure that future national assessments, including NAPLAN, align with the Australian Curriculum as the successive phases are implemented.

This recommendation follows a discussion in the report on the high stakes nature of the tests and the potential for teachers to teach to the tests. ACARA actively discourages this practice in its communications about NAPLAN and promotes broad-based teaching practices. Once students' familiarity with the test form is assured there is no benefit in repeated test practice. The best way to develop literacy and numeracy skills is through students' experience of a full, rich curriculum.

NAPLAN is a test of literacy and numeracy skills not a content based test. The main purpose of the NAPLAN tests is to identify whether all students have the literacy and numeracy skills and knowledge which provide the critical foundation for other learning and for their productive and rewarding participation in the community. Inadequate attention to the fundamental areas of literacy and numeracy undermines students' ability to participate effectively in other important areas of the curriculum.

Recommendation 11

The committee majority recommends that ACARA and MCEECDYA move to include more contextual information about schools on the My School website, reflecting the complex range of factors that affect schools, and acknowledge to users of the website their awareness of the limitations of comparisons based on raw performance data due to extrinsic factors. The committee majority further recommends that ACARA commit to ensuring this contextual information is available ahead of the reporting of 2011 NAPLAN results.

Australian Government Response

The Australian Government agrees with this recommendation.

The Australian Government agrees there is more to a school than academic results, and My School shows a range of features about a school in terms of its operating environment (eg proportion of Indigenous students, staff numbers), resources (income) and performance (eg Year 12 attainment) in addition to NAPLAN results.

The framework for information published on the My School website was agreed by education ministers, and reflects research and expert advice that the most appropriate indicators to publish about schools are those that provide insight into three aspects of a school:

The indicators published on the original My School, and additional indicators on My School 2.0, have been developed in response to this framework, and published prior to 2011 NAPLAN testing.

On My School, there is already a significant amount of information provided to reflect the makeup of the school for example, the type of school, the year range, student and staff numbers, location, the level of socio-educational advantage of the school student body, the proportion of students with an Indigenous background, and as well as student attendance rates.

Schools are able to draw attention to their particular circumstances through the statement about their school on the school profile page. Here schools have the opportunity to outline their enrolment policies, promote any special programs that they operate or detail their student profile. A link to the school's own website is provided so that users of the My School website can better understand unique characteristics of the school.

For My School 2.0 an indicator has been added to report on the proportion of students with a language background other than English and information is provided for the first time about a school's finances.

In the future ACARA will add the option for principals to comment on their NAPLAN and senior secondary outcomes, information from student, teacher and parent satisfaction survey data, information about student destinations and information on students with disabilities.

The My School website has been designed to avoid comparisons of raw performance data through presentation of this contextual information and through use of the Index of Community Socio-Educational Advantage (ICSEA) to limit school comparisons to schools with similar student intakes. ACARA is committed to enhancing contextual information and working to enhance the understanding of the media and the public of the school data made available through the My School website.

Recommendation 12

The committee majority recommends that ACARA and MCEECDYA comprehensively revise the type of information available on the My School website to shift the focus from raw school performance data to value-added measurement of school performance.

Australian Government Response

The Australian Government agrees in part with this recommendation.

My School 2.0 reports NAPLAN results in a number of ways to allow different aspects of performance to be shown. The NAPLAN mean scale scores and the spread across the bands were depicted on the original My School, and My School 2.0 also shows student gain for students who remained in each school between 2008 and 2010.

For any numerical score that is published, My School 2.0 also shows margins of error to reflect the accuracy of the estimate average and the degree of confidence one can have in this estimate.

In relation to student performance “value added” is a term that is used in a variety of ways that can mean different things. Value added modelling is a relatively complex econometric approach to measuring school's performance. Value added models essentially seek to measure residuals i.e. the difference between a school's observed outcomes and its predicted outcomes.

While value added models are designed in an attempt to ensure that the residual only approximates the contribution of the school to student performance, this is difficult in practice. This is because residuals reflect whatever other influences there are on student outcomes that have not been captured in the value added model.

There is a risk with this approach that what becomes important is how much better or worse the school did compared to the predictions, and the performance of the students themselves can be lost. A school may perform as well as expected but could still have unacceptably low levels of literacy and numeracy.

There would be a num