How's your school?

Kirsten Storry has a terrific oped in the Newcastle Herald (republished in Online Opinion). It concludes with her three-point wishlist regarding test scores:

If the Australian Government or Opposition want to score a few points with information-starved parents in their election campaign, here are my top three.

First, test all students from Grade 2 to Grade 7 at those schools in which a majority of children repeatedly fall short of the national benchmarks. Snapshots of years 3, 5, 7 and (from next year) 9 are not enough in schools in remote Indigenous communities where children can fall far behind in two years and can move frequently. These are the key learning years for fundamental literacy and numeracy skills and indigenous children in remote communities need these skills to have a fighting chance in secondary boarding schools.

Second, replace minimum benchmarks with performance bands in the National Report on Schooling in Australia. The current national benchmarks represent the minimum literacy and numeracy that children need to progress through school. But there is no magical line in the sand between literacy and illiteracy and the benchmarks do not represent the level of literacy or numeracy that children need to prosper.

Third, publish “value-added” school performance data, or at least make the data freely available to education policy researchers and economists who will happily analyse them. In some Cape York schools, the average child is falling nine months behind in reading for every year of school. Without the data, it is too easy to blame low school attendance rates for the entirety of the shortfall.

Kirsten also gives bouquets to the Western Australian government for being the only state or territory government in Australia to publish test scores. She’s right – but they do it in a very odd way. Their website only shows school results in a bar graph form, and a colleague of mine got a very firm ‘no’ when he recently asked them for the numbers underlying the pictures.

Advertisements
This entry was posted in Australian Politics, Economics of Education. Bookmark the permalink.

17 Responses to How's your school?

  1. The NSW situation is strange. Banning the publication of information does not stop pasrents from seeking such information. It just makes it harder to obtain accurate information. It is not like parents don’t obtain information from a variety of non-published sources. All that the NSW law achieves is to reduce the signal value of the information. Parents may well opt for alternative private schools with good reputations simply because they are not allowed to know that the local public school is outstanding. This can cost such parents a lot of money. Perhaps the NSW government is secretly trying to reduce the number of public school students so that they can reduce its education expenditure?!!!

  2. Thanks Andrew.

    On the topic of WA, I thought I could get numbers by looking school by school on their website, but it was a very tedious process and may have changed.

    Even worse from an indigenous policy perspective, WA has a policy of not publishing the literacy, numeracy or attendance numbers for any grade cohort with fewer than 10 children. This, of course, rules out all remote community schools. However, I had already complained about this in an oped in the West Australian and I was pushing the word limit.

  3. Des Griffin says:

    Probably, getting accurate information about performance from summative information like testing makes a difference providing the information is used to help the student. But it was clear years ago from research by Black and Wiliam in the UK that it is formative evaluation – frequent discussion with the student – which makes the most difference to educational attainment. And if you think about it why wouldn’t that be true? Even if all we do is accept that stuff from half a century ago about the lights being turned on and off.

    And it is more than abundantly clear that the greatest difference to ultimate educational performance generally (and a great deal more) is made by high quality early childhood education, targeted especially at lower socio-economic groupings. You have written about this yourself Andrew, as I have reminded you before. (I’m sorry if that sounds rude.) I have recently read up on the major studies about this as well as attending an extremely interesting talk by Federation Fellow at UTS Michael Keane.

    The early childhood issue seems to be unclear to the present Commonwealth government if I heard the recent discussion on 2BL correctly with Minister Hockey proclaiming that unless we got more people (mothers with children not yet working seemed the main target group) into the workforce we will be in great strife. So far as I could tell he did not say anything which recognised what we know about high quality early childhood intervention. He mentioned (I think) that if mothers could not get childcare, well, they didn’t have to go to work.

    Putting more money into projects like art education (studies at the Guggenheim in NY and Gardner Museum in Boston) and music (witness the 240,000 young people from poor backgrounds in the orchestras in Venezuela) wouldn’t be a bad idea either.

  4. Andrew Leigh says:

    Kirsten, I went to the WA “Schools Online” website (down at the moment, but usually available here). I’m pretty sure it only shows graphs, but if I can actually get numbers for each school, I really want to know!

  5. cba says:

    Andrew: in the worst case, you could web crawl it to retrieve the sample sizes and graphs for every school. It wouldn’t be too hard to come up with an algorithm to pick out the data points from the graphs based on pixel height of each blue bar. I’ve had to do similar things in the past…

  6. captain_america says:

    …hey, cba you know what you could do? Write that algorithm, put the scores into a spreadsheet and sell it to the West Australian so they can publish a school league table. I’m sure that they’d pay a tidy sum for it – it’s a surefire circulation spike for them (UK newspapers do it every year).

  7. cba says:

    sadly, my employer deems receiving compensation for anything outside of part-time teaching contracts and occasional ebay sales a conflict of interest. they’d go bananas over me receiving payment from the “foreign press”. sucks to be me 🙂

    anyway, if you can’t find an easier way to get that data andrew, just let me know… (don’t you have an army of RAs to take care of these problems?)

  8. Patrick says:

    er, don’t tell them! Unless you actually work for the WA state.

  9. cba says:

    how about this:

    * I write the algorithm and send the spreadsheet to Andrew and one of you
    * Andrew writes a nice academic paper
    * The West Australian gets their story
    * You get paid (at least some of us are proud capitalists)
    * The WA Dept of Ed and Training is furious
    * Future WA test scores are available only in hard copy and sold on a per school basis

    😀

  10. Patrick says:

    hey, sounds like a deal to me 😉 – although I suspect Andrew’s involvement alone is sufficient 😦

  11. Andrew, I had a look and I must have been thinking of school attendance, for which they do provide the numbers.

    CBA / Captain America, I think you may be too late on your scheme. I was speaking to a WA school principal yesterday and the West Australian apparently published a league table of schools last year, based on the percentage of kids at each school who passed the literacy and numeracy benchmarks. I understand that it was not well received!

  12. insider says:

    This is turning into a very interesting thread..

    Who was it not well received by? The WA labor government? The state Labor governments are pathetic. They say that they are releasing school performance information and then lose their nerve and try and weasel out of it by publishing it in strange formats or at such a coarse level of detail that it is useless (have a look at what Anna Bligh did with the Year 12 tables in Queensland.. she is soooo overrated).

    What is even worse is that the state Labor governments are not making any moves to analyse or redress educational inequality at the inter-school level. If they’re going to introduce a league table system then they should also be collecting good quality socio-demographic information at the school level as well. But they don’t because:

    1). They simply don’t have the skills in their policy bureaucracies to do the job. The bureaucrats have very, very limited skills in understanding or handling even basic statistics. Furthermore, the senior policy positions have been stacked with political appointees who are out of their depth. I mean what credentials does Michael Coutts-Trotter (NSW Education Dept head) have apart from extensive involvement in the labor party and being married to a federal MP? In general, the Premiers depts in each state are also stacked with poorly qualified Labor luvvy political appointees. This lack of effective leadership has completely stunted policy development futher down.

    2). The Labor elites are full of hypocrites. They pretend to be stout defenders of public education but many senior Labor politicians and bureaucrats send their kids to the top private schools in each state. I have nothing against private schooling at all but if you’re going to pretend to be some defender of public education then you should walk the talk, y’know. No wonder they are so lazy when it comes to improving the public system.

    Sorry to fly off into a rage but I see people like Kirsten and Andrew fighting a good, polite fight on a lot of education policy issues and it is just banging on a brick wall.

    These Labor guys will not do anything unless they are forced to by events and severe public criticism. It will be worse if Rudd is elected and they have complete power in the country…these guys have had loads of time to start their “Education Revolution” at the state govt level and their record is very poor.

    Insider

  13. Verdurous says:

    A different perspective.

    Some talking points to consider:

    -At medical school, we learnt that it is foolish, indeed unethical, to order investigations which were unlikely to alter one’s management. Might this sometimes relate to schools? If we have no choice, then do we want to hear how bad our school is doing?

    -In Queensland at least, there is almost no indigenous cultural education for students and there is virtually no encouragement or allowance for bilingual education. Most indigenous languages are either extinct or teetering on the edge of vanishing forever. A healthy culture makes healthy, resilient people. The majority of the syllabus is of no relevance to life in Cape York/Torres Strait for example.

    -If we suggest to young indigenous people that success = going away at age 12 to a boarding school and getting the hell out of their community then a) what positive things does this say about the community from which they come and b) doesn’t this lead to a hollowing out of the demographic of the community (mainly the sick, frail, substance abusers remain, lack of young people.).

    Responses welcome.

  14. Insider, one of the reasons that the published league tables were not well received (and I am sure there are many) was that they were the raw statistics without any value added component. In remote community schools, children can move around a lot and may arrive on a school’s door step at age 8 (Year 3) without any record of having attended school elsewhere. Even good remote schools struggle to get these children to read at grade level to pass the Year 3 benchmarks. Schools in urban middle class suburbs may not face these issues at all or at least not on the same scale.

    That said, the percentage of indigenous kids in WA achieving the literacy and numeracy benchmarks has declined over the last 7 years (assuming that the real percentage falls in the 95% confidence interval) and this is not the case in any other state or territory, even the NT. It does raise questions as to the accuracy of testing, especially in the NT where figures of around 20% of indigenous children in remote communities passing benchmarks seem exaggerated.

    I can see arguments for and against publishing league tables in newspapers. What I am keen to see is that departments make information on school achievement publicly available so that parents can compare schools in their area and exercise school choice.

    I can’t, of course, comment on the politics involved as CIS is non-partisan.

  15. insider says:

    Yep, Kirsten these are all good points that you make – socio-demographic factors underpin the raw scores (and must be controlled for), value-added scores provide a different type of performance signal and there is crucial measurement error in testing….

    ….All these issues should be a key consideration in any system of school performance indicators. My point is that the Labor state education bureaucracies are completely incapable of implementing this type of system. Their policy people don’t have the basic statistical skills to do it and the politicised leadership of the public service can’t get their heads around it. Until these people are directly and heavily attacked the arguments of people like you and Andrew will be ignored.

    Some of these bureaucrats might nod and stroke their chins when you guys give talks and seminars but they’re not going to lift a finger in the name of serious, empirically-validated policy.

  16. conrad says:

    I wouldn’t say all the bureaucrats don’t know enough about the statistics/issues (although its obvious that sometimes they don’t — the recent Steiner school stuff in Victoria comes to my mind as an incidence of ignorance of the issues). This is because these issues are often highly politicized and sometimes they don’t want to find or release the true results. I know some people that previously worked in some of these government agencies, and it wasn’t always lack of expertise or knowledge that was causing things not to get released or poorly designed. For example, part of the job of one of my friends was to get rid of contentious categories in categorical data. This one you can see for yourself, rather than just hear as an anecdote. Next time you read a government report, try imagining what categories you think should be there and what categories are. In case you think categories are missing (e.g., redistributed into the general population) or the other category is too big, you know what is happened. This can happen for groups that are obvious as they are long term problems for governments (e.g., how are blacks doing in the public service?), and groups that are not neccesarily obvious but would create controvesy (e.g., crime statistics and racial groups).

  17. Yobbo says:

    This just in: Governments attempt to cover up their poor performances.

Comments are closed.