When more money doesn't always buy better results

Chris Ryan and I have a new paper out today, looking at trends in the literacy and numeracy of Australian schoolchildren. Here’s the abstract (click on the title to read the full paper):

How Has School Productivity Changed in Australia?
Andrew Leigh & Chris Ryan
Using two series of data that ask overlapping questions to successive cohorts, we estimate how the literacy and numeracy skills of young Australian teenagers (aged 13-14) have changed over time. We find a small but statistically significant fall in numeracy over the period 1964-2003, and in both literacy and numeracy over the period 1975-1998. The decline is in the order of one-tenth to one-fifth of a standard deviation. Adjusting this decline for changes in student demographics does not affect this conclusion; if anything, the decline appears to be more acute. Next, we estimate long-run changes in real per-child school expenditure. This estimate varies somewhat according to the treatment of private spending, and the chosen price index, but our preferred estimate suggests that real per-child school expenditure increased by 10 percent over the period 1975-1998, and by 258 percent over the period 1964-2003. This increase in spending funded a substantial reduction in student-teacher ratios. Measuring productivity in terms of literacy and numeracy points per dollar, our results imply that the productivity of Australian schools may have fallen over the past 3-4 decades. Although we cannot account for all the phenomena that might have affected test score results, we identify a number of plausible factors that might have led to a drop in school productivity.

Some media coverage in today’s press: SMH, Age, Oz, Herald Sun, Canberra Times, ABC News, AM.

Advertisements
This entry was posted in Economics of Education. Bookmark the permalink.

16 Responses to When more money doesn't always buy better results

  1. Fred Argy says:

    Andrew, this is a fascinating research result. You canvas various explanantions. One you do not mention is the possibility that our education spending has not been well targeted at the disadvantaged – not enough spent in early years and in later years, not enough special attention given to those who are most vulnerable and at risk.

    I realise other countries have experienced similar trends. Is this true of the Nordi countries, which have tended to target their education spending better?

  2. Andrew Leigh says:

    Fred, I’m not aware of any long-run data on Scandinavian test score trends. The one people keep referring to is the US NAEP, which has basically flatlined since the early-1970s.

  3. conrad says:

    I’ll say the opposite as Fred here based on looking through your report. At least for mathematics ability (Figure 3, panel B), there seems to be a drop-off in the amount of kids excelling at maths, but the left-tail seems much the same. Perhaps we are spending too much time and money trying to help the often quite unhelpable (like people with terrible language disorders in Melbourne, it appears), and not enough on helping those that can really learn a lot. [any hypothetical reasons for why the individual difference graph looks so different to the schools graph, incidentally?]

    The second half of that statement is backed up by the TIMMS, where very few kids are making it to the higher levels of maths in Australia compared to countries in East Asia. I might have previously thought that there were perhaps genetic differences in this, but looking at the 1975 figures suggests to me that our schools are basically hindering more adept students from achieving their potential. This is somewhat different to the East Asia mentality, where people seem far more willing to help kids who are good at things like maths do even better.

  4. Jenny Lane says:

    Andrew can you see any links between the decline in spending on preservice teacher education and the decreased productivity in schools? In WA we have a teacher shortage and we are failing to attract high achieving students to teacher education courses.

  5. Brendan says:

    An excerpt of an interview with Paul Romer in the Mauritius Times:

    http://www.mauritiustimes.com/080208inter.htm
    “When you talk about the need for improving education for sustainable economic growth, what kind of tangible, measurable quality improvements can one consider as being on the right track?

    Many governments try and measure the inputs that go into their education system like the number of teachers on their staff list, the number of students enrolled, but that’s not what you should measure. What should be measured are the outputs. Part of the goal of the government should be not just to spend more resources but also to get more productivity, more learning for the resources that you’ve got. OECD tests, for example, allow one to see how high school students in Europe compare with those in China, India, the USA in (subjects like) Science, Maths, etc. Mauritius should be using such tests to see how well its student population is doing compared to other countries’. You should have tests to measure what fraction of the population can read your newspapers. You are a successful enough country where the ambition should be such that every citizen should be literate at the level where they can read the newspapers. You should measure to ascertain whether you are achieving that objective. If not, you should figure out how to change the educational system to reach that goal.
    A country like China has done a good job with literacy all across the population. India has not done as well as China on basic literacy, and it would be interesting to find out whether Mauritius is closer to India or China or even behind India, and then do the needful.”

  6. Terje (say tay-a) says:

    It stands to reason that running smaller classes will in and of itself require higher wages* or lower standards. If class sizes have fallen and wage have remained constant in real terms then standards will need to have fallen even further.

    * Smaller classes means more teachers. More teachers equals greater demand. Greater demand suggests higher prices (all else being equal).

    p.s. I’m reminded of a past argument with John Quiggen about this issue:-

    http://johnquiggin.com/index.php/archives/2006/02/21/ad-hominem-ad-nauseam/#comment-44642

  7. Fred Argy says:

    Andrew, I have been glancing at a book by James R. Flynn on “What is intelligence?”(CUP 2007). Drawing on data from 30 nations, he concludes that:

    (i) IQ’s have been increasing steadily over the last six decades.
    (ii) The gains are not however in arithmetic or vocabulary but in predominantly in “tests that require assembling puzzles, matching symbols, reordering pictures and identifying similarities”.
    (iii) IQ’s are greatly influenced by environmental factors – especially parental background and environments that make cognitive demands.

    Point (ii) might be relevant to your findings. Point (iii) might perhaps throw doubt on Conrad’s argument.

  8. conrad says:

    Fred, there’s great evidence that East Asians show a bigger difference between visual than verbal IQ than Europeans, although the overall means are mess because they constantly go up (and, in the last few years down), and they can change massively within short amounts of time (the Dutch being the obvious example). Whether they happen to be genetic or environmental doesn’t make much difference if you are wondering about scores on mathematics tasks of 14 year olds.

    Based on the graph that appears to show Western countries have gone down on mathematics (and I assume East Asian countries have stayed rather similar, although I don’t have the data — it may be retrievable in HK since people there love to document things), I have thought of one environmental reason, which you can blame the government for. This being that all those tables and other little procedures (e.g., long division) we used to all memorize and do in primary school (unless you are younger than me 🙂 have been largely removed (replaced by calculators). My bet is that all those little memory games (still loved in Asia) help kids learn concepts like less-than, greater than, and so on, all of which help develop visual IQ. Removing them from the primary school curriculum may have had a long term effect on mathematical skills (and I doubt having teachers with extremely poor numeracy helps either). It should be easy to test as well. You could pull out groups of kids who have gone to old-school primary schools that still do this type of stuff and see if they do better at maths in high school than the other kids. I’ll certainly bet they do.

  9. MikeM says:

    Terje,

    According to Economics 101 greater demand for teachers necessarily implies higher salaries, but that may not be the case in this instance. Supply of teachers is driven by the number of training places that are funded. Demand for tertiary education usually exceeds supply, so when the number of teacher training places increases, the HSC or equivalent mark used to select trainees drops until the places are filled.

    Result: no effect of salary, but lower calibre teachers (assuming – big assumption – that the selection exam mark does reflect calibre: it may well do, not because the exam measures a person’s potential professional value but because low entry mark signals low status, unattractive profession).

    As teachers’ salaries become increasingly uncompetitive, the rate increases at which qualified teachers who are capable of doing something else drop out of the profession. Demand for newly trained ones increases disproportionately to decreasing class size, leading to a further drop in calibre.

    The process may be eventually self-correcting, as students realise what a poor career teaching has become and the pool of potential trainees shrinks to below demand.

    In this event Terje, in the long term you may well be right, but the long term, while possibly before we are dead, must be many decades. After all, the Leigh-Ryan paper covers a period of close to 40 years and the situation is still to reach a crisis point where even the teachers’ unions realise that there needs to be drastic change.

  10. Patrick says:

    To someone who thinks already that the OECD data, not to mention human history, shows fairly clearly that there is much more to it than money, ie choice, this is just more grist for the mill.

    Vouchers now!

  11. Chris Curtis says:

    Andrew,

    I have read your and Chris Ryan’s paper. It provides food for thought, but the whole issue is very complex. As someone who gave up on Economics in 1974, I am unable to understand the more technical aspects of it, but I wonder if 2008 data would confirm or contradict your findings. Students in year 9 have been at school for over nine years and have been influenced by all the teachers, education policies, etc of that period. The increase in total spending per student doesn’t seem to appear in specific expenditures. Teacher salaries have dropped dramatically relative to average earnings and the increase in teacher numbers does not seem to correspond with the percentage increase in expenditure. To put it another way, if education expenditure per student as a percentage of GDP does not remain constant, the falling relative salaries of teachers will push more able people out of the profession and lead to a decline in the quality of education; i.e., you have to spend more in real terms just to stand still. If we returned to classes of 40, 50 and 60, even more able teachers would leave the profession. If we increased teaching loads, the same result would occur.

    Let me add a historical perspective. I started teaching in a disadvantaged school in West Heidelberg in 1974. Classes were under 25 almost always. Teaching loads averaged 15 hours 30 minutes when I was the timetabler. Teachers at the top of the scale were paid $100,000 relative to today. The school was well enough staffed to have 13 per cent of its teachers devoted to remedial English and maths classes (literacy and numeracy today). Considerable funding came in from the Commonwealth Disadvantaged Schools Program. The teachers ran the school in accordance with their professional judgement, with principals who understood that they were each the first among equals, not the “line manager”. The school was part of a system, with centralised appointments but curriculum freedom, not a small business left to sink or swim as the ideologues of today want. Test scores showed noticeable improvement in student learning. Perhaps we could go back in time to solve our problems.

  12. Andrew Leigh says:

    Fred, we have a discussion of Flynn effects on pp15-17. Your point ii is one that we highlight (though we cited earlier papers rather than the book, so should perhaps revise our citations!).

  13. Dale Bailey says:

    Andrew, I just hope you’ve received your invitation to the PM’s 2020 ideas forum! And your co-authors from “Imagining”.

  14. Ben says:

    Andrew,
    Your comparison of numeracy and literacy results across time is very interesting.

    Given them, though, I was confused by you defining productivity as average test score divided by dollars spent. Productivity is a measure of physical output divided by physical inputs. A measure of labour productivity might be test scores divided by number of teachers, but not test scores divided by the teachers wages (real or otherwise). Taking something that should be called the inverse of the ‘cost of teaching’ and calling it the ‘productivity of teaching’ confuses the issue.

    Interpreting this properly, your study has simply shown that the cost of output in a non-traded sector with modest productivity growth has increased more rapidly than that in a traded sector with fast productivity growth, an example of Baumol’s cost disease.

  15. Pingback: Skills vs. Education at John Barrdear

  16. Pingback: Club Troppo » Missing Link: Last Bumper Edition!

Comments are closed.