BCA Goes to School

The BCA has a report out today on education, which seems to have received blanket coverage in the AFR. I think it’s splendid to have more people throwing ideas into the policy mix, and the report is appropriately cautious/modest about what it’s contributing to the policy mix.

My favourite parts of the report are where they focus on the equity issues – pointing out correctly that if you leave school with low literacy levels, your life chances are pretty grim. Quality schools are one of the best social policies we’ve yet developed. And the BCA is right to suggest that early childhood intervention, and paying talented teachers more to work in disadvantaged areas, are potentially promising idea to experiment with.

The only bits of the report I find frustrating are where the BCA looks at education as an input into the production process. It’s true that their members would currently like to be able to hire more people with VET training. But that doesn’t mean that getting a VET qualification is a good career move. (Indeed, my own work suggests that the per-year economic payoff to TAFE qualifications is considerably lower than the payoff to attending school or university.) The report also cites some rather odd static modelling.

Research by the Centre for the Economics of Education and Training suggests that: in the decade 2006–16, the VET sector will be required to supply 2.47 million qualified people; 70 per cent of these will be required at trade and post-trade levels (Certificate III, Diploma, Advanced Diploma); based on current supply, new entrants and skilled migration, there will be a projected shortfall of 240,000 people with VET qualifications; and as a result, one in seven jobs requiring VET qualifications either will be unfilled or filled with an inappropriately skilled person.

This would be an excellent and insightful analysis… if wages were fixed.

OK, I’m quibbling. The report shows that our peak business body is thinking seriously about school productivity (unlike many education reports, there’s more discussion of outputs than inputs), and concerned about targeting resources to education as a means of helping Indigenous and low-income children. That’s definitely something for which they should be commended.

About these ads
This entry was posted in Economics of Education. Bookmark the permalink.

28 Responses to BCA Goes to School

  1. Andrew, the assumption that wages are fixed just puts them on a par with most of the nonsensical discussions about skill shortages. If people want to claim that there are skill shortages, they need to explain what it is that is stopping wages in the relevant markets from rising to clear those markets. I suspect that most of the businesses who complain about skill shortages are really complaining about the inability to get suitably skilled people at a wage below the market clearing wage.

  2. conrad says:

    Well, I’ll say it again — as far as I’m aware, there isn’t any reliable evidence that any early intervention programs for literacy help if you have a good curriculum (which should be easy for everyone except sociologists) a reasonable teacher, but a reading disorder.

    In addition, since they claim it in a paragraph, I’d love someone to tell me what results they are really talking about when they say “recent research in neuroscience..”, apart from those based on hyperbole.

  3. Andrew Leigh says:

    Conrad, the reliable evidence on early intervention programs is extremely limited. I think we know that high-impact programs work on poor kids. I doubt we’re yet at the stage of looking at differential effects across kids in schools with good/bad curricula.

  4. Des Griffin says:

    I was somewhat surprised to see the report because I don’t have a lot of faith in the blasts from the BCA. The main findings and the recommendations are unexceptional.

    But I was also surprised because the publicity for the report spoke of the need to introduce performance pay and individual contracts in order to reverse the tendency of students to leave school early.

    The actual report refers to performance pay only as something the Commonwealth Government is increasingly interested in and individual contracts are not mentioned at all. This publicity is very unfortunate because quite a few people will not bother to pay attention to it, especially if they have heard Della’s (NSW Education Minister) spray on it yesterday.

    Andrew, in your response to Conrad you say the “reliable evidence for early intervention programs is extremely limited”. I found this odd having in mind your recent article (On Line Opinion July 26) on intervention in early childhood education.

    I wonder if you are aware of the studies by a mob in Toronto including a fellow with the strange name of Fraser Mustard and a recent book entitled, “Enriching Children, Enriching the Nation: Public Investment in High-Quality Prekindergarten by Robert G. Lynch (got to this link: http://www.epi.org/content.cfm/book_enriching#exec ) In the latter half of the intor which is printed at this link are some very interesting statistics about the economic and other benefits of intervention.

  5. Andrew Leigh says:

    Des, I didn’t mean to say that we have no evidence. There are a handful of very good studies, and I believe their result (in particular as it applies to black kids in the US, and probably as it applies to disadvantaged kids in Australia). My point was merely that we don’t yet have enough studies that people have begun talking about what econometricians call ‘heterogeneous treatment effects’.

  6. conrad says:

    DG: My first comment is only in relation to literacy programs — I haven’t gone through the literature on other types of programs (although I know that results from some of the dyscalculia projects are also pretty poor).

    As for my second comment, I say hyperoble and neuroscience together as I would claim that there is not a single neuroscience paper using PET, fMRI or EEG (the three most common methods for looking at normal processing — not to be confused with cognitive neuropsychology) that has told us anything useful that we didn’t already know via simpler methods about how normal people read and learn to read. I might go further and claim something similar about language processing in general, although then I’d have to specify what useful means, and there are a small number of results that would be debetable as to their usefulness.

  7. Andrew Leigh says:

    Conrad, my guess is that the neuroscience reference comes from someone having recently read this paper.

  8. conrad says:

    I read that paper, and it doesn’t conflict with my claim. Most of the examples are just talking about critical periods in development and how they relate to cognitive skills — something we’ve known about before any modern neuroscience techniques came around. In addition, I can’t think of anyway modern neuroscience paper that has informed us meaningfully (as in, someone could make a decent prediction about some educational practice based on them) over and above the cognitive literature.

  9. E.D. says:

    Damien,

    I would agree that shortages would disappear if employers paid more (many tradespeople don’t work in their trade so could be enticed back) but what if the market clearing wage is beyond the profitability of the firm? For example, motor mechanics can probably earn up to $100,000 pa in the mines yet no more than $50,000 to $60,000 working in the city. City-based businesses have two choices, pay the mines rate and go broke, or offer the city rate and go without.

    On the supply-side, how can an employer entice an 18 yr old to do an apprenticeship paying ~$250pw compared to working as a bar attendant earning at least double that? Trades occupations are unattractive to most young people for other reasons as well.

  10. Sacha says:

    Something relevant to this conversation is that the state and federal govts have decided to introduce national testing of Yrs 3, 5, 7 and 9 school students in literacy and numeracy from next year (I mention this as I’m involved in working on this area at work) which will give some measure of how Australian kids are doing in these areas directly comparably across states and territories and hopefully time (if the tests occur over a number of years).

    There are ways to compare the performance of kids over time and also to attempt to pick out kids on the tails. Hopefully national testing will be beneficial to individual kids (e.g. those on the tails needing help) and to give system-wide information.

  11. Andrew Leigh says:

    Sacha, if we do NAEP-like tests (comparable across regions and time), it’ll be extraordinarily valuable. We’re only 38 years behind the US on this, after all.

    The next question will be: how much data will be released? Will we get only means, or tabulations for poor kids separately? Will we get to see it at a school level? And will we get other moments of the distribution, eg. 25th and 75th percentiles?

  12. conrad says:

    Actually, cross-culturally, I’m not 100% sure about how valuable national testing is — there is certainly a limit to it. There are huge differences across cultures in the amount of testing that is done (cf. Finland vs. Singapore), but the final outcomes (measured by a test :) are often quite similar.

    I also think that too much testing crushes creativity, as people focus on tests too much and not actually learning something (of course, creativity is very difficult to measure, so its hard to quantify that). I think it is already a problem — try and give your undergraduates an assignment where they are not told how to do absolutely everything and how absolutely everything will be marked, and see how many complaints this generates. Its surprising how little many people are able to think for themselves.

  13. Sacha says:

    conrad, as I understand it, the essential difference between national testing and the current state/territory tests is that the one* test is used across all the states/territories. At present, each state/territory may construct its own test and the tests have different flavours. It is possible to compare how kids on one state test might do on another, but there are uncertainties with this kind of analysis of course.

    Andrew, I don’t know how much data will be released. My guess is that the kids on the left tail may be flagged as possibly needing intervention programs (a positive) and at least as much information as if given from state-based tests will be released. But I don’t know.

    If national testing is retained for a number of years, it would no doubt be done so as to enable longitudinal studies.

    *it may be more than one – it may or may not be the case that a different test will be created for the Northern Territory or some part of it

  14. Sacha says:

    Andrew – the distributions of kids and subpopulations can be easily created as can performance by school. But the extent to which information is publically released is another matter. I don’t know what will be released. You could ask the Federal Minister for Education and see what they say!

  15. conrad says:

    Sacha — when I think of national testing, I think of things like TIMMS or perhaps PISA. These are good because they are out of the government’s control (i.e., not open to political interference — as you already seemed to have alluded too in what is likely to get released), and they are used all over the world, so you can get a better comparison than any Australia-wide test. With national testing, you could get an overall drop in performance (like has happened with maths, for example), but some states are still going to claim they are doing well based on a comparison with other states.

    Incidentally, the problem with trying to catch outliers via national testing is that you start adding and more and more tests, which is not neccesarily a positive — someone has to administer them on everyone, versus just get a decent sample, which is going to cost much less (both to the government and the schools).

  16. Sacha says:

    conrad – you can catch many outliers by tweaking the questions in the test, e.g. by having many more easier and harder questions than usual – the more questions there are at a certain “ability level”, the more precise the measurement of a kid’s ability at that level. “Usually”, you would have something like a normal distribution of questions ordered by difficulty (technically, by location) but you can change that of course.

    Alas, there are many constraints on the length of the test – a big one being a kid’s attention span – and so you usually only have N (less than 100) questions in the test. In these N questions, you want to have a range of items in terms of difficulty and content strand, and then there are all these substrands. It’s like a multidimensional jigsaw. If the NT has a very large number of students at the left end of the tail, for arguments sake, the NT may want a “special” version of the test so that their kids are targetted (and their abilities are accurately measured), but then you can have another test for another special region, and so on, and so on, and then it all unravels. Of course, one could posit that each state has its own special curriculum and the test should cater to its curriculum.

    The point about PISA and TIMMS is taken – in fact my company often puts TIMMS questions into tests to make results on one test comparable to the results of TIMSS. This could be done in any national test – although the TIMMS questions are publically available and people don’t usually like putting publically available questions into tests (teachers might teach to them etc).

    I don’t think the content of the tests is really subject to political influence (any such influence would be subtle) but the presentation of the test results may be. But anyone who can understand the technical reports would be able to tease out the reality. Understanding the technical reports would require knowledge of statistics and a bit of psychometrics, but once you have that it’s not too hard to understand.

  17. Sacha says:

    PS – TIMMS and PISA may not cater to the requirements of national testing. I think that TIMMS covers only Years 4 and 8 students.

    I’m mentioning national testing as I’m helping to put together the technical reports on the trial tests for numeracy and literacy at the moment. It’s a well-defined job but enormous.

  18. conrad says:

    Sacha, I think that if you want to test relatively young groups (when all the learning disorders should be caught in my books), you are really looking at far less than 100 questions (I have the same sorts of problems in some of the work that I do). I’m not sure that I’d use more than 30 or so questions on 6-7 year olds and expect decent results if I could possibly help it, otherwise you are just finding the kids with lower attention than those with real disorders.

    It also depends on your resources — if you can get someone to do the testing individually with the kids (versus self-report), there are things like the Woodcock for reading that only take a few minutes to run and have pretty decent validity. This will give you much better results than just lumping all the kids together in a class and doing one test — I guess it depends on whether you want to test a smaller sample more accurately or a larger sample less so. For the second of these, you are not going to be able to pick up the left end of the distribution nearly as well. That seems ok if you just want a number (Victoria does bettern than NT), but if you want teachers to use the test later on to pick up individuals that need help, it is not such a good method.

  19. Sacha says:

    Hi conrad, yes, I didn’t mean 100 questions – certainly 30 questions for really young kids is a fair number of questions. For a lot of these things, it’s all about resources. I suspect that with national testing, the idea is to get an accurate snapshot of the population (not a sample) and to identify kids who may need particular help.

  20. Sacha says:

    Even organising the trialling is an enormous job and I’m glad I don’t do that!

  21. conrad says:

    I can imagine what a pain it is. Even dealing with one or two schools and a single government agency is a pain. I think I’d shoot myself before doing country-wide stuff.

  22. Sacha says:

    Usually ~250 kids do each trial test – think about the organisation required for 80 trial tests… (some kids would do more than one test).

  23. christine says:

    Coming in late to the discussion, but since we’ve got some of the experts on this particular test here: is this particular test going to be a low-stakes/information gathering exercise more like the NAEP, or is there actually going to be money flowing as a result?

    Also, the question of how the data is recorded and whether it’s accessible to researchers is a big deal. Ontario has a standardised test given to all kids in grades 3, 6 and 9. But I gather that it is a bit difficult for researchers to access and work with the data. And it does need to be thought out fairly early on in the process; you’ll probably be getting guys like Andrew knocking at your door as soon as data starts rolling in.

  24. Sacha says:

    Christine, these are questions to ask of the public service. The contractor will probably deal with the tests and analyse the results but the public service is in charge of what is done with them.

    Personally, I’m in all favour of open information – after all, I trained as a mathematician! In principle, I can’t see why the test result analyses couldn’t be made available to researchers (provided that privacy concerns are dealt with of course).

  25. conrad says:

    I’ll be interested to see the confidentiality of the results too. One of my friends worked as a data analyst for the public service, and part of his job was to redistribute (and hence delete) some of the politically contentious categories in some of the reports, otherwise known as aboriginies. I also believe that entire reports sometimes don’t make the light of day from some agencies.

  26. conrad says:

    Actually, now I think about it Sacha, I can’t think of how you would actually test little kids for reading skills without some sort of tester — most early reading problems occur at a single word level (and nonword reading is important at that level too). In addition, some of the problems kids have is to do with reading speed, rather than errors per se (although they are well correlated). Are you guys still using things like the old ACER tests where you get a passage and multi-choice questions?

  27. Sacha says:

    conrad, I’m not the person to ask about the test content for literacy. I deal with numeracy tests (although I can help out with the literacy ones) and their structure/content is usually specified by the client.

    I can tell you that reading tests often involve several passages of text and a set of multiple choice questions for each passage of text, with kids selecting one of usually four options for their answer. I think that in reading tests there may alsobe a few questions in which kids have to write an answer. This structure of reading tests is quite common in Australia – I don’t know the reason/justification for this structure.

    In spelling tests, one structure often used is that a teacher reads a passage of text out while the kids read the passage on a booklet in front of them, except that a number of words are missing from the kids’ text (and replaced with empty rectangles). The teacher reads the passage out again and pauses at each rectangle, and the kids have to write the missing word in.

  28. conrad says:

    Sacha — It sounds like they’re the structure of the standard tests ACER has been using for ages (at least the reading stuff). I think the origininal version is based on Denny-Nelson (1973 ?), although the grammatical structure gets simpler across the years according to someone I know who has been an English teacher for possibly longer than I have existed (I’m told this is due to second language speakers). I’m not a super fan of them as they are not good for really young kids — they do not have thrilling correlations with some of the other tests nor even reading speed. In my books, they’re like a general language test confounded with reading ability, but better than nothing.

Comments are closed.