Randomised Trials (again)

According to the NYT, the US army is carrying out randomised experiments on trauma treatments. Their rationale is that with a lot of soldiers dying in Iraq, it’s worth running experiments now to save lives in the future. 

A similar ethical argument is often made to justify randomised policy trials: getting clear answers about what works in education and welfare policy has huge payoffs for policy design. Indeed, in an ANU/Canberra Times debate tonight, I’m going to try pitching the idea that the federal government should create a pot of money, specifically to fund states and territories to run randomised policy trials.

Update: It turns out that there’s similar stuff going on even closer to home. Commenter Mark alerts me to the Head Injury Retrieval Trial being run through NRMA CareFlight. From the NRMA CareFlight website:

How does HIRT work? 

  • NRMA CareFlight identifies severe head injury patients by monitoring 000 calls.
  • Patients are randomly assigned to either the current ambulance/paramedic system or NRMA CareFlight helicopter/specialist physician team.
  • If the case is assigned to HIRT, a dedicated helicopter transports the NRMA CareFlight specialist physician team to the patient within 15 minutes of the 000 call.
  • The HIRT team follows up on the progress, after six months, of those patients in both the treatment and standard care arms of the trial.
  • Results are compared to determine whether providing head injury trauma treatment at the scene of an accident improves the recovery outcomes.
  • The Trial will operate for three years during daylight hours.
  • To achieve statistical validity, the Trial will involve about 510 unconscious patients, half of whom will be treated by HIRT and half of whom will be treated by standard care as currently provided by the Ambulance Service.
  • Advertisements
    This entry was posted in Australian Politics, Iraq. Bookmark the permalink.

    9 Responses to Randomised Trials (again)

    1. Mark says:

      Andrew one of the most important randomised clinical trials of this type anywhere in the world is running right here right now in Sydney. It’s the Head Injury Retrieval Trial being run through NRMA CareFlight. In a nutshell involves rapid delivery of a specialist to treat head injuries at the scene rather than waiting to get them to hospital. Its biggest ethical issue is randomisation – which means they are called off on missions where they could make a life-or-death difference. This is causing a lot of soul searching in the clinical and paramedic world but is deemed necessary to prove the worth of this very expensive service. Ultimately it could change the whole world of emergency medicine. More details on http://www.careflight.org.

    2. Kevin Cox says:

      Surely you can run trials where people can optin to them if they wish to and with appropriate statistical techniques account for the “non randomness” of these volunteers. The problem with randomisation is that even if you think they are randomised they probably won’t be.

      A good of mine had nose cancer. He heard about a “randomised trial” experiment going on for treatment. He then badgered people to be one of the lucky few to get onto the program and ten years later he is still alive.

    3. Steve Edney says:

      Kevin,

      I think you mistake the random element in the trial. its not who is included that’s random. Rather the randomness is whether or not they are one of the people who get the treatment trialled or the control group.

      If your friend was one of the ones in the randomly selected control group then he may not still be alive.

    4. Kevin Cox says:

      Steve perhaps I have not made myself clear. My understanding of randomness in random trials is that as people come for treatment you allocate randomly whether they will get treatment A or treatment B. If someone who is to be included in a random trial says I want A and gets it then the randomness is “destroyed”. I am questioning the need have random trials on two grounds.

      1. In practice the procedure may not be as random as is thought.
      2. Does it matter anyway? You will be looking at all the important characteristics of the people and if these characteristics are “not random” between the two groups then you can account for it in the statistics.

      I know that randomness is meant to take into account all those charactersitics that you do not know and so can’t be accounted for but I would question whether there are too many of those anyway and experimenters have a pretty fair idea on what is likely to be important.

      If what I say is true then perhaps some of the objections to trials are eliminated because you can give people a choice and so eliminate the moral dilemma of randomly saving some people and not others. In effect you let them choose.

    5. conrad says:

      Kevin,

      randomized trials are used over a huge range of experiments, so the extra effort you need in running them and what you get for it depends on the experiment. They are generally considered best practice in many areas — and at least for some areas (like new drugs), the vast majority of trials fail (i.e., don’t show the efficacy wanted). So its clear that experimentors don’t know what is important in their experiments a lot of the time, or at least can’t predict it well. Its also why they have to do different demographic groups (kids, adults etc.) — because the factors influencing the efficacy are not well known.

    6. Bruce Bradbury says:

      Kevin,

      the problem with non-random trials is in your statement “You will be looking at all the important characteristics of the people”. Unfortunately, this is rarely the case. Usually there are important characteristics associated with outcomes that the researcher cannot feasibly measure. This is particularly the case when either the service provider or the recipient is able to make a choice about whether to receive treatment.

      To give one example. A couple of years ago I was involved in evaluating a trial where some elderly people were given extra services with the hope that this might delay their entry into nursing home. We had very detailed information on their previous use of health services – which we controlled for using a sophisticated statistical methodology. However we found that those receiving the services were _more_ likely to go into nursing home. It appeared that those delivering the service had more information than us on the person’s need for assistance, and were providing the service to the most needy. Now if the trial had been random, and we had still found this result, then it would have been very interesting. As it was, the non-randomness meant that we couldn’t really say anything about the effectiveness of the program.

      In the health field, the moral dilemma of excluding some people is resolved by recognising that the excluded will:
      a) still get the pre-existing level of support (and so will not be made worse off by the experiment)
      b) might even be better off if the intervention doesn’t work (not at all uncommon).

      I don’t see why these principles can’t extend to other sorts of evaluations.

      There are of course other limitations with random experiments. They are expensive, narrow in focus and often don’t tell us a lot about the causal mechanisms involved. If the program being evaluated is relatively small, you will probably get better ‘bang for the buck’ by doing a more process-oriented evaluation (in essence, asking the people involved whether they think it works). This is what we did in the above-mentioned evaluation. The service recipients felt it was a great program, and so it was continued.

    7. Kevin Cox says:

      Bruce I understand how randomised trials are meant to work and your example did not appear to be a randomised trial because the people were not chosen at random and yet you managed to understand what had happened and I expect you have a good idea on the effect of the home visits because you had the information on the health of the people at the start of the project and you were able to explain your “experimental results”.

      I would also expect that if you allowed the people themselves to choose whether they received more home visits or went to a nursing home that those who chose themselves would stay out of nursing homes much longer. In other words do not “run experiments” but give people choices and measure what happens and I suggest you will get just as much information . I know the theory of randomised trials – I just question the need for them from the point of view gaining knowledge in situations where you can give the subjects a choice of whether to have a “treatment”.

    8. Bruce Bradbury says:

      Kevin, yes, our study was not a randomised trial – but that was my point – that’s why it didn’t work.

      Your suggestion about letting people choose is interesting, but in fact not that different to how these programs work in practice. People are offered services but decide themselves whether to use them. Giving people choices might be good, but the best way to know this is to run a randomised trial. That is, some randomly selected people are given the choice of whether to take up the new services and others only receive the existing options.

      Without such a trial, we will always suspect that those who take up the home care services are those who would have stayed at home anyway. (In our case the opposite appeared to occur, because the service providers rather than the recipients were making the choices).

    9. Kevin Cox says:

      Bruce I don’t understand. When you say your trial did not work surely it did? You came to the conclusion that providing home care was a sensible way of allocating resources. What was the purpose of the trial if not to help you make this decision? What I contend is that there was no need to have a randomised trial – just have a trial.

      I am really questioning the need to use experimental designs that were designed for testing non sentient subjects (like plants). I suggest that when the subjects have choices and can influence the outcome of the trials then randomisation is not going to add much to the knowledge obtained. Given this then don’t bother trying to run experiments just run alternative treatments and measure what happens.

    Comments are closed.