A critical aspect of both evidence-based medicine (EBM) and science-based medicine (SBM) is the randomized clinical trial. Ideally, particularly for conditions with a large subjective component in symptomatology, the trial should be randomized, double-blind, and placebo-controlled. As Kimball Atwood pointed out just last week (me too), in EBM, scientific prior probability tends to be discounted while in SBM it is not, particularly for therapies that are wildly improbable strictly on the basis of basic science, but for both the randomized clinical trial remains, in essence, where the "rubber hits the road," so to speak. Indeed, when the prior probability of a therapy working based on preclinical basic science investigations appears high, EBM and SBM should be (and are, for the most part) more or less indistinguishable.
The ethics of clinical trials, however, demand a characteristic known as clinical equipoise. Stated briefly, for purposes of clinical trials, clinical equipoise demands that there be a state of genuine scientific uncertainty in the medical community over which of the drugs or treatments being tested is more efficacious and safer. One reason (among many) why a prospective randomized, placebo-controlled clinical trial testing an unvaccinated group versus a vaccinated control group to determine whether vaccines cause autism would be completely unethical is that it egregiously violates the principle of clinical equipoise. The unvaccinated group would be left unprotected against potentially life-threatening vaccine-preventable diseases, and that is completely unacceptable from an ethical perspective. Consequently, we have had to rely on less rigorous trial designs to demonstrate that there is no correlation between vaccines and autism. Even so, the accumulated weight of such evidence is enough, and for some questions that is the best we can do because scientific rigor sometimes conflicts with human subjects research ethics. This is an extreme example of lack of clinical equipoise, but it illustrates the point. If we know (or have good scientific reason to suspect) that one treatment is better than another, it is unethical to randomize patients to the arm that receives what is, based on what is known at the time of the trial, likely to be an inferior treatment.
Sometimes, however, the question of whether clinical equipoise exists in a clinical trial is not so obvious as it is for trials proposed by cranks. This situation sometimes crops up in clinical trials for cancer. I was reminded of this issue by a front page story in the New York Times yesterday, New Drugs Stir Debate on Basic Rules of Clinical Trials. In it, reporter Amy Harmon uses a classic human interest story to highlight the issue of clinical equipoise in a clinical trial for a new drug for melanoma that shows great promise. In brief, it is the story of two cousins, one of whom is receiving the new "wonder drug" (whether it is truly a wonder drug or not remains to be seen) in a clinical trial and one of whom is receiving the current standard of care for stage IV melanoma, which, to put it bluntly, sucks in that it has very little effect in prolonging life:
And when, last year, each learned that a lethal skin cancer called melanoma was spreading rapidly through his body, the young men found themselves with the shared chance of benefiting from a recent medical breakthrough.Only months before, a new drug had shown that it could safely slow the cancer's progress in certain patients. Both cousins had the type of tumor almost sure to respond to it. And major cancer centers, including the University of California, Los Angeles, were enrolling patients for the last, crucial test that regulators required to consider approving it for sale.
"Dude, you have to get on these superpills," Thomas McLaughlin, then 24, whose melanoma was diagnosed first, urged his cousin, Brandon Ryan. Mr. McLaughlin's tumors had stopped growing after two months of taking the pills.
But when Mr. Ryan, 22, was admitted to the trial in May, he was assigned by a computer lottery to what is known as the control arm. Instead of the pills, he was to get infusions of the chemotherapy drug that has been the notoriously ineffective recourse in treating melanoma for 30 years.
Even if it became clear that the chemotherapy could not hold back the tumors advancing into his lungs, liver and, most painfully, his spine, he would not be allowed to switch, lest it muddy the trial's results.
The melanoma drug in question, PLX4032 (RG7204) was developed by U.S.-based biotechnology firm Plexxikon, which partnered with Swiss pharmaceutical firm Roche. The drug targets a specific (but common) mutation in melanoma in an oncogene (BRAF--pronounced "b-raf," not "braf"). Interestingly enough, I've actually participated in research with a collaborator who is a melanoma surgeon to detect this very mutation (which was at the time erroneously referred to as the V599E mutation) that results in activation of the BRAF oncogene using PCR-based methods, but it's been five years since I have done any work in this area. In any case, it is gratifying to see that others have developed drugs that target these activating mutations.
Melanoma, for those unfamiliar with the disease, is a cancer that develops in melanocytes, the pigmented cells in the skin. It is highly curable when it is caught early, because surgical excision and testing of the regional lymph node basins is highly effective. It can even occasionally be controlled, resulting in long term survival, in the metastatic setting if the metastasis can be completely excised surgicall. However, once it's stage IV and metastatic to multiple sites or sites from which the tumors cannot be surgically resected, melanoma is incurable. Worse, it's notoriously resistant to chemotherapy. The very best, standard of care chemotherapy regimens including dacarbazine or temozolomide, the former of which is usually first line therapy and the latter of which is sometimes preferred because it can be given orally. Neither work very well.
So in melanoma basically what we have is a disease that is incurable and fatal when it reaches stage IV and for which, unlike the case with, for example, colorectal cancer, there is no good therapy that can markedly prolong survival. Enter PLX4032. PLX4032 is remarkable because it is a targeted agent directed at the V600E BRAF mutation that, in phase I trials, demonstrated significant activity against melanomas harboring that mutation. Since V600E is a common mutation in melanoma, being present in around 50-60% of tumors, that means more than half of patients with metastatic melanoma could potentially benefit from the drug. What do I mean by "significant" activity? In a multicenter dose-escalation phase I trial whose results were recently reported in the New England Journal of Medicine and the press, 81% of patients with V600E mutations showed tumor shrinkage of at least 30%. Moreover, side effects were not prohibitive, with cutaneous (skin) side effects, fatigue, and arthralgias (joint pain) predominating. The worst potential side effect was squamous cell carcinoma of the skin.
These results are quite impressive for a phase I study. Remember that phase I trials are not designed to detect efficacy; they are safety studies. The idea is to increase the dose in humans until dose-limiting toxicities are encountered. If tumor responses are seen, so much the better. Typically, less than 25% of subjects in a phase I trial will demonstrate measurable tumor shrinkage, making 81% very impressive indeed. Moreover, among subjects receiving the drug, the estimated median progression-free survival (PFS) has been 7 months, compared to historical controls of around 2 months. The duration of these responses was on the order of 8 months, which means that the tumors did nearly all start growing again, but, even so, this is far better than the standard of care chemotherapy for metastatic melanoma, dacarbazine, which only slowed tumor growth in 15 percent of patients for an average of two months. Of course, the results of this phase I trial are no guarantee that the apparent increase in PFS (which may or may not be real, given that there is no control arm in a phase I trial) will translate into an increase in overall survival (OS)--remember Avastin--but it is very promising, which is why this new clinical trial was begun. Indeed, I view PLX4032 as a prime example of SBM at work, starting with a basic science observation (many melanomas harbor BRAF mutations that drive their growth) and using that basic science to develop a therapy, ultimately bringing that therapy to clinic.
So here's the question. Given the result of this phase I trial, is there truly clinical equipoise regarding PLX4032 in melanoma that has the V600E BRAF mutation? It's not an easy question, and some physicians come down on either side of the issue, as discussed in the NYT article. First, the side arguing that drugs like PLX4032 are challenging our current cancer clinical trial paradigm:
But critics of the trials argue that the new science behind the drugs has eclipsed the old rules -- and ethics -- of testing them. They say that in some cases, drugs under development, PLX4032 among them, may be so much more effective than their predecessors that putting half the potential beneficiaries into a control group, and delaying access to the drug to thousands of other patients, causes needless suffering.
"With chemotherapy, you're subjecting patients to a toxic treatment, and the response rates are much lower, so it's important to answer 'Are you really helping the patient?' " said Dr. Charles L. Sawyers, chairman of human oncology at Sloan-Kettering. "But with these drugs that have minimal side effects and dramatic response rates, where we understand the biology, I wonder, why do we have to be so rigorous? This could be one of those defining cases that says, 'Look, our system has to change.'"
But does it? On the other side:
Defenders of controlled trials say they are crucial in determining whether a drug really does extend life more than competing treatments. Without the hard proof the trials can provide, doctors are left to prescribe unsubstantiated hope -- and an overstretched health care system is left to pay for it. In melanoma, in particular, no drug that looked promising in early trials had ever turned out to prolong lives.
PLX4032 shrinks tumors in the right patients, for a limited time. But would those who took it live longer? No one knew for sure.
"I think we have to prove it," said Dr. Paul B. Chapman, a medical oncologist at Memorial Sloan-Kettering Cancer Center who is leading the trial. "I think we have to show that we're actually helping people in the long run."
Both are powerful arguments. On the one hand, if drugs like PLX4032 really are far and away more effective than previous generations of experimental drugs in the pipeline to market, then rigidly sticking to the old system has the potential to result in the loss of potentially savable lives and in increased suffering that could potentially have been ameliorated. However, we have been fooled before. Drugs that look highly promising in preliminary studies have ultimately fizzled, and increases in response rates and even increases in PFS have not always translated into increases in OS. Again, remember Avastin. Then, as was suitably mentioned in the NYT article, there is the cautionary example of bone marrow transplantation for advanced breast cancer, which during the 1990s was thought to prolong survival. As a result, both physicians and patient advocacy groups lobbied, using their political muscle to persuade health insurance companies to pay for bone marrow transplants for breast cancer. However, when the careful clinical trials were done, it was found that bone marrow transplantation was no more efficacious than high dose chemotherapy and that it caused death in some cases. The history of cancer research is littered with drugs that appeared efficacious in early clinical trials and then failed when subjected to more rigorous testing. How do we know that PLX4032 isn't one of those drugs?
We don't. Clinical trials are how we figure out the answer to that question:
"My goal," Dr. Chapman shot back, "is to find out as quickly as possible in as few patients as possible whether this works. If we never know, then we're never going to be able to build on anything."
One of the melanoma field's senior clinicians, Dr. Chapman had lived through trial after trial of drugs that failed to live up to early promise. Almost every oncologist knew, too, of a case nearly 20 years earlier when bone marrow transplants appeared so effective that breast cancer patients demanded their immediate approval, only to learn through a controlled trial that the transplants were less effective than chemotherapy and in some cases caused death.
"Making patients' tumors go away is gratifying," Dr. Chapman told critics. "But that's not the business I'm in. I'm in the business of making people live longer. That's what I want to do."
Which brings us back to the whole debate on whether PFS in the absence of OS is reason enough to approve a new anticancer drug, the very same question faced for Avastin. Again, we have no idea that PLX4032 even prolongs OS compared to the standard of care, although we do have an indication that it does prolong PFS. On the other hand, PLX4032 has considerably less toxicity than dacarbazine, meaning that the potential to cause harm would seem to be less, at least from what we know now. All of this leads to a critical question: Is a phase I trial with stellar results enough to claim that there is not equipoise between PLX4032 and the standard of care? Add to that information in the NYT article that describes the "Lazarus" effect observed in some patients who were clearly weeks or even days from death could get off of oxygen and out of bed after placed on PLX4032, sometimes even for months. Is this all enough information to destroy clinical equipoise for this drug and make performing a randomized trial against standard of care unethical?
One thing that puzzled me initially is why the design for this trial was chosen. It's a straightforward open label randomized trial comparing PLX4032 against dacarbazine that does not allow patients in the dacarbazine arm to cross over if they are receiving no benefit. Originally, the principal investigator of the current trial wanted to do a test of PLX4032 verus standard of care in only the sickest patients. If, reasoned Dr. Paul Chapman, the drug did indeed show this "Lazarus effect" in some patients, it would be justification to get it approved as rapidly as possible, even before evidence of improvements in OS were shown. On the other hand, it appears that the drug company (Roche) feared that such a trial would only provide justification for approval for PLX4032 only in that small group of the sickest patients. It wanted approval for the widest indications possible, which requires a large, phase III randomized clinical trial.
In other words, economics appears to have trumped science and ethics. The best design from an ethical standpoint, one that would also be scientifically strong, would have been to do a randomized trial of dacarbazine plus or minus PLX4032. One group would receive dacarbazine plus placebo; the other dacarbazine plus PLX4032. Both groups would be receiving the standard of care, and there would still be genuine uncertainty whether the combination is better than dacarbazine alone. This is how clinical trials testing many new anticancer drugs are performed these days. Because clinical equipoise would tell us that testing a new anticancer drug versus placebo would be unethical in most cases, the two choices left are to test a new drug either against standard of care or added to standard of care. In addition, for trials in which clinical equipoise is not as clear as it should be, in general there is a built-in planned interim analysis of PFS and OS, wherein the trial is stopped if one group is doing significantly more poorly than the other group to the point where the results are so clear-cut that statistically there is no way that the addition of the remaining patients could change the results. In addition, often patients are allowed to cross over to the other group if after a certain period of time they are receiving no benefits. This can go both ways, either control crossing over to experimental drug or those receiving experimental drug crossing over to the control group. Such a design makes the analysis of the data harder to do, but it does mitigate some of the ethical concerns. Unfortunately, in this case, the investigators chose clarity rather than allowing patients to cross over.
Interestingly enough, it is the basic scientists who are most in favor of the contention that this drug should be more widely available:
Some of the strongest criticism came from laboratory researchers who study the biology of the disease and see the drug as fundamentally different from its predecessors. The previous red herrings, they argued, never had such a high response rate. Few other drugs had shrunk tumors in as high a percentage of patients with melanoma or any other solid tumor as PLX4032 had in its first human trial.
"Many of my colleagues who are outstanding clinical investigators have been able to convince themselves that this is a fair thing to do," Dr. David E. Fisher, a leading melanoma biologist at Massachusetts General, said of the controlled trial. "My personal view is it's nuts. I don't know anyone who hasn't shuddered at the concept that we can't let patients on the control arm cross over because we need them to die earlier to prove this point."
Quite frankly, even though I am a staunch advocate of science-based medicine, I, too, shudder at this design. On the other hand, the basic scientists seem rather cavalier to me in their belief that improvements in response and PFS will translate into improvements in OS. After all, even though the toxicities observed in the phase I trial were relatively mild, one of them was squamous cell carcinoma. As promising as PLX4032 is, there is still power to the argument that we can't know for sure if it is superior unless we actually do the careful clinical trial, keeping the failures of promising treatments the past in mind.
In oncology clinical trials, as in clinical trials for treatments of any life-threatening disease, there is always a tension between wanting the "cleanest" possible results versus doing the best for each individual patient. It is a balancing act that relies on the ethics of physicians and a combination of hope and altruism in the patients who become subjects in such trials. Both patients and physicians want the drugs being tested to work, but in some cases clinical trials are very much a case of, to invoke my Star Trek geek tendencies, a case where the good of the many is weighed against the good of the few--or the one--with the balancing of these factors done using knowledge with huge gaps in it. How to maximize the good for as many patients as possible is the goal, but, as we have seen, this is a goal that is not so easily accomplished, just as clinical equipoise is a concept that is easy stated but not so easily applied. PLX4032 teaches us that.
- Log in to post comments
More like this
Excellent analysis, as usual...and timely.
I've received a number of communications about this article and was wondering why there was no real placebo and one of my correspondents wondered why there was no x-over.
Interesting.
Excellent article Orac. It's certainly an ethical minefield, though I tend to follow the same path that you are treading through it. You really need to do the big, well constructed trials to ensure that the anti-tumour effects seen in pre-clinical studies and early clinical trials actually translate into imoproved overall survival (though I'd also include improved quality of life in the equation - QALY is a useful measure. though not always easy to measure).
You mention avastin reminded me of another complication, testing the new drug in combuination with existing standard of care may not always be such a good idea. An example of this is Herceptin, which exacerbates the cardiotoxic effect of anthracyclines. This finding highlights the need to evaluate such likely comnbinations pre-clinically, which seems to me what the science-based approach is all about.
You've probably heard of combined phase 2/3 trials in which the ratio of patients receiving tested drug vs control is adjusted during recruitment for the trial. This is done based on initial and previous results from the same or previous trials if they are available so as to maximize the number of patients in the most efficient arm. For example, if the effect of a drug is very strong, you do not need as many patients in your control arm in order to get significant results. Who says the numbers have to be equal as long as it still random?
That was an excellently written and very interesting article. I had read that article and was eagerly waiting to see what you or Pal would have to say about it. Thanks Orac.
Beebeeo, suggestion mkes me wonder if it might be a good idea to monitor and record the treatment and outcomes of cancer patients in large cancer centres in order to gain an estimate of the effectivness of different existing treatments, againt which new treatments could be evaluated in cases where a more standard phase II trial might violate the principle of clinical equipoise.
This would of course not be as rigorous as a fully controlled, double-blinded clinical trial, but if enough patients were recruited into the control databank then it might be a good alternative, and at the very least help to verify that the control arms of the combined phase 2/3 trials that beebeeo mentioned are a good reflection of the clinical situation.
Of course this would raise issues of patient confidentiality and cost, but these need not be insurmountable.
I could have walked away from this article learning about clinical equipoise and have been satisfied for the day. To read the whole thing was...well the next best thing to afternoon delight.
Is there anything that says that the "control" group must be the same size as the experimental group? The key feature is the absolute numbers, not the fraction.
Of course, it usually makes sense to keep them equal because if you only have 100 people in your trial, you need to have both groups as large as possible. Putting 75 in one and 25 in the other would not be a good idea because you don't have enough in the small group.
I'd be interested to know how many more patients would likely be needed to show improved overall survival if crossing over from control to PLX4032 were allowed. I realize any such estimate is likely to be highly uncertain, but it would still be interesting to know the best estimates.
If the difference is quite large, it adds a further ethical complexity. Namely, doing the cleaner trial without crossing over might prove the clinical benefit much more quickly, making PLX4032 available to all patients sooner. If a crossover design would require several extra years of study, that's extra years before the drug is on the market. IOW, the clean trial design might actually prolong more lives overall, but at greater cost to the people who are in the control arm of the trial.
I'm absolutely baffled why anyone would need a control group on this trial. The "state of the art" treatment is well documented, and there should be plenty of data around to find out if the new drug prolongs life without regenerating the data from the control group; actually, the size of the sample available is probably orders of magnitudes larger than the control in the trial. For a simple comparison, if I measure the melting point of a new compound, I don't have to measure the melting point of water too, I can look it up, even if it's the reference value.
You're referring to the use of historical controls, which is a very dodgy issue in clinical research. The results can depend on so many other factors (stage mix of the cancers, supportive care, other medications, etc.) that using historical controls is rarely indicated. In fact, the only time the use of historical controls can be considered potentially valid in cancer research is when the difference between the overall survival (not the progression-free survival) of a pilot study and historical controls is so huge that there's no plausible way the difference could be made up. Also, remember that pilot studies often appear much more "positive" than the subsequent randomized clinical trials.
@Mu
Part of the reason for a control group is to control for possible confounding variables. Prospectively enrolling subjects for the control helps to address this issue before any experimental treatment is given. Furthermore, if you are prospectively enrolling subjects in the experimental arm, you cannot use randomization if your control is only historical. This could lead to selection bias affecting results.
This sort of trial certainly illustrates the fine line that researchers and associated individuals walk. Approve it too quickly, and you run the risk of putting people at risk from an ineffective or unsafe product. Approve it too slowly, and you risk losing patients to the disease when they might otherwise have been saved...or at least spared some pain and discomfort.
In addition to Avastin, I'd throw out AZT as another example of where this dilemma reared its head.
As soon as I saw the article when I got up and brought the paper in Sunday morning, I knew I'd be reading about it on Respectful Insolence on Monday :)
I unfortunately was pretty busy yesterday and didn't get to finish the article. However, one thing really bugged me about the article itself: I feel that Avastin (or the bone marrow transplant thing) should have been mentioned on the front page.
I can totally envision the very same person who a few months ago complained about "these evil pharma scientists wanting to use humans as guinea pigs and pushing drugs through before they are tested" now complaining about "these evil pharma scientists who are so obsessed with their slide rules and their clinical trials that they are withholding care from the desperately ill in the name of some sick experiment."
The article taken as a whole may have done a good job of capturing the nuance of this difficult decision -- but for people who only read the first few paragraphs, whether it be because they were busy (like me) or because they are hopelessly incurious, there needed to be a better job of balance. Just my two cents.
Re: The lack of crossover... I'm just a lowly engineer, I know nothing of clinical trials, but even just reading the first few paragraphs, I was like, "Wait, don't most studies let people switch if the data is strong enough early on?" That seems like the biggest problem here...
I had just finished reading this (http://motherjones.com/environment/2010/09/dan-markingson-drug-trial-as… I came across your post here. I was caught by your sentence: "In other words, economics appears to have trumped science and ethics." Is there a proposal or an effort to remove economics from the clinical trial process, or is it just too pie-in-the-sky to look for that?
I would imagine that,like much of cancer treatment,it's a question of do you want to just do away with the present tumor,or carcinoma,or do you want to cure the patient,a problem oncologists and cancer surgeons must wrestle with every single day.
One reason (among many) why a prospective randomized, placebo-controlled clinical trial testing an unvaccinated group versus a vaccinated control group to determine whether vaccines cause autism would be completely unethical is that it egregiously violates the principle of clinical equipoise. The unvaccinated group would be left unprotected against potentially life-threatening vaccine-preventable diseases, and that is completely unacceptable from an ethical perspective.
But you,of all people,ought to know the antivaxers who scream for this are,for the most part either germ theory denialists,or believe diseases like measles and mumps are no more dangerous than an ingrown toenail.There is no way to argue with them over this.For now the best thing for those who believe in SBM to do is to ignore them,and maybe they'll eventually go away,or lose any influence they have.Where we need to work is educating doctors,who think autism is just a "mental illness",so they will actually treat a nonverbal child or young adult who is in their ER dying of metabolic organ failure,and not just pull their name up on the computer,see that they have an autism diagnosis and not treat them.This may not be that far fetched.If it happened to me,it happened to others.
This also brings up the fact possibility many oncologists may not be even aware that PLX4032 exists in the first place.I don't know a lot about cancer,but I do know doctors,all too well.Far too many have not kept up with new diagnoses,or treatments since they left medical school.It may be that if you have an oncologist who is 70 years old, and I have been seen by woefully out of touch doctors that old,they may still be treating cancer the way they did,when LBJ was in The White House.A fact too few doctors want to admit about their colleagues.
I would argue that, in the broad sense, one cannot remove economics from anything.
There is so much wrong with the logic in this article, I haven't the time to address them all. Perhaps an honest discussion of the severe limitations inherent to RCT designs would have helped, but the author(s) of this article appear to firmly adhere to the general concept that only an RCT can answer anything. At least the article acknowledges that the requirements of clinical equipoise may not be met in this case. Actually, the randomized trial for PLX4032 against dacarbazine obliterates the clinical equipoise argument. As for running an add-on trial (dacarbazine plus PLX-4032) vs dacarbazine alone or PLX-4032 alone - why? What evidence is there that dacarbazine does much of anything at all? (Actually the FDA and NCI are both on record that it doesn't do much, and dozens of trials using the drug alone and in combinations over more than three decades have never demonstrated a survival advantage afforded by dacarbazine for metastatic melanoma patients.) Also, what would be the scientific basis for combining these two drugs? Is there any molecular reason to think these two drugs would be complimentary? In the absence of any science indicating that combining them makes sense - running such a trial would be nothing but a blind swing in the dark - with its own ethical problems. This is what happens when, over a 50 year period a concept like simplistic statistical comparisons that compare the median patient outcome in one arm to the median patient outcome in another arm, is a great replacement for real science. What about what happened to all those patients who ended up not being the median patient? RCTs can't even ask those questions, let alone answer them. This debate isn't just about ethics. It also is about whether our clinical research and regulatory systems are finally going to move beyond the RCT (an ignorance-based approach developed in the 1950's at the FDA and codified into the Food Drug and Cosmetic Act by Congress in 1962).The science is literally screaming at us that we have to move forward to new, scientific (as opposed to simplistically statistical) ways of asking the clinical questions we really need answered. There really isn't any chance at all that dacarbazine is going to beat PLX4032. None. Patients need PLX4032 right now. Instead, they are waiting, and dying while they wait, for an obsolete and grossly unethical trial to be completed to satisfy reactionary regulators in the FDA's Office of Oncology Drug Products. And this is not a unique situation. It is what the FDA has been doing with resolve for the last 7 years with targeted drugs for cancer. This is exceedingly bad regulation, and even worse "science."
There is so much wrong with the logic in this article, I haven't the time to address them all. Perhaps an honest discussion of the severe limitations inherent to RCT designs would have helped, but the author(s) of this article appear to firmly adhere to the general concept that only an RCT can answer anything. At least the article acknowledges that the requirements of clinical equipoise may not be met in this case. Actually, the randomized trial for PLX4032 against dacarbazine obliterates the clinical equipoise argument. As for running an add-on trial (dacarbazine plus PLX-4032) vs dacarbazine alone or PLX-4032 alone - why? What evidence is there that dacarbazine does much of anything at all? (Actually the FDA and NCI are both on record that it doesn't do much, and dozens of trials using the drug alone and in combinations over more than three decades have never demonstrated a survival advantage afforded by dacarbazine for metastatic melanoma patients.) Also, what would be the scientific basis for combining these two drugs? Is there any molecular reason to think these two drugs would be complimentary? In the absence of any science indicating that combining them makes sense - running such a trial would be nothing but a blind swing in the dark - with its own ethical problems. This is what happens when, over a 50 year period a concept like simplistic statistical comparisons that compare the median patient outcome in one arm to the median patient outcome in another arm, is adopted as a replacement for real science. What about what happened to all those patients who ended up not being the median patient? RCTs can't even ask those questions, let alone answer them. This debate isn't just about ethics. It also is about whether our clinical research and regulatory systems are finally going to move beyond the RCT (an ignorance-based approach developed in the 1950's at the FDA and codified into the Food Drug and Cosmetic Act by Congress in 1962).The science is literally screaming at us that we have to move forward to new, scientific (as opposed to simplistically statistical) ways of asking the clinical questions we really need answered. There really isn't any chance at all that dacarbazine is going to beat PLX4032. None. Patients need PLX4032 right now. Instead, they are waiting, and dying while they wait, for an obsolete and grossly unethical trial to be completed to satisfy reactionary regulators in the FDA's Office of Oncology Drug Products. And this is not a unique situation. It is what the FDA has been doing with resolve for the last 7 years with targeted drugs for cancer. This is exceedingly bad regulation, and even worse "science."
Steve or Steven,
If you think that orac or his friends at Science Based Medicine
or that they recommend a research program wherein
Unfortunately, too much of evidence based medicine (NOT science based medicine) consists of doing just that.
And, the ethical concerns of denying effective treatment to patients, especially when the existing standard is not very good, are what cause researchers like orac such mental anguish.
Perhaps it would be better if this trial had used a carefully defined protocol for transferring patients from the standard group to the new therapy group if the new therapy proved to be substantially better. This has been done before in other trials.
But, in order to make such a major decision, you need a way to reliably and definitely show that the new therapy really is better and be sure you are not fooling yourself.
For the moment, the RCT is the best way to do this.
If you have a better suggestion, I think we would all like to see it.
Unfortunately, all that glitters is not gold and not every drug that looks promising turns out to be really beneficial. We conduct trials like this so that we can tell the difference, not just to "satisfy reactionary regulators".
Steve Walker (#16 & 17) bemoans the inadequacy of the randomised clinical trial (RCT) and urges us to "move forward" to.....what?
That's where Mr. Walker's jeremiad seems to collapse. After pointing out what he sees are the flaws of RCT's, his argument fizzles out without providing even a hint of what he might suggest as a replacement.
Some of his comments also suggest that Mr. Walker doesn't have as firm a grasp of the RCT as he might seem. For example:
If Mr. Walker truly thinks that RCT's only compare median outcomes, he is terribly mistaken. This comment suggests he is either building a "straw-man" argument or is simply reading from a list of "talking points". Either way, it doesn't advance his argument.
Mr. Walker continues in the same vein:
Mr. Walker - if he truly believes what he is writing - needs to learn a lot more about the stochastic nature of biology and the impact of randomness, not to mention basic statistical analysis. For example, there is currently no way to tell in advance which melanoma patients will live to (or beyond) the median survival time. Additionally, there is a range of survival times even without treatment, so it is important to compare median survival (the so-called "median patient") - and, of course, the standard deviation, range, etc. All of which are statistical measures of the populations, "simplistic" or not.
He finally winds up (or winds down) with:
At this point, Mr. Walker should have begun an exposition of his proposed solution to the problems he has (rather poorly) outlined. Absent this, we are left with the impression that even Mr. Walker feels that the RCT, while it has manifold flaws, is still the best way to evaluate medical treatments.
Or is it that Mr. Walker thinks we should abandon RCT's in favor of not evaluating medical treatments? Is he arguing - obliquely - that the FDA (and equivalent governmental agencies outside the USA) should be disbanded in favor of a more laissez faire arrangement where patients and their physicians choose from whatever is available?
Unfortunately, Mr. Walker doesn't answer these questions, despite posting his comment twice.
I think that the limitations and flaws of the RCT are not unknown to people in the medical research field - as Orac's post above makes very clear. Mr. Walker adds nothing to the discussion except his misunderstandings.
Prometheus
Ditto everything Prometheus wrote.
Mr. Walker, if you return, I'd very much like to hear what you think should be done in place of the current system.
Patients only "need PLX4032 right now" if it actually works. For all your complaints about RCTs in general and this one in particular, at least it is an attempt to determine if PLX4032 really does prolong anyone's life.
If you know a better way to determine that without RCTs, please share.
Excellent analysis, Orac. I had read this article yesterday and figured you'd be commenting on it. I had also thought of the crossover design when I saw the article yesterday but I admit I'm a little lost as to how such a trial would be analyzed. Would the patients be analyzed on an "intention-to-treat" basis? Would their outcome be added to the experimental group after crossover? Is crossing over an endpoint in and of itself?
@Jack
Based on my (limited) exposure to cross-over designs, subjects that cross over would be analyzed with ITT. If they are in the control group and cross over to the experimental group, for purposes of the study, their data would be kept in the control group up to the point they left the group. There may be a secondary analysis looking at their experience once on the study drug, but there is definitely some trickiness in analyzing that data; it would be less robust than the primary data and likely more of a curiosity analysis than useful for publication or to support prescribing recommendations.
That's my understanding of it, anyway. I may very likely be wrong.
All that said, though, if a subject is not receiving any benefit from whatever treatment they are on, they should have the opportunity to switch to the other group, even though it may cause headaches for the researchers. Of course, there is always the option to drop out of the study and seek other treatment options.
I teach a biotechnology class in high school and ethics is a part of our core curriculum. Having been involved with many (unsuccessful) and one successful clinical trials, I thank Orac for his analysis, and I thank those who contributed to both sides of this issue. Now to distill this into terms my kids can understand is my lot, but it's a good ethical conundrum.
Thanks, Todd W. I guess mortality would not be a robust endpoint in a crossover trial then? If many patients are crossed over before death, the data would indeed be messy. But an ITT analysis would account for some of that. Perhaps the act of crossing over would be a valuable secondary endpoint as a proxy for treatment failure? Nevertheless, as you pointed out, messy data is a price worth paying for an ethical trial.
These stories about magic bullets for cancer always remind me of the one drug that was actually as good as it seemed - Imatinib. The phase I clinical trials with CML showed a ridiculously high response rate and it received accelerated approval before the completion of a phase III trial.
Actually, Imatinib is a great example of a crossover trial. You can read page 4-7 of this FDA summary for an example of how it's done:
http://www.accessdata.fda.gov/drugsatfda_docs/nda/2003/021588s000_Gleev…
I was just wondering, in cases like this, where a drug shows promise while having relatively mild toxicity / side-effects, would it be practical to amalgamate a phase III trial with the post release-to-market observation?
That is, make the drug available to the public with the condition that the patient must make the same information available to the researches and follow the same protocol as if they were in the experimental arm of the study? Effectively, you'd have one massive experimental arm that can then be compared to historical controls. If it does substantially better or worse, then the question is answered, but if owing to the inaccurate nature of historical controls the data is ambiguous, then a carefully run trial can be run for confirmation.
I have no experience in medicine or clinical trials, so its just a random thought that seems okish.
@26, the problem with that approach arises when (or if) complications resulting from the drug, like the squamous cell carcinoma mentioned above, become apparent in the patients who have been marketed the new "miracle" drug. Trials also tease out the side effects of new drugs, not just immediate ones, but longer term ones that affect OS.
The company could be sued for a very hefty amount, and a lot of very angry people would be left declaring "they do it all for profit!!!". Ultimately, in the face of the generally ill-informed public, the FDA and drug manufacturers can't win. Either they take too long, or they are only interested in profit. At least when they're "taking too long", the drug can be more safely and comprehensively evaluated, hopefully minimizing long-term harm (as in the bone marrow transplant treatments).
RCT is still the safest, most comprehensive method of human study out there. The other plus with RCTs is that the data gathered can be used for years in further research, especially if a lot of data is properly gathered and processed (I'm in the business). A good RCT is worth its weight in gold.
Orac, you and all of your supporters are inexplicably the most ignorant and closed minded people I've ever encountered. For years there have been study after study after study to prove that mercury causes autism. As well, SV40 causes cancer. I bet you don't believe that either. I can't wait until you come down with an SV40 caused cancer. Who will you turn to then?
@douglas stevens:
If you've read some of Orac's posts, apparently there really aren't study after study after study that prove that mercury causes autism. Would you care to name the studies and discuss why these should be convincing?
Re SV40: I suppose it's possible it does. And?
Mephistopheles, he's referring to controversy over polio vaccines contaminated with SV40 virus.
Here's the thing with SV40, as far as I can recall. Studies have found SV40 in human tumor cells. The problem with the "SV40 causes cancer" thing, though, is that looking into it a little more, it appears that the data do not suggest a causal link.
Another review of the literature also found a lack of support for a causal connection: Is there a role for SV40 in human cancer? (PMID 16963733)
Another one from Norway: Is there an association between SV40 contaminated polio vaccine and lymphoproliferative disorders? An age-period-cohort analysis on Norwegian data from 1953 to 1997. (PMID 16287082)
There was even one review that found that differences in detection method applied to the same set of data led to three different results: Polio vaccines, SV40 and human tumours, an update on false positive and false negative results. (PMID 16566440)
So it seems that, at present, there is no clear evidence either way on a causal connection between SV40 and human cancers, though some of the more recent research suggests there is no connection.
Vasha,
Thanks for that. I'm trying to figure how a contaminated vaccine (which is certainly something to be concerned about, and which - if it really did put people at risk - is a serious indictment of the manufacturer's fitness to function) becomes an indictment of vaccines as a whole.
Of course, if, as Todd W. suggests, there is no particular evidence that SV40 causes tumors (much less cancer) then I find the entire post inexplicable.
You know how anti-vax logic works. If a batch of vaccine from 60 years ago was contaminated with something that might cause vaccines, then a modern vaccine against a completely different disease must cause autism.
MA, it is the same logic that says if real medicine actually ever harmed someone, then homeopathy works!
Not that I don't love a good troll bashing, but I'm going to try to change the topic back to clinical trials.
@27
Yes, but this will also come out in the post-marketing surveillance (i.e. the phase IV trial). This isn't really a good enough reason to hold up approval.
@26
This is essentially what occurs when phase III trials show evidence of a strong treatment effect before their completion. As I wrote about before, it's what occured for Gleevec (imatinib).