I’ve finished Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients
Here’s the main thesis of the book:
Drugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analysed using techniques which are flawed by design, in such a way that they exaggerate the benefits of treatments. Unsurprisingly, these trials tend to produce results that favour the manufacturer. When trials throw up results that companies don’t like, they are perfectly entitled to hide them from doctors and patients, so we only ever see a distorted picture of any drug’s true effects. Regulators see most of the trial data, but only from early on in a drug’s life, and even then they don’t give this data to doctors or patients, or even to other parts of government. This distorted evidence is then communicated and applied in a distorted fashion. In their forty years of practice after leaving medical school, doctors hear about what works through ad hoc oral traditions, from sales reps, colleagues or journals. But those colleagues can be in the pay of drug companies – often undisclosed – and the journals are too. And so are the patient groups. And finally, academic papers, which everyone thinks of as objective, are often covertly planned and written by people who work directly for the companies, without disclosure. Sometimes whole academic journals are even owned outright by one drug company. Aside from all this, for several of the most important and enduring problems in medicine, we have no idea what the best treatment is, because it’s not in anyone’s financial interest to conduct any trials at all. These are ongoing problems, and although people have claimed to fix many of them, for the most part they have failed; so all these problems persist, but worse than ever, because now people can pretend that everything is fine after all.
We’re nearly 30 years into Prozac Nation. The pills are cheap and published research shows that they work well. Is everyone cheerful?
In 2008 a group of researchers decided to check for publication of every trial that had ever been reported to the US Food and Drug Administration for all the antidepressants that came onto the market between 1987 and 2004. The researchers found seventy-four studies in total, representing 12,500 patients’ worth of data. Thirty-eight of these trials had positive results, and found that the new drug worked; thirty-six were negative. The results were therefore an even split between success and failure for the drugs, in reality. Then the researchers set about looking for these trials in the published academic literature, the material available to doctors and patients. This provided a very different picture. Thirty-seven of the positive trials – all but one – were published in full, often with much fanfare. But the trials with negative results had a very different fate: only three were published. Twenty-two were simply lost to history, never appearing anywhere other than in those dusty, disorganised, thin FDA files.
How about the efforts at reform? Goldacre shows that they’ve failed on both sides of the Atlantic.
The 1997 FDA Modernization Act created clinicaltrials.gov, a register run by the US government National Institutes of Health. This legislation required that trials should be registered, but only if they related to an application to put a new drug on the market, and even then, only if it was for a serious or life-threatening disease. The register opened in 1998, and the website clinicaltrials.gov went online in 2000. The entry criteria were widened in 2004. But soon it all began to fall apart. … But the first problem for the US register, which could have been used universally, was that people simply chose not to use it. The regulations required only a very narrow range of trials to be posted, and nobody else was in a hurry to post their trials if they didn’t have to.
In 2007 the FDA Amendment Act was passed. This is much tighter: it requires registration of all trials of any drug or device, at any stage of development other than ‘first-in-man’ tests, if they have any site in the US, or involve any kind of application to bring a new drug onto the market. It also imposes a startling new requirement: all results of all trials must be posted to clinicaltrials.gov, in abbreviated summary tables, within one year of completion, for any trial on any marketed drug that completes after 2007. Once again, to great fanfare, everyone believed that the problem had been fixed. But it hasn’t been, for two very important reasons. First, unfortunately, despite the undoubted goodwill, requiring the publication of all trials starting from ‘now’ does absolutely nothing for medicine today. … But there is a second, more disturbing reason why these regulations should be taken with a pinch of salt: they have been widely ignored. A study published in January 2012 looked at the first slice of trials subject to mandatory reporting, and found that only one in five had met its obligation to post results.62 Perhaps this is not surprising: the fine for non-compliance is $10,000 a day, which sounds spectacular, until you realise that it’s only $3.5 million a year, which is chickenfeed for a drug bringing in $4 billion a year. And what’s more, no such fine has ever been levied, throughout the entire history of the legislation.
The search functions on the FDA website are essentially broken, while the content is haphazard and badly organised, with lots missing, and too little information to enable you to work out if a trial was prone to bias by design. Once again – partly, here, through casual thoughtlessness and incompetence – it is impossible to get access to the basic information that we need.
So: let’s say we want to find the results from all the trials the FDA has, on a drug called pregabalin, in which the drug is used to treat pain for diabetics whose nerves have been affected by their disease (a condition called ‘diabetic peripheral neuropathy’). You want the FDA review on this specific use, which is the PDF document containing all the trials in one big bundle. But if you search for ‘pregabalin review’, say, on the FDA website, you get over a hundred documents: none of them is clearly named, and not one of them is the FDA review document on pregabalin. If you type in the FDA application number – the unique identifier for the FDA document you’re looking for – the FDA website comes up with nothing at all. If you’re lucky, or wise, you’ll get dropped at the Drugs@FDA page: typing ‘pregabalin’ there brings up three ‘FDA applications’. Why three? Because there are three different documents, each on a different condition that pregabalin can be used to treat. The FDA site doesn’t tell you which condition each of these three documents is for, so you have to use trial and error to try to find out. That’s not as easy as it sounds. I have the correct document for pregabalin and diabetic peripheral neuropathy right here in front of me: it’s almost four hundred pages long, but it doesn’t tell you that it’s about diabetic peripheral neuropathy until you get to page 19. There’s no executive summary at the beginning – in fact, there’s no title page, no contents page, no hint of what the document is even about, and it skips randomly from one sub-document to another, all scanned and bundled up in the same gigantic file. … unlike almost any other serious government document in the world, the PDFs from the FDA are a series of photographs of pages of text, rather than the text itself. This means you cannot search for a phrase. Instead, you have to go through it, searching for that phrase, laboriously, by eye.
Clinical research trials is increasingly outsourced to private companies and increasingly done on patients outside the U.S. and EU. How much money is involved when a trial is done in the U.S.? And what’s the cost savings for going offshore?
In the US, meanwhile, the use of private community doctors to conduct trials has expanded enormously, with incentives approaching $1 million a year for the most enterprising medics.
As the former chief executive of GSK explained in a recent interview, running a trial in the US costs $30,000 per patient, while a CRO can do it in Romania for $3,000.
In the past, only 15 per cent of clinical trials were conducted outside the USA. Now it’s more than half. The average rate of growth in the number of trials in India is 20 per cent a year, in China 47 per cent, in Argentina 27 per cent, and so on, simply because they are better at attracting CRO business, at lower cost. At the same time, trials in the US are falling by 6 per cent a year (and in the UK by 10 per cent a year).
One problem with these trial is that they generally use patients who have only one condition and who may not be getting any treatment for it. Goldacre asks “Are those findings really transferable, and relevant, to American patients, on all their tablets?”
What does it take to get a drug approved by the FDA?
In general, however, a company would expect to have to provide two or three trials, with a thousand or more participants, showing that its drug works. This is where the smoke and mirrors begin. Although the notion of a simple randomised trial should be straightforward, in reality there are all kinds of distortions and perversions that can come into play, in the comparisons that are made, and the outcomes that are measured for success. For me, ‘What works?’ is the most basic practical question that every patient faces, and the answer isn’t complicated. Patients want to know: what’s the best treatment for my disease? The only way to answer this question when a new drug comes along is by comparing it against the best currently available treatment. But that is not what drug regulators require of a treatment for it to get onto the market. Often, even when there are effective treatments around already, regulators are happy for a company simply to show that its treatment is better than nothing – or rather, better than a dummy placebo pill with no medicine in it – and the industry is happy to clear that low bar.
A paper from 2011 looked at the evidence supporting every single one of the 197 new drugs approved by the FDA between 2000 and 2010, at the time they were approved.15 Only 70 per cent had data to show they were better than other treatments (and that’s after you ignore drugs for conditions for which there was no current treatment). A full third had no evidence comparing them with the best currently available treatment, even though that’s the only question that matters to patients.
This same perverse problem of inadequate comparators also exists in the EU.17 To get a licence to market your drug, the EMA does not require you to show that it is better than the best currently available treatment, even if that treatment is universally used: you simply have to show that it is better than nothing. A study from 2007 found that only half the drugs approved between 1999 and 2005 had been studied in comparison with other treatments at the time they were allowed onto the market (and, shamefully, only one third of those trials were published and publicly accessible to doctors and patients).
Goldacre identifies some subtler problems with drug trials, e.g., the cancer drug Bevacizumab being tested in 1000 trials on different kinds of cancer. Some of those trials showed a statistically significant benefit, but by running so many trials it was of course statistically likely that at least some would show a benefit just due to pure random variation.
Goldacre suggests using massive health care systems with computerized records, such as the UK’s, to run experiments on different drugs where currently physicians have no reason to prefer one versus the other and he has built some software toward this end.
What about all of those drug ads? They’re illegal in most countries.
When ads were first made legal again in the US, they could only appear in print, because of a requirement to include all the side-effect information from the drug label. Since 1997 the rules have been relaxed, and now the side effects can be abbreviated (they are read out at jabbering speed over the end of the TV ads). After this change the pharmaceutical industry’s annual advertising budget rose from $200 million to $3 billion in the space of just a few years.
One study observed patients visiting their doctors in Canada, where direct-to-consumer drug advertising is still banned, and the US. It found that those in the US were more likely to believe they needed medication, more likely to request specific drugs that were advertised on television, and more likely to receive a prescription for that drug.
Trained actors, posing as depressed patients, were sent to visit doctors in three American cities (three hundred visits in all).5 They all gave the same background story, about the problems they were having with low mood, and then were randomly assigned to act in one of three ways at the end of the consultation: to ask for a specific named drug; to ask for ‘medicine that might help’; or to make no specific request. Those who did what ads drive patients to do – ask for a specific drug, or ‘medicine’ – were twice as likely to receive a prescription for an antidepressant pill.
If we relaxed the rules forbidding UK- and European-trained doctors to practice here, would they find the salaries attractive enough to make it worth the trip? (A Massachusetts woman recently called her primary care doc to get an appointment for an annual physical. She was offered one in 2015, about 18 months from the time of her phone call.)
The UK [pay] scales are all publicly accessible: junior doctors are paid between £25,000 and £40,000 per annum for the first five or ten years, and then consultants go on to around £70,000. [That’s $43,000 to $68,500 per year for the junior docs.]
Given the massive size of the industry, the huge quantity of tax dollars that are spent on pills (many of which are useless or harmful), and the fact that intense regulation doesn’t seem to have accomplished anything, the book is worthwhile reading for any citizen.
More: read Bad Pharma
I love the first quote. It actually reads like Douglas Adams – sarcastic and lightly cynical. “These are ongoing problems, and although people have claimed to fix many of them, for the most part they have failed; so all these problems persist, but worse than ever, because now people can pretend that everything is fine after all.”
Compare this to,
“This planet has – or rather had – a problem, which was this: most of the people living on it were unhappy for pretty much of the time. Many solutions were suggested for this problem, but most of these were largely concerned with the movements of small green pieces of paper, which is odd because on the whole it wasn’t the small green pieces of paper that were unhappy.
And so the problem remained; lots of the people were mean, and most of them were miserable, even the ones with digital watches. ”
Or is it just me?
Modern drugs do work somewhat, but not up to the hype. Biplar, depressed, MS, autistic, & epilleptic patients have probably 1/10 more productivity than they did 100 years ago. They’re still nowhere nearly as independent as the lives we take for granted, but it’s better than nothing.
If only our government can step-in and regulate drug advertisement, marketing and promotion to doctors just like it did for cigarette, then we will see some positive outcome?
I’m being serious; have you watch nightly news lately on major networks? What about some major magazine? The amount of drug advertisement is so overwhelming: you see someone in pain and how this wander-drug will transform your life, if only you ask for it from your Dr. But wait for final 2 seconds of the commercial and see if you can read the fine prints about the side effect or effectiveness warning.
I don’t think drugs has extended our life expectancy; it is better education, and extra safety margins we created.
This book sounds like yet another polemic about the evils of the drug industry. I worked in Pharma for 20 years teaching reps how to interact with docs. Guess what? There were no classes on mustache twirling.
Stop for a minute and think about the medicine you or a loved one takes or has taken. Have you taken antibiotics? If you have be mindful that death from infection has been cut geometrically in the antibiotic age.
Oh and about those anti-depressants that no one thinks work; it seems that after the FDA changed the “class labeling” to include a warning on increased suicidality among teens, that fewer adolescents got the medicine they needed and suicide rates actually INCREASED in that population.
And even though it’s too bad about the dead kids and grieving parents at least we can take solace that the FDA took swift decisive action based on shoddy research and anecdotal reports and utterly failed patients.
Seems like the problem is not “intense regulation”, but lack of regulation.
Jack: Goldacre doesn’t say that the newest drugs never work, just that they may not be better, when true efficacy and side effects are factored in, than established off-patent drugs that can be purchased for pennies.
George: As noted in the original posting, most governments simply ban direct-to-patient advertising.
SLR: For the modern pharma industry to justify its rake off the GDP with antibiotics (a pretty old story; see http://www.pbs.org/newshour/rundown/the-real-story-behind-the-worlds-first-antibiotic/ ) is like a 70-year-old talking about his days as a child prodigy.
Tiago: If you think that the regulation that the FDA imposes is not intensive enough (direct spending at the FDA will be about $4.7 billion this coming fiscal year, according to http://www.fda.gov/AboutFDA/ReportsManualsForms/Reports/BudgetReports/ucm395076.htm , and then presumably the costs of compliance by the pharma companies is many times greater), you could lobby Congress to increase their budget beyond the $4.7 billion…
The same kind on (mis)conduct happens in the for-profit field of vaccines. The media has portrayed Dr. Andrew Wakefield as a discredited crazy anti-vaccine nutjob. But when you hear him speak directly he is actually vindicated (through lawsuits wins), rational, pro-vaccine scientist who unwittingly walked into a firestorm when his research questioned the efficacy and safety of vaccines.
Philig – Actually the real need for antibiotics is now. Because doctors have thrown an antibiotic prescription at everyone who walks through their door and the fact that many patients don’t complete their course of medication emergence of bacterial resistance is, according to many reports, a threat to humanity nearly on par with climate change. A brand new antibiotic for treating carbapenem-resistant Enterobacteriaceae has recently shown extremely positive data. So much for the industry resting on its laurels. Beyond the discovery of penicillin, the means to mass produce it for World War II were developed by industry based on deep tank fermentation methods.
BillG– I would refrain from the use of the word nutjob but Wakefield’s “research” has been thoroughly discredited. Anti-vaccine rhetoric is not just benign debate. Parents who don’t understand the science and choose not to vaccinate are exposing their children and others children to the risk of increased morbidity and mortality from diseases that have been nearly extinct for decades.
SLR: The author of “Bad pharma” is a doctor who prescribes drugs every day. He doesn’t suggest shutting down the pharma industry. He just wants some unbiased research done to figure out whether Pill A is better than Pill B. His claim is that the current alliance of commercial pharma, government regulators, medical journals, and universities in the US and EU produces research that is so tainted by commercial interest that it isn’t useful for making decisions regarding treatment.
A very thorough and useful review. I look forward to it. For those interested in the subject of healthcare and money, Maggie Mahar’s book, “Money-Driven Medicine” on healthcare is also worth checking out.
Philg: A brief anecdote. Shortly after beginning a career in Pharma I was on vacation and wearing a logo hat from my company. I was in a bar when a woman remarked to her partner, “Dotty, look it’s that drug you like from that company”. Her partner looked me square in the eye and said, “No I don’t like that drug and I don’t like that company”. I had never seen her before and never said a word to her or her partner. I later found out she was a highly placed regulator at the FDA. If that’s what people think is a cozy relationship between government’s and industry, I’d hate to see hostile.
Second, the Pill A vs Pill B debate is not always the be all and end all. Most patients try several anti-depressants before they settle on one that works. Many depression patients require poly-pharmacy. Hypertension is the same way. Until we have much more definitive criteria based on genetics, the selection of these agents is largely empirical.
Lastly it is important to note that their are several paradoxes in drug research. Some people complain that drugs are not adequately tested in women, but given the potential side effects in pregnancy or during lactation it has been considered unethical to test in this population in most instances. For a cancer drug to go to trial it is most often in patients who have failed standard therapy and are terminal. Show a benefit is a very high bar in these cases. Also while the relationships between institutions, doctors, regulators and pharma may seem too close basic science will never and should never live within pharma for a host of reasons and while user fees at FDA may be viewed as payoffs by some, the government is unwilling to fund the agency adequately to do the job without user fees.
The case against pharma is open and shut for many people. You and I may never agree on the issues but trust me on one thing. The inconvenient truth is far more complex and nuanced than what you read in the press or in books like Bad Pharma.
SLR: I’m not a specialist in this area so I’m not primarily offering my own opinion regarding the pharma industry or the effectiveness of FDA regulation. Mostly I’m trying to summarize accurately Goldacre’s perspective.
[My own experience teaching probability theory (to MIT undergrads) and watching other people try to reason about things that require an understanding of probability theory does tend to confirm some of Goldacre’s points. And my experience just watching people take pills of various kinds without dramatic results also tends to confirm Goldacre’s points. See http://philip.greenspun.com/blog/2008/04/21/statins-cholesterol-health-fancy-employee-compensation-ebitda-and-c/ for something that I wrote earlier on this subject.]
executive summary at ted:
Phil, this one, perhaps, may be your next read to learn more about Pharma by an MD, PhD and a Cochrane Collaboration (an independent organization evaluating efficacy of meds) researcher: http://www.amazon.com/Deadly-Medicines-Organised-Crime-Healthcare/dp/1846198844/ref=sr_1_1?ie=UTF8&qid=1405037117&sr=8-1&keywords=deadly+medicines+and+organised+crime+how+big+pharma+has+corrupted+healthcare
It is more inflammatory than Goldacre’s books (both Bad Pharma and his earlier Bad Science) but is also richer with documenting meticulously the facts and science. I fully support Goldacre’s AllTrials campaign and hope you and your readers will too: http://www.alltrials.net/ It is not possible to make a proper clinical judgment about med’s effectiveness when half of the data may never see the light of the day- hopefully that will be changing. As for the drug promotion, here is an opinion from the trenches that you and your readers (especially since most of your readers are men..) may find interesting : http://www.ncbi.nlm.nih.gov/pubmed/23939515
Philg: Completely aside from the Pharma issue. I thought your article was interesting. I was getting my MBA in finance at NYU at the height of the mortgage backed securities craze. It would be difficult to exaggerate the hype that we were exposed to as students regarding securitization of sub-prime assets. Church on Sunday is less evangelical. But as the Lampoon said in a “letter” from Jean Paul Satre, “Whoops, I went to hell”.
My objection to critics like Goldacre, Kessler, Steve Nissen, Sidney Wolfe and others is their reductionist point of view.
“Dramatic effect” is not typically the result of medical intervention. Sure, if you replace a hip or knee that’s pretty easy to be impressed by. My sister who is a doc constantly marvels that “you’re not dead and you should be” isn’t enough for a lot of people. Your 18 year old self is never coming back. As an example consider spinal fusion, unlike joint replacement the result is often more akin to “your life will now not suck as much as it would have”. Risk intolerance also looms large. I know of several drugs that were either pulled from the market or never made it to market because of “signals” of risk or improperly constructed endpoints. The study populations or patients on these medications often beg for continued access but once a compound is adjudicated to be not worth the risk it will never see daylight again.
To your comments about probability theory, I think that any field of inquiry, sufficiently nuanced is fraught for the individual who weighs in uniformed. One thing that doctors are taught though, is to value data more than anecdotal experience. My only caution to an expert in probability theory would be that the entanglers in medicine can quite much more practical than theoretical. Markets are simple in comparison with human biology. In studies involving surgical technique for instance, it is well documented that the benefit of certain procedures are usually underestimated because of the highly variable skill of the surgeons involved.
SLR: “This book sounds like yet another polemic about the evils of the drug industry”
SLR: “The inconvenient truth is far more complex and nuanced than what you read in the press or in books like Bad Pharma.”
SLR, did you read the book in the 13 or so hours between those comments? You seem to be knowledgable and experienced, so your comments would seem more valid and thus be more interesting to me if you read the book.
Bad Pharma is perhaps a misleading title. It doesn’t suggest that drugs are bad or that pharmaceutical companies are bad. But it does say that the way drugs are tested and marketed is bad. Goldacre doesn’t suggest that new drugs should (or even can) always be radically different/better. Nor than it’s as simple as A versus B. What it does say is that the data on which drug decisions are made is not as good as it could and should be.
There are a number of ways to approach drug choice. The system that we have (let doctors and patients decide) reminds me of Churchill’s remark about democracy – it’s the worst possible system, except for all the others. What happens when some official Big Brother decides that Drug A is THE best drug for Disease Y (and maybe not coincidentally, is also very cheap for Big Brother to pay for) and you have Disease Y and you try Drug A and it just doesn’t work for you. At that point, Big Brother has spoke and you are S.O.L. The current system is flawed but you have to be careful that the cure is not worse than the disease.
Regarding antidepressants, this is a really good rundown of the situation:
http://slatestarcodex.com/2014/07/07/ssris-much-more-than-you-wanted-to-know/
Upshot: Antidepressants do work; the case against them isn’t strong enough to overcome that. One point made therein: a lot of those hidden “failed” trials actually do show an effect, but the effect size is only *very slightly less* than a completely arbitrary number chosen as a cutoff to distinguish “clinically significant” from not.
Quote:
“when you try to control for publication bias, the effect size of antidepressant over placebo is 0.31. […] According to McAllister and Williams, who are working off of slightly different data and so get slightly different numbers, the effect size of placebo is 0.92 and the effect size of antidepressants is 1.24, which means antidepressants have a 0.32 SD benefit over placebo.”
Incidentally, it seems to me Goldacre is crazy to criticize drug approval based on “it’s better than placebo” when we don’t know that “it’s better than the best available drug.” Consider breakfast cereals. We allow people to market new ones if they consist of food that somebody might like. Nobody says “you shouldn’t be able to sell that cereal unless it’s provably objectively better than Cheerios” and that’s a good thing because different people want or need different things in their breakfast. A new drug entering the market means you have more choices. If one drug doesn’t work well for YOU there’s another option now. Even if the core *desired* effect is the same, the *side* effect profile is probably different, the interaction profile with other drugs is probably different, the dosage model is probably different, the new drug might be cheaper or more convenient or (due to competition) might make the OLDER drug cheaper and so on.
The right questions ought to be: (1) will this drug hurt you? (2) does it seem to work? (3) do people want to try it?