In Big Pharma’s Gaming of Medical Studies: How Science Is Twisted and Undermined, flaws in the so-called gold standard of modern medicine, the randomized double-blind placebo-controlled trial, are delineated. There is, though, another dimension to the manipulation of medical trials:

Medical trials are routinely gamed by manipulating statistics. The old adage about lies, damned lies, and statistics was never more true than in modern drug studies.

It isn’t necessary to have a deep understanding of statistics to comprehend some of the tricks being used. This article discusses some of them in plain English. If you spot them in the report of a study, it’s a good indication that the results are being misrepresented by the authors of the study itself. Modern news media virtually never examines the studies themselves. Not even medical outlets bother. They simply spew the conclusions claimed by the researchers and the press releases written for them.

Study Goals and Designs

One of the most common tricks for giving an impression of a treatment’s benefit is to move the goal posts. Heart disease is a prime example of this. The goal of the individual is to maintain or regain health. That is, the patient wants to maintain the ability to breathe freely, to be without pain, and to avoid heart attacks, strokes, and their negative effects.

The goal of a pharmaceutical corporation is different. Big Pharma aims to sell drugs. However, it’s much harder to demonstrate that a drug can prevent a heart attack than it is to show it can change some marker that may be associated with an increased risk of heart attack, even though changing that marker may have no benefit on the real goal of preventing heart attacks. Cholesterol is often used as a stand-in for heart attacks in drug studies.

Let’s imagine that PhakePharmaCo has a new drug called negastat. They set up a trial with a stated end point of lowering cholesterol. It shows a 5% average decrease in cholesterol among study participants. So, PhakePharmaCo starts selling negastat as a heart attack preventive, and the doctors prescribe it for the purpose of preventing heart attacks.

Of course, the drug trial didn’t even attempt to show that negastat prevents heart attacks. All it did was show that it lowers cholesterol. The drug company moved the goal posts from lowered cholesterol to fewer heart attacks! Does negastatin prevent heart attacks? Who knows? PhakePharmaCo didn’t even try to find out.

Tricks of the Medical Numbers Trade

Moving the goal posts is a primary statistics-related trick. There are a host of others. Here are a few:

Relative Risk

Let’s say that drug A adds an average of 2 days to the lifespans of trial subjects, while another drug adds an average of 1 day to subjects’ lifespans. Neither one is an impressive result, especially when, as is so often the case, that extra day is bought at enormous financial cost and adverse effects.

The pharmaceutical researchers get around that by claiming the drug has doubled the life expectancy! They base it on the fact that 2 days is twice as long as 1 day. It’s a lie by statistics. Though Relative Risk is appropriate in some instances, it is never a valid measure of efficacy or safety when comparing a drug against a placebo or another drug.

Relative Risk is a useful tool in epidemiological studies that’s misused in drug trials. Applying epidemiological tools to drug trials, though, is a common trick, as much of the following documents.

Results Reported in “Person-Time”

The concept of Person-Time—person-days, person-weeks, person-months, or person-years—was developed in epidemiological research, where it’s useful in the measurement of incidence in a population. In an epidemiological context, viewing the rate of incidence in terms of person-years, rather than in terms of number of people, can be of value, as it helps deal with problems of people moving in and out of a study population. It’s referred to as incidence rate, cumulative incidence, or incidence proportion.

In drug trials, the number of subjects is, or should be, fixed. If people are moving in and out of such trials, the reasons are often related to the condition or drug being studied. The use of Person-Time can hide those reasons.

What we, as consumers, need to know—and what doctors as purveyors of medical treatments need to know—is how many people are likely to benefit from a treatment, not the benefit per person-days or -months or -years.

Yet, more and more, drug study results are given in terms of Person-Time. It makes no sense, unless the purpose is to mislead.

Intention to Treat

Like Person-Time and Relative Risk reporting, intention to treat comes from epidemiology. It’s an analysis based on the method of treatment initially intended, not on what was finally done. It should seem obvious that Intention to Treat has absolutely no place within drug trials:

Let’s take, for example, a trial of the imaginary drug, novaplac. The intent is to give all non-placebo subjects a certain dose of novaplac. If, however, the treatment is changed for some subjects for any reason at all, the results are meaningless. We’ve learned absolutely nothing about the efficacy of novaplac. 

Intention to Treat reporting would give the results as if everyone had actually received novaplac. Hard as it is to believe, some drug trials game the system by using the obviously bogus Intention to Treat system of reporting results.

Changing Endpoints

The problem of moving goalposts is similar, but is applied before a study starts. Changing Endpoints is applied after it’s found that the results of a study are not to the researchers’ (or pharmaceutical company’s) liking, so they change the endpoint to one that the data would make look good.

The problem with this technique is that the study wasn’t designed to investigate the redefined endpoint, so may not be meaningful. Changing Endpoints is, however, a common technique when the original results aren’t what  the researchers or pharmaceutical firm wanted to see—so they insure that we don’t get to see the undesirable result.

Changing Treatment Duration

The Changing Treatment Duration trick is becoming more common. It’s used to hide adverse effects or loss of efficacy, and is generally implemented by shortening a trial’s length. When study researchers suddenly stop a trial, it’s usually with the claim that the results were so good it would be unethical to continue the trial and deprive the placebo group of treatment benefits.

This game was used to great effect in gaining approval and popularity for hormone replacement therapy. Ultimately, the stunt resulted in immense harm to thousands, possibly millions, of women. A major trial was stopped while benefits seemed apparent, but just before the harm of HRT would show up.

Short Term Trials

Trials that are short when the drug or treatment will likely be long term, as seems to be the goal in most cases, are obviously unacceptable. They are, though, the rule.

Even when the use of a drug is meant to be short term, the long term effects still need to be addressed. Short term beneficial effects must be balanced against the potential of long term harm.

Publication Control

The fact is that trials controlled or run by pharmaceutical corporations are almost never published unless the results are to the company’s liking. The FDA can’t get its hands on them when the pharma company refuses. They justify this on the basis of proprietary rights, by saying that the information is protected as a trade secret. It’s unconscionable, but it’s routine practice.

Publication Control consists of a range of tricks, including:

• Not publishing negative results.
• Withholding trial data while publishing claimed results, making it impossible to verify the legitimacy of claimed results..
• Using ghost writers to produce the articles for publication, thus turning the researchers into cogs in the research mill by removing their names from the process.
• Using famous names as authors, even though they had nothing to do with the project, thus giving an illegitimate impression of authority and helping assure publicity.

In the context of statistics, though, the salient concern in Publication Control is that negative results are not reported, as the issue around P-Values demonstrates.

P-Value

This is a slippery devil. The P-Value is supposed to be a rating of the likelihood of obtaining the results reached by the study. The lower the number, the better, because it implies that the chances of reaching the specific conclusion is low.

So, let’s say a study has a P-Value of .05. That would imply a 5-in-100—1-in-20—chance of getting the obtained result. That sounds pretty good.

What you don’t know, though, is how often similar trials have been done with negative results. What if you were to learn that 22 such trials had been unable to attain the same result? That puts the P-Value in a different focus. When the odds are 1-in-20, then achieving the result once out of 22 trials is hardly surprising.

Why is this relevant? Because the vast majority of Big Pharma trials are never published! It isn’t unrealistic to assume that a large number of failed trials have been produced and hidden away. P-Values have little meaning unless you’re also privy to the full range of studies performed on the same topic.

Statistical Undermining of Medical Research

This is a sampling of some common statistical and study design tricks brought to bear in medical pseudo science. All are used, and not rarely. The study that doesn’t use at least one of these statistical tricks is probably the exception.

The fact that the “gold standard” of randomized double-blind placebo controlled trials has come to rely on statistical manipulation clarifies the fraudulent nature of the concept. Close examination of the gold standard behind evidence-based medicine shows that the so-called evidence doesn’t exist.

The gold standard of medicine is actually tinsel.

Sources:

• Why Most Published Research Findings Are False
• The tricks that drug companies do live after them, their patients are oft interred with their trials