The Case for More False Positives in Anti-doping Testing

(by Kaiser Fung)

In a prior post on my book blog, I suggested that anti-doping authorities are currently paying too much attention to the false positive problem, and they ought to face up to the false negative problem.

This thought was triggered by the following sentence in Ross Tucker’s informative article (on the Science of Sport blog) about the biological passport, a new weapon in the fight against doping in sports (my italics):

the downside, of course, is that cyclists who are doping can still go undetected, but there is this compromise between “cavalier” testing with high risk of false positives and the desire to catch every doper.

Kudos to Tucker for even mentioning the possibility of false negatives, that is, doping athletes who escape detection in spite of extensive testing. Most media reports on doping do not even acknowledge this issue: reporters often repeat claims by accused dopers that “they had tested negative 100 times before” but a negative test has little value because of the high incidence of false negative errors.

Unfortunately, having surfaced the problem, Tucker failed to ask difficult questions, opting to take the common stance that the overriding concern is minimizing false positive errors. This attitude is evident in the description of the false negative issue as “the desire to catch every doper”, or put differently, the desire to achieve zero false negative error.

In fact, the opposite is true of today’s drug testing regime. There is a desire to achieve near-zero false positive errors. The inevitable statistical result of that objective is to admit large quantities of false negative errors. Elsewhere in Tucker’s article, he cited a study in which the researchers desired a false positive rate of only 0.1% (arguing that 1% would be too high for comfort). The flip side, which Tucker also reported (without comment), is that the same test picked up only “5 out of 11 doping athletes”, which means the false negative rate was over 50%!

Thus, it is unfair to brand the people complaining about false negatives as hoping to “catch every doper”!

***

The implication of high false-negative errors is twofold: (1) the majority of dopers would escape undetected and unpunished so that those testing positive can consider themselves rather “unlucky”; and (2) the negative predictive value of anti-doping tests is low, making a mockery of accused dopers who point to large numbers of prior negative results.

The anti-doping authorities today concern themselves with minimizing false positives, and turn their heads away from the false negative issue. As I explain in Chapter 4 of Numbers Rule Your World (link), there is little hope of reform from within: outside lab experiments, false negative errors are invisible because few athletes would voluntarily disgrace themselves after passing drug tests! (The few cases of admission occurred long after the athletes retired.)

***

What we just discussed are results from lab experiments; what is happening in the real world is likely to be even worse. Any error estimate should be treated skeptically as the best-case scenario.

A useful analogy is the testing of the ballistic missile defense system. At the early state of development, the interceptors are asked to destroy known targets, objects of known number, shape and trajectory launched from known locations at known times.

Similarly, the error rates of anti-doping tests are established by testing athletes in a lab setting, known to be doping, with a known doping schedule, using known compounds at known dosage and known timing. The real-life problem of catching dopers is significantly tougher.

***

For me, the difficult question in the statistics of anti-doping is whether the current system is too lenient to dopers. If the risk of getting caught is low, the deterrence value of drug testing is weak. In order to catch more dopers, we have to accept a higher chance (than 0.1%) of accusing the wrong athletes. That is the price to be paid.

Where do you stand on this?

 

Editor’s note: This reminds me of our work on death sentencing reversals. Given the complexity of the legal system and the high frequency of mistakes of some sort or another, I think the death penalty really only makes sense if you’re willing to occasionally execute an innocent person. Once you decide that you never want to make a mistake in that direction, the system pretty much falls apart.

About these ads

8 Responses to “The Case for More False Positives in Anti-doping Testing”


  1. 1 Paul April 5, 2011 at 1:16 am

    Hi–interesting idea.
    I don’t think it’s actually a mistake to minimize false positives. To put in other terms, the recall of the rule is low, but the precision is extremely high. The downside with false positives is that everyone will claim to be the false positive–consequently no one will be punishable. The way it is now with no false positives, a positive test carries indubitable proof that the person is guilty. The cost of that is that we miss some guilty people, yes, but the alternative is actually one in which we don’t catch any guilty people at all.

  2. 2 zbicyclist April 5, 2011 at 4:00 pm

    There’s a word for false negatives: peleton

  3. 3 Markk April 7, 2011 at 2:17 pm

    Are you kidding? How about we make bloggers at a 1% level lose their jobs, get stigmatized for life and lose a bunch of their future income for no reason. You are saying we should take a several riders every year, fire them and wreck their lives knowingly on bad data?

    That is what accepting a 1% false positive rate means. It would destroy pro cycling in a year or two – much more than just regulating drug use. Look, since the late 1980′s there have been really effective performance boosting treatments readily available and institutionally acceptable, even if illegal, for the first time in history. The goal is to try to minimize the effect and make it so the sport is still about riders (or runners …), not doctors. In the long run this will likely fail, but for me the passport and similar over time tracking is the only way to go.

    The Science in Sport article shows that testing was horrible for the last 15 years or so, institutionally and technically. The answer is better testing and things like the passport, not loosening up and allowing crappy tests because that is all we’ve got. If they stink that much, then lets just get rid of them and regulate PED use by athletes. It may be healthier for them in the long run that way.

  4. 4 Craig Nathanson April 7, 2011 at 4:34 pm

    I agree that the balance should shift to catching more real dopers rather than letting the dopers get away with it but being overly protecting those who play fair. It’s tricky business, to be sure, and the media furor that surrounds a /suspected/ doping case, regardless of whether or not the athlete is a doper, doesn’t help. To the extent that the public believes in a meritocratic approach to sports where fairness is rewarded and cheating is punished, I imagine (or at least hope) they would be on board with nabbing real dopers even if it means that those who aren’t dopers get unfairly flagged.

  5. 6 Kaiser April 8, 2011 at 2:59 am

    Markk: This is certainly a debatable topic which is why I raise the issue. What about the countless clean athletes who lose their jobs, lots of prize money, and the chance to become famous because by design, we give lots of dopers cover with negative test results? Unless we don’t issue positive results at all, there will always be someone who is falsely accused – that’s why every year we find out that innocent people have been wrongly convicted of heinous crimes, and many do spend half or more of their lives in prison or worse as a result.

  6. 7 Akshay Bhat April 14, 2011 at 1:27 pm

    I cant understand your arguments.

    Given an option, an athlete would rather come second and lose to a false negative, than risk a 1% probability of getting banned for life!

  7. 8 Seth with One24 July 15, 2011 at 11:30 pm

    Hi Akshay Bhat,

    Did you read Chapter 4 of “Numbers Rule the World”? It helps to explain the article a little better. :)

    This was a really great share!

    I am a big sports fan and am very interested in learning more about the testing procedures used on athletes.

    Seth


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




About

The Statistics Forum, brought to you by the American Statistical Association and CHANCE magazine, provides everyone the opportunity to participate in discussions about probability and statistics and their role in important and interesting topics.

The views expressed here are those of the individual authors and not necessarily those of the ASA, its officers, or its staff. The Statistics Forum is edited by Andrew Gelman.

A Magazine for People Interested in the Analysis of Data

Follow

Get every new post delivered to your Inbox.

Join 57 other followers

%d bloggers like this: