Challenge:to arouse or stimulate especially by presenting with difficulties

A challenge has been a very common and successful way to prove a point. Its pretty much a fundamental precept in science, "prove to me that you can do this, I'll give you a reward if you can do it". Inf act this model has recently been incorporated by the various X-prize competitions that have arisen, which have fostered intense competition for private space exploration, and more recently automotive fuel efficiency, Lunar Lander, genetic based personalize medicine, and they have even left space for future X-Prizes.

Even the Department of Defense has taken on the challenge model. They have done the $2 million autonomous car grand challenge. And more recently, the $1 million Wearable Power Pack challenge. Its a good model for engineering, I think we can expect to see more.

Does the challenge paradigm work for everything?


Challenge:to dispute especially as being unjust, invalid, or outmoded

1. Junk Science Global Warming Challenge (and they do promote a lot of junk science!)
2. Kent Hovind Evolution Challenge
3. James Randi's Million dollar Paranormal Challenge
4. Ernst-Singh £10,000 Homeopathic Challenge
5. Jock Doubleday's Vaccine challenge [or here]
   

These challenges are clearly different. These aren't designed to stimulate by providing some difficulty, they are in fact set up the dispute claims. Are all challenges equal? As we will see, even in simply disputing claims, these challenges are not all built the same. Can we just presume that if no one has fulfilled the qualifications of the challenge that the premise of the challenge is true?

Consensus is true, until verified evidence to the contrary comes along This doesnt mean that something is true because its the consensus, it just means that prior evidence has lead to a consensus. One of the fundamental precepts of science is that nothing is proven right. Nothing. We have laws of gravity, laws of conservation of energy, laws of this and laws of that. Not one of these is immune to the entry of new conflicting data. For example, I can prove that Newtons Laws of motion are just plain wrong, simply by providing an example where they don't work right? Behold, Mercury. So what happens? Newtons laws of motion, which still, to this day, are vitally important and useful, is superseded by relativity which is more accurate at large speeds and large masses.

Specific positive claims are not true, unless evidence is provided
Things can be proven wrong, in fact, we assume positive claims are wrong until evidence is made available to change that thinking. This is the goal of using the Null hypothesis. If we make a claim, the goal in showing the veracity of that claim is to try to reject the null hypothesis, and we only accept the original claim if we can not. So for example if I my claim is that I can lift 10 tons of steel with my bare hands, we assume that I can not, and it is up to me to prove that I can. If, miraculously, I really can lift the weight, then we must reject the presumption that I can not, as this evidence contradicts the null hypothesis. There is a statistical meaning of null hypothesis which we will not use here, for the purposes of this discussion we simply mean that we presume that the positive claim is false, and it requires evidence that it is true. This is the heart and soul of skeptical thinking. Falsifiability for any theory is required.

One more misconception of science should be met here. It is a very rare occurrence that a previous scientific theory is proved wrong. More often, the old way is still correct for a given set of boundary conditions, like mass or speed. Usually what happens is that as you poke and prod near the boundaries themselves the older theory becomes less and less accurate, and the new theory fills this new space. The new theory will generally reduce to the old theory when you return back to the original boundary conditions. For an example, we once again return to Einstein and Newton. When to return to macroscopic masses, or slower speeds, relativity simply reduces to Newtonian physics.

Similarly, contrary to what New Scientist says, Darwin was not wrong, he was inaccurate. And the newer theorized mechanisms of evolution take in his observations, but also encompass new ones.

So what about these challenges? Are they valid in light of the way science is done? Lets look at these 5 challenges:

The Global Warming Challenge

The winning entry will specifically reject both of the following two hypotheses

UGWC Hypothesis 1

Manmade emissions of greenhouse gases do not discernibly, significantly and predictably cause increases in global surface and tropospheric temperatures along with associated stratospheric cooling.

UGWC Hypothesis 2

The benefits equal or exceed the costs of any increases in global temperature caused by manmade greenhouse gas emissions between the present time and the year 2100, when all global social, economic and environmental effects are considered.

Whats wrong with Hypothesis 1? Its a negative claim. If you know how to prove a negative claim then you can prove to me that you didn't steal 5000 dollars from me (note: this is why Saddam was doomed when he was told to prove he didn't have WMD). To do this , you would have to presume a positive claim (the greenhouse gasses cause global surface temperature rise). This gets in the way of how we generate theories in the first place. Further it flies in the face of how we understand the world around us. Anthropogenic Global warming is a result of interpretation of an incredible amount of evidence, and it is falsiable, i.e. more GHGs leading to lower temperatures despite natural phenomenon, the problem is that the timespan for that sort of change takes a long time.

Whats wrong with hypothesis #2? Its impossible to do. Literally. Economy itself is largely non-linear and chaotic. Any sort of prediction past a few months is inaccurate at best. That is because small events can have huge impacts. Then combine that with social effects (how would you even quantify that financially?) and environmental effects, and you have as much garbage as you have using the same data to prove the converse. The number of assumptions you would have to make invalidate any sort of proof you would come up with.

Evolution Challenge
I'm not the first to point out the flaws in this challenge. You can find it all over the net. Basically, Hovind has spread the net of evolution so wide that it include cosmology, and chemistry. He has built a strawman of what evolution is and then challenged people to tear down the strawman for him. That's like me challenging you to prove that humans actually exist when I define humans to be anything that moves on the planet.

Evolution is falsifiable, meaning there are a number of ways to disprove evolution, none have been evidenced. There is not a single scientist on the planet who will say the theory of evolution (as opposed to the observation, the fact, of evolution) is 100% fact. There is no 100% certain in science. There is always the possibility of new observations adjusting the theories and the predictions of theories, but these adjustments are made to make the science more accurate.

So any challenge on a general scientific subject is suspect, regardless of who is promoting the challenge, how much money is involved, or any other aspect of the challenge itself. Because science is self reflecting, always adjusting for accuracy and accepting new data, because science is self correcting, there is never a way to prove 100% that a theory is true.

Instead of wasting time and posturing, these folks should be spending time digging up the falsifiable evidence for the theory that they hate so much. To, Kent Hovind, I say, go find the rabbit in the cambrian, or find us a real crocoduck, or show a creature being spontaneously generated. Any of these things would throw evolution out the window. That is how science is done.

To Steve Milljoy, purveyor of junk science, I say instead of bloviating about how you think global warming is a myth because you said so, falsify it! Show in a lab set up that increasing the CO2 in a tank with a dark floor and some incident light actually cools the air in the tank. Show where forests are continually growing towards the equator, show arctic ice expanding.

In science we don't ask for proof of something, we ask for evidence and predictions we can test. If we want to show that an idea or something is wrong, we disprove it.

Specific Claim Challenges:
But what about the other challenges? They don't ask to prove a scientific concept. They are far more like the the Ansari Prize, they ask someone to prove something they are already claiming that they can do.

Doubleday is asking someone to inject themselves as follows:

...offers $20,000.00(U.S.) to the first medical doctor or pharmaceutical company CEO who publicly drinks a mixture of standard vaccine additive ingredients in the same amount as a six-year-old child is recommended to receive under the year 2000 guidelines of the U.S. Centers for Disease Control and Prevention (CDC).

The mixture will not contain viruses or bacteria dead or alive, but will contain standard vaccine additive ingredients in their usual forms and proportions. The mixture will include, but will not be limited to: thimerosal (a mercury derivative), ethylene glycol (antifreeze), phenol (a disinfectant dye), benzethonium chloride(a disinfectant), formaldehyde(a preservative and disinfectant), and aluminum

Sounds good to me! Too bad I am not a doctor or CEO of a Pharma Co. I can't understand a single reason why credentials matter if its really a poison. does it only poison CEOs and doctors? I'm not actually sure why he has not been taken up on this offer.....until now. Well, go to that link, you can see what happened, its pretty funny, also with other people who tried to take the challenge, knowingly or not. So, a number of people have applied, none have been able to get through an administrative process to actually get to a point where the prize money is actually a possibility.

This is the big complaint people have about other challenges, no one gets through the initial filter. Doubleday doesn't accept people because they don't have the correct credentials, and even if they do, then he fumbles around with paperwork and asks for 5000 dollars in order to get in. Its true, no one has taken the challenge because Doubleday won't give anyone the chance to take his challenge.

Homepathy Challenge
The homeopathy challenge is done in a similar vein. Homeopathy is simple to test in a double blind, controlled and statistically significant way. And it has, often in fact, it is tested in the exact same way you would test any medicinal treatment, be it homeopathy or Viagra. The result is almost always the same, homepathy is placebo. Of course there is a huge industry around homeopathy, and it is embedded into your taxes here in the form of NCCAM (in America, other countries have their own battles ridding themselves of this).

So, Ernst and Singh have created the homeopathy challenge. to me, they need to be putting some more money where their mouth is, 10 grand (even if it is in british pounds) is hardly enough for anyone to put their career on the line). Further there is no reason to have a deadline on it, do they believe their contention that homeopathy is bunk or not? If I had a million dollars I would hold my own challenge and be safe.

Further, their challenge smacks of a long of the same non-distinct, and subject to interpretation results that all the previous challenges have suffered from. Take a look:

We challenge homeopaths to demonstrate that homeopathy is effective by showing that the Cochrane Collaboration has published a review that is strongly and conclusively positive about high dilution homeopathic remedies for any human condition.

Strongly. Conclusively. High. All of these terms make success in this challenge completely amorphous. Even if Doubleday doesn't let you take his challenge, at least his description of what success would be was pretty good. Ernst and Singh should describing how much better than placebo the homeopathic treatment should work. The word they should be using is significantly. For example, we have treatments for skin infections and erectile dysfunction. We know these things work because when we test them against placebo, we can say that at least 95% of the time the treatment works better than placebo, and we can even say how much better it works.

I would suggest that they should be allowing the challenger to define what "works" means and then define the test based on that claim. It is basic statistics that will discern if the treatment is working. For example, if the homeopath proclaims that their treatment works 10% better than placebo, then you would work with them to decide what "better" means and then explain to them how the statistics will work. For such a small improvement you will be needed a lot of participants!

Randi's Challenge
What I am proposing here, is very similar to how the James Randi Challenge works. That challenge simple: Prove you can do what you say you can do. Sadly, because people do not understand statistics, and it sure isn't a natural comprehension for humans, when many people realize what it means to prove their claim, they back out. That is because it doesn't mean "claim a success when you are successful and ignore failure". It means, do better than randomness. Further it means that the test must be made so you can not use trickery or scientifically explainable methods to pull the stunt.

Unlike Doubleday, when Randi says that no one has made it past preliminary testing, that doesn't mean they were not tested. Hundreds of people have gotten through the administrative part and have actually been tested. Better yet, the skeptics and the applicant work together to develop the preliminary test. Immediately you can see the different between this test and Doubleday's.

I have read of a few times the Randi got cranky at people for proposing stupid things, like not having to eat, ever. Its a dumb test, one in which only death will prove Randi right. I don't think it is unreasonable for some claims to be rejected if debunking the claim requires death.

Wrap Up:
If you are going to hold a challenge there are good ways to do it, and good topics to do it on. Holding a challenge on a whole subject of science is idiotic, especially when using unprovable negative claims. The way to work with science is to provide evidence to falsify the hypothesis. The reasons evolution and global warming are good science is because both are falsifiable, there is lots and lots of evidence for their existence, and both make testable predictions (which have been predicted and tested).

If you are going to hold and honest challenge: follow these rules:

1. Make sure the claim is positive, quantifiable and falsifiable. Asking someone to prove something isnt true is how we went to war. Proving a whole scientific theory with one test is a waste of time. Science is an endeavor of following where the evidence leads you, not making stuff up and asking everyone else to prove you wrong.
2. Make judgment of entry and success be unbiased. If you are holding the challenge you should not be the one who disqualifies people due to their application. Accepting everyone is the other simply way to get around this. If you are setting the bar in a challenge, be sure that the challenge you are setting up is specific and quantifiable in a way that other people, besides you can judge success. Be sure that success is defined before people enter the challenge.
   
3. Make sure nothing in the administrative process disqualifies people. If someone is making a claim they can do something, it should not matter how much money they have, what educational level they have achieved, or where they live.
4. Make the prize money worth it. This is one of the problems with the ansari prize, it cost more than 10 million to develop the winning entry. Psychics and homeopaths make plenty of money scamming their customers, why should they stop doing that?
5. Make the challenge as public as possible. All paperwork, discussions and test procedures should be available in advance of the challenge. The DoD wearable powerpack was a model in this respect. James Randi has a message board doing this.
   

With these 5 rules in mind, I think only the Randi challenge and the Military competitions are viable.