Defending the P-value

Abstract

Attacks on the P-value are nothing new, but the recent attacks are increasingly more serious. They come from more mainstream sources, with widening targets such as a call to retire the significance testing altogether. While well meaning, I believe these attacks are nevertheless misdirected: Blaming the P-value for the naturally tentative trial-and-error process of scientific discoveries, and presuming that banning the P-value would make the process cleaner and less error-prone. However tentative, the skeptical scientists still have to form unambiguous opinions, proximately to move forward in their investigations and ultimately to present results to the wider community. With obvious reasons, they constantly need to balance between the false-positive and false-negative errors. How would banning the P-value or significance tests help in this balancing act? It seems trite to say that this balance will always depend on the relative costs or the trade-off between the errors. These costs are highly context specific, varying by area of applications or by stage of investigation. A calibrated but tunable knob, such as that given by the P-value, is needed for controlling this balance. This paper presents detailed arguments in support of the P-value.