"Accept null hypothesis" or "fail to reject the null hypothesis"?

  • I'm trying to conduct a Student's t-test for a table of values while trying to follow the explanation and details found on this website. I understand that if the p-value is

    • <.01 then it's really significant
    • >.05 it's not significant
    • in between then we need more data

    But on that page they seem to accept their null hypothesis no matter what the p-value is. So I'm really not understanding now when to accept or reject the null hypothesis.

    • When you do you accept or reject the null-hypothesis?
    • Is it true that you are never supposed to accept your null hypothesis, but rather reject or fail to reject the null?

    You are right: If the $p$-value is >.05, we often say that we fail to reject the null hypothesis or that we don't have evidence to suggest that the means are different. This does *not* mean that the null hypothesis is true. But they explain it on the website: "In science, when we accept a hypothesis, this does NOT mean we have decided that the hypothesis is correct or that it is probably correct." I doubt if "accept" is the best term in this case as it can lead to confusion.

    This article here also advocates that the term "accept" should not be used by scientists.

    @COOLSerdash thanks for the response, the usage of the word "accept" definitely threw me off because it almost said "we accept but we don't accept". I was just having trouble understanding how I can effectively use t-tests during science experiments. Thanks again

    The explanation on that website is very poorly worded.

    "fail to reject the null hypothesis" (or something similar) is the way I generally put it on the rare occasions when I formally test a hypothesis and don't reject the null. I almost never think the null has a chance to be actually true so it's more a lack of evidence against the null than in any sense an acceptance that the null is the case.

    All this begs the questions of why we need formal hypotheses vs. estimating a quantity of interest and reporting confidence intervals. There is no bifurcation implied by estimation.

    This seems to be largely a duplicate of Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis? which although a later question, seems to have received more attention (votes and answers).

    @COOLSerdash The article you mentionned disappeared. I'd be curious to know what it was if you can recall the title/author. Thx

    @JasonV I don't recall the author, but I think it was this text.

  • I would suggest that it is much better to say that we "fail to reject the null hypothesis", as there are at least two reasons we might not achieve a significant result: Firstly it may be because H0 is actually true, but it might also be the case that H0 is false, but we have not collected enough data to provide sufficient evidence against it. Consider the case where we are trying to determine whether a coin is biased (H0 being that the coin is fair). If we only observe 4 coin flips, the p-value can never be less than 0.05, even if the coin is so biased it has a head on both sides, so we will always "fail to reject the null hypothesis". Clearly in that case we wouldn't want to accept the null hypothesis as it isn't true. Ideally we should perform a power analysis to find out if we can reasonably expect to be able to reject the null hypothesis when it is false, however this isn't usually nearly as straightforward as performing the test itself, which is why it is usually neglected.

    Update: The null hypothesis is quite often known to be false before observing the data. For instance a coin (being asymmetric) is almost certainly biased; the magnitude of this bias us undoubtedly negligible, but not precisely zero, which is what the H0 for the usual test of the bias of a coin asserts. If we observe a sufficiently large number of flips, we will eventually be able to detect this miniscule deviation from exact unbiasedness. It would be odd then to accept the "null hypothesis" in this case as we know before performing the test that it is certainly false. The test is certainly still useful though as we are generally interested in whether the coin is practically biased.

    And what if the Power (from simulations) for that test is really high. Could I say "accept the Null"?

    That would be better, but there are still a lot that is left implicit, for instance H1 may not be the only alternative to H0 and the relative prior probabilities of H0 and H1 (which only really enter into the test indirectly via $\alpha$ and $\beta$) may be strongly against H0. The real problem is that we really want to know the probability that H0 is true, which a frequentist test can't give us, so it is a good idea to avoid terminology that could be interpreted that way, I like "fail to reject the null hypothesis" as it implies that its meaning is subtle (which it is!).

    There is also the point that sometimes you know a-priori that the null hypothesis is certainly false (e.g. that a coin is ***exactly*** unbiased), and it would be odd to accept something that you know isn't true even before performing the test.

License under CC-BY-SA with attribution

Content dated before 6/26/2020 9:53 AM