Concept | Hypothesis testing#

Recall that a hypothesis test allows you to evaluate a pair of competing hypotheses (the null and alternative hypotheses) about the properties of a population, such as its distribution or parameters of the distribution.

Hypothesis test cards in Dataiku display the purpose of the test, to guide in choosing the correct test for our use case.

To create a card, specify one or more test variables, and in some cases, specify additional parameters.

Dataiku sets a significance level (alpha) of 0.05 to manage Type 1 errors (rejecting a true null hypothesis). It calculates a test statistic and compares it to a pre-defined distribution (cumulative probability density function). If the statistic falls in the low-probability tail (<5%), we reject the null hypothesis (p < 0.05). This approach controls the risk of false positives while leveraging the chosen test statistic.

In summary:

Reject the null hypothesis if \(p \leq alpha\)
Determine that the test is inconclusive if \(p > alpha\)

The header of any hypothesis test card includes a question icon that provides detailed information about the test, such as what kind of test it is, the underlying assumptions for the test to be meaningful, and so on.