Concept: Adjustment Method

Dataiku DSS provides an Adjustment Method parameter for hypothesis test cards that perform several comparisons simultaneously. For example, the Pairwise Student’s t-test and the Pairwise Median Mood Test.

When testing a hypothesis, an acceptable significance level such as 0.05 or 0.01 is typically used. This significance level corresponds to the probability of making a type I error — that is, incorrectly rejecting a null hypothesis. However, when several statistical tests are being performed simultaneously, using the same error rate for the set of all comparisons can increase the probability of making type I errors. Using the adjustment method parameter in Dataiku DSS can avoid this situation.

If you choose to use the Adjustment Method parameter, you can specify either the Bonferroni or Holm-Bonferroni adjustment method. For each hypothesis that is tested, these methods adjust the observed p value, which is then compared to the pre-specified significance level.