Will Hartog

We develop methods that control the family-wise error rate (FWER) using the classical closure principle on a modern testing object called e values. We give efficient algorithms for computing the test, which inherits nice properties such as allowing for a data-dependent choice of significance level, and having strong always-validity.

The closure principle is a standard tool for achieving family-wise error rate (FWER) control in multiple testing problems. In general, the computational cost for closed testing can be exponential in the number of hypotheses. In this study, we extend the graphical approach for closed testing with weighted Bonferroni to e-values. With valid e-values, a modern alternative to p-values, we can derive strictly more powerful local tests based on weighted averages of e-values. Consequently, this e-value-based closed test is more powerful than the corresponding graphical approach with inverse e-values as p-values. The properties of e-values allow for a data-dependent choice of FWER level and controlling the probability of any false rejection at any time in the sequential setting. Although the computational shortcuts for the p-value-based graphical approach are not applicable, we develop efficient polynomial-time algorithms using dynamic programming for e-value-based graphical approaches with any directed acyclic graph. For special graphs, such as those used in the Holm’s procedure and fallback procedure, we develop tailored algorithms with computation cost linear in the number of hypotheses, up to logarithmic factors.