Stanford University

“Frequentist Local False Discovery Rate Estimation in Empirical Bayes”

ABSTRACT

In many settings when testing multiple hypotheses, a practitioner may want to estimate the probability an individual hypothesis is null with no effect. Under a Bayesian framework, this is possible, but requires correct prior specification. However, most frequentist multiple hypothesis testing approaches estimate the overall percentage of nulls or false positives. The frequentist local false discovery rate (lfdr) is one solution to this problem, which is an estimate of the probability that a hypothesis with a given test statistic value is a null hypothesis. We develop a method that estimates the lfdr consistently when these test statistics are given by an empirical Bayes (EB) methodology, which estimates the unknown prior jointly across all hypotheses. We prove that our estimator is consistent for any properly convergent EB estimate, including the non-parametric maximum likelihood estimate (NPMLE), for suitable EB test statistics, despite the correlations between these statistics. Our method is flexible to many types of EB procedures, and we show that it empirically estimates an upper bound for the true lfdr which can be used in practice to assess the estimated probability of a null for a given hypothesis. We evaluate our method on a dataset of discrimination in hiring among companies.