Extensions of Fisher's method

Dependent statistics

A principal limitation of Fisher's method is its exclusive design to combine independent p-values, which renders it an unreliable technique to combine dependent p-values. To overcome this limitation, a number of methods were developed to extend its utility.

Known covariance

Brown's method

Fisher's method showed that the log-sum of k independent p-values follow a χ2-distribution with 2k degrees of freedom:

: X = -2\sum_{i=1}^k \log_e(p_i) \sim \chi^2(2k) .

In the case that these p-values are not independent, Brown proposed the idea of approximating X using a scaled χ2-distribution, cχ2(k’), with k’ degrees of freedom.

The mean and variance of this scaled χ2 variable are:

: \operatorname{E}[c\chi^2(k')] = ck' , : \operatorname{Var}[c\chi^2(k')] = 2c^2k' .

where c=\operatorname{Var}(X)/(2\operatorname{E}[X]) and k'=2(\operatorname{E}[X])^2/\operatorname{Var}(X). This approximation is shown to be accurate up to two moments.

Unknown covariance

Harmonic mean ''p-''value

Main article: harmonic mean p-value

The harmonic mean p-value offers an alternative to Fisher's method for combining p-values when the dependency structure is unknown but the tests cannot be assumed to be independent.

Kost's method: [[Student's t-distribution|''t'' approximation]]

This method requires the test statistics' covariance structure to be known up to a scalar multiplicative constant.

Cauchy combination test

This is conceptually similar to Fisher's method: it computes a sum of transformed p-values. Unlike Fisher's method, which uses a log transformation to obtain a test statistic which has a chi-squared distribution under the null, the Cauchy combination test uses a tan transformation to obtain a test statistic whose tail is asymptotic to that of a Cauchy distribution under the null. The test statistic is:

: X = \sum_{i=1}^k \omega_i \tan[(0.5-p_i)\pi] ,

where \omega_i are non-negative weights, subject to \sum_{i=1}^k \omega_i = 1 . Under the null, p_i are uniformly distributed, therefore \tan[(0.5-p_i)\pi] are Cauchy distributed. Under some mild assumptions, but allowing for arbitrary dependency between the p_i, the tail of the distribution of X is asymptotic to that of a Cauchy distribution. More precisely, letting W denote a standard Cauchy random variable:

: \lim_{t \to \infty} \frac{P[X t]}{P[W t]} = 1.

This leads to a combined hypothesis test, in which X is compared to the quantiles of the Cauchy distribution.

References

References

1. (1975). "A method for combining non-independent, one-sided tests of significance". Biometrics.
2. (2002). "Combining dependent P-values". Statistics & Probability Letters.
3. (1958). "Significance tests in parallel and in series". Journal of the American Statistical Association.
4. (2019). "The harmonic mean ''p''-value for combining dependent tests". Proceedings of the National Academy of Sciences USA.
5. (2020). "Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures". Journal of the American Statistical Association.