Rademacher distribution

Mathematical formulation

The probability mass function of this distribution is

: f(k) = \left{\begin{matrix} 1/2 & \text{if }k=-1, \ 1/2 & \text{if }k=+1, \ 0 & \text{otherwise.}\end{matrix}\right.

In terms of the Dirac delta function, as

: f( k ) = \frac{ 1 }{ 2 } ( \delta (k - 1) + \delta (k + 1)).

Bounds on sums of independent Rademacher variables

There are various results in probability theory around analyzing the sum of i.i.d. Rademacher variables, including concentration inequalities such as Bernstein inequalities as well as anti-concentration inequalities like Tomaszewski's conjecture.

Concentration inequalities

Let {xi} be a set of random variables with a Rademacher distribution. Let {ai} be a sequence of real numbers. Then

: \Pr\left( \sum_i x_i a_i t | a |_2 \right) \le e^{ -t^2/2 }

where ||a||2 is the Euclidean norm of the sequence {ai}, t 0 is a real number and Pr(Z) is the probability of event Z.

Let Y = Σ xiai and let Y be an almost surely convergent series in a Banach space. The for t 0 and s ≥ 1 we have

: \Pr\left( | Y | st \right) \le \left[ \frac{ 1 }{ c } \Pr( | Y | t ) \right]^{ cs^2 }

for some constant c.

Let p be a positive real number. Then the Khintchine inequality says that

: c_1 \left[ \sum \left| a_i \right|^2 \right]^\frac{ 1 }{ 2 } \le \left( E\left[ \left| \sum a_i x_i \right|^p \right] \right)^{ \frac{ 1 }{ p } } \le c_2 \left[ \sum \left| a_i \right|^2 \right]^\frac{ 1 }{ 2 }

where c1 and c2 are constants dependent only on p.

For p ≥ 1, c_2 \le c_1 \sqrt{ p }.

Tomaszewski’s conjecture

In 1986, Bogusław Tomaszewski proposed a question about the distribution of the sum of independent Rademacher variables. A series of works on this question culminated in a proof in 2020 by Nathan Keller and Ohad Klein of the following conjecture.

Conjecture. Let X = \sum_{i=1}^n a_i X_i, where a_1^2 + \cdots + a_n^2 = 1 and the X_i's are independent Rademacher variables. Then

: \Pr[|X| \leq 1] \geq 1/2.

For example, when a_1 = a_2 = \cdots = a_n = 1/\sqrt{n}, one gets the following bound, first shown by Van Zuijlen.

: \Pr \left( \left| \frac{ \sum_{i = 1}^n X_i } \sqrt n \right| \le 1 \right) \ge 0.5.

The bound is sharp and better than that which can be derived from the normal distribution (approximately Pr 0.31).

Applications

The Rademacher distribution has been used in bootstrapping. See Chapter 17 of Testing Statistical Hypotheses for example. The distribution is particularly useful in high-dimensional statistics.

The Rademacher distribution can be used to show that normally distributed and uncorrelated does not imply independent.

Random vectors with components sampled independently from the Rademacher distribution are useful for various stochastic approximations, for example:

• The Hutchinson trace estimator, which can be used to efficiently approximate the trace of a matrix of which the elements are not directly accessible, but rather implicitly defined via matrix-vector products.
• SPSA, a computationally cheap, derivative-free, stochastic gradient approximation, useful for numerical optimization.

Rademacher random variables are used in the Symmetrization Inequality.

Related distributions

• Bernoulli distribution: If X has a Rademacher distribution, then (X+1)/2 has a Bernoulli(1/2) distribution.
• Laplace distribution: If X has a Rademacher distribution and Y ~ Exp(λ) is independent from X, then XY ~ Laplace(0, 1/λ).

References

it:Distribuzione discreta uniforme#Altre distribuzioni

References

1. (1994). "Probability in Banach Spaces".
2. Montgomery-Smith, S. J.. (1990). "The distribution of Rademacher sums". Proc Amer Math Soc.
3. (1993). "The distribution of vector-valued Radmacher series". Ann Probab.
4. Khintchine, A.. (1923). "Über dyadische Brüche". [[Mathematische Zeitschrift.
5. (1992-09-01). "On the product of sign vectors and unit vectors". Combinatorica.
6. (2017-08-31). "Tomaszewski's Problem on Randomly Signed Sums: Breaking the 3/8 Barrier".
7. (2021-08-03). "Proof of Tomaszewski's Conjecture on Randomly Signed Sums".
8. van Zuijlen, Martien C. A.. (2011). "On a conjecture concerning the sum of independent Rademacher random variables".
9. (2022). "Testing statistical hypotheses". Springer.
10. (October 2022). "Improved central limit theorem and bootstrap approximations in high dimensions". Annals of Statistics.
11. (2011). "Randomized algorithms for estimating the trace of an implicit symmetric positive semidefinite matrix". Journal of the ACM.