Skip to main content

03. Gamma and Beta Functions — A Factory for Normalization Constants

Background 3/5. A probability density is complete only after a normalization constant makes its total integral equal to 1. Both the Gaussian's 2π\sqrt{2\pi} and the Beta distribution's B(α,β)B(\alpha,\beta) come from the two special functions on this page.

Gamma function​

Γ(z)=∫0∞tz−1e−t dt(Re⁡(z)>0)\Gamma(z) = \int_0^{\infty} t^{z-1} e^{-t}\, dt \qquad (\operatorname{Re}(z) > 0)

It is "the integral over the entire product of the polynomial tail tz−1t^{z-1} and exponential decay e−te^{-t}." A substantial portion of the integrals in probability theory take this form after a single substitution.

Core properties​

1. Recurrence relation (proved with one integration by parts)

Γ(z+1)=z Γ(z)\Gamma(z+1) = z\,\Gamma(z)

2. Continuous extension of the factorial

Starting from Γ(1)=∫0∞e−tdt=1\Gamma(1) = \int_0^\infty e^{-t} dt = 1 and repeatedly applying the recurrence gives

Γ(n)=(n−1)!(n=1,2,3,… )\Gamma(n) = (n-1)! \qquad (n = 1, 2, 3, \dots)

The Gamma function is the unique log-convex function extending the factorial to the real numbers (and further to the complex numbers), by the Bohr–Mollerup theorem. This means there is an answer to "why this particular extension?"

3. Half-integer value

Γ(12)=π\Gamma\big(\tfrac{1}{2}\big) = \sqrt{\pi}

The proof reduces via the substitution t=u2t = u^2 to the Gaussian integral ∫−∞∞e−u2du=π\int_{-\infty}^{\infty} e^{-u^2} du = \sqrt{\pi}. The normalization constant 2π\sqrt{2\pi} of the Gaussian distribution comes from here. The Gamma function and the Gaussian integral share the same root.

4. Stirling's approximation

Γ(z)∼2πz(ze)z(z→∞)\Gamma(z) \sim \sqrt{\frac{2\pi}{z}} \Big(\frac{z}{e}\Big)^{z} \qquad (z \to \infty)

This is an essential tool for handling factorials and binomial coefficients of large numbers (for example, in local versions of the CLT and entropy calculations).

Beta function​

B(α,β)=∫01tα−1(1−t)β−1 dt(α,β>0)B(\alpha, \beta) = \int_0^1 t^{\alpha-1} (1-t)^{\beta-1}\, dt \qquad (\alpha, \beta > 0)

This is "the entire area of a mass shaped like tα−1(1−t)β−1t^{\alpha-1}(1-t)^{\beta-1} on [0,1][0,1]." Because the density of the Beta distribution is precisely this integrand, the Beta function is exactly the normalization constant of the Beta distribution.

Relationship to the Gamma function (the most important identity)​

B(α,β)=Γ(α) Γ(β)Γ(α+β)B(\alpha, \beta) = \frac{\Gamma(\alpha)\,\Gamma(\beta)}{\Gamma(\alpha+\beta)}

Proof sketch: write Γ(α)Γ(β)\Gamma(\alpha)\Gamma(\beta) as a double integral and substitute s=x+ys = x + y, t=x/(x+y)t = x/(x+y). The integral separates into Γ(α+β)⋅B(α,β)\Gamma(\alpha+\beta) \cdot B(\alpha,\beta). This substitution is the same calculation as the famous property of Gamma distributions that "the sum and ratio are independent" (the Gamma–Beta relationship).

Useful special values and properties​

  • B(1,1)=1B(1,1) = 1 — why Uniform(0,1) is Beta(1,1)
  • B(α,β)=B(β,α)B(\alpha,\beta) = B(\beta,\alpha) — symmetry
  • For natural numbers m,nm, n:
B(m,n)=(m−1)! (n−1)!(m+n−1)!=[(m+n−1)(m+n−2m−1)]−1B(m, n) = \frac{(m-1)!\,(n-1)!}{(m+n-1)!} = \Big[ (m+n-1)\binom{m+n-2}{m-1} \Big]^{-1}

Its form as the reciprocal of a binomial coefficient foreshadows the deep relationship between the Beta distribution, the binomial distribution, and order statistics (detailed on the Beta distribution page).

In practice: patterns for calculating normalization constants​

Most recurring calculations in probability theory follow one of two patterns.

Pattern 1 — Gamma-type integral: an integral of the form ∫0∞xa−1e−bxdx\int_0^\infty x^{a-1} e^{-bx} dx becomes, with the substitution u=bxu = bx,

∫0∞xa−1e−bx dx=Γ(a)ba\int_0^{\infty} x^{a-1} e^{-bx}\, dx = \frac{\Gamma(a)}{b^{a}}

Pattern 2 — Beta-type integral: ∫01xa−1(1−x)b−1dx=B(a,b)\int_0^1 x^{a-1}(1-x)^{b-1} dx = B(a,b). The integral is finished as soon as the exponents are identified.

For example, the kkth moment of a Beta distribution is

E[Xk]=1B(α,β)∫01xk+α−1(1−x)β−1dx=B(α+k,β)B(α,β)=∏j=0k−1α+jα+β+j\mathbb{E}[X^k] = \frac{1}{B(\alpha,\beta)} \int_0^1 x^{k+\alpha-1}(1-x)^{\beta-1} dx = \frac{B(\alpha+k, \beta)}{B(\alpha,\beta)} = \prod_{j=0}^{k-1} \frac{\alpha+j}{\alpha+\beta+j}

The result follows entirely from reading exponents and applying a recurrence, without ever evaluating an integral "directly." Developing this computational intuition is the goal of this page.

Digamma function (briefly)​

ψ(z)=ddzln⁡Γ(z)\psi(z) = \frac{d}{dz} \ln \Gamma(z)

This appears in logarithmic expectations of Beta and Dirichlet distributions, E[ln⁡X]=ψ(α)−ψ(α+β)\mathbb{E}[\ln X] = \psi(\alpha) - \psi(\alpha+\beta). It is worth at least recognizing the name because the function is unavoidable when using Beta/Dirichlet distributions in variational inference.

What to remember from this page​

  1. Γ(z+1)=zΓ(z)\Gamma(z+1) = z\Gamma(z), Γ(n)=(n−1)!\Gamma(n) = (n-1)!, and Γ(1/2)=π\Gamma(1/2) = \sqrt{\pi}—these three handle most calculations.
  2. B(α,β)=Γ(α)Γ(β)/Γ(α+β)B(\alpha,\beta) = \Gamma(\alpha)\Gamma(\beta)/\Gamma(\alpha+\beta) is one of the most frequently used identities in probability theory.
  3. Once Gamma-type and Beta-type integral patterns become familiar, calculating normalization constants and moments reduces to "reading exponents."
  4. Both the Gaussian's 2π\sqrt{2\pi} and the Beta distribution's B(α,β)B(\alpha,\beta) are faces of the Gamma function.

Next step​