Known Continuous Distributions

Published

Sunday Jun 8, 2025

1 Uniform Distribution

Definition 1 Continuous random variable \(Y \sim U(a, b)\) if its pdf is, \[ f(y) = \frac{1}{b - a}, \qquad a < y < b (a < b) \]

Theorem 1 Suppose that \(Y \sim U(a, b)\). Find \(Y\)’s cdf.

\[ F(y) = \int_a^y \frac{1}{b-a} \, dt = \frac{y - a}{b - a}, \qquad a < y < b \]

1.1 Interactive Widget

Show the code

import { Inputs } from "@observablehq/inputs"
import { Plot } from "@observablehq/plot"

viewof ua = Inputs.range([0, 10], { label: "Minimum (a)", step: 0.1, value: 2 })
viewof ub = Inputs.range([0, 10], { label: "Maximum (b)", step: 0.1, value: 8 })

uniform_xs = Array.from({ length: 100 }, (_, i) => ua - 1 + (i / 99) * (ub - ua + 2))

uniform_pdf = uniform_xs.map(x => ({
  x,
  y: x >= ua && x <= ub ? 1 / (ub - ua) : 0
}))

uniform_cdf = uniform_xs.map(x => ({
  x,
  y: x < ua ? 0 : x > ub ? 1 : (x - ua) / (ub - ua)
}))

Show the code

Plot.plot({
  title: `PDF of Uniform(${ua}, ${ub})`,
  marks: [
    Plot.line(uniform_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.max(...uniform_pdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

Plot.plot({
  title:  `CDF of Uniform(${ua}, ${ub})`,
  marks: [
    Plot.line(uniform_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...uniform_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomUniform(ua, ub)})).plot()

2 Normal Distribution

Definition 2 (Normal Distribution) A random variable \(Y\) has the normal distribution with parameters \(a\) and \(b^2\) if its pdf is of the form

\[ f(y) = \frac{1}{b\sqrt{2\pi}} e^{-\frac{(y - a)^2}{2b^2}}, \quad -\infty < y < \infty \, (-\infty < a < \infty, b > 0) \]

We write \(Y \sim \mathcal{N}(a, b^2)\).

2.1 Interactive Widget

Show the code

viewof normal_mu = Inputs.range([-10, 10], { label: "mean", step: 0.1, value: 0 })
viewof normal_var = Inputs.range([0, 10], { label: "variance", step: 0.1, value: 1 })

normal_xs = Array.from({ length: 200 }, (_, i) => 
  - 4 * 4 + (i / 199) * 8 * 4  // from μ - 4σ to μ + 4σ
)

// PDF of Normal Distribution
normal_pdf = normal_xs.map(x => ({
  x,
  y: (1 / (normal_var * Math.sqrt(2 * Math.PI))) * Math.exp(-0.5 * ((x - normal_mu) / normal_var) ** 2)
}))

// CDF of Normal Distribution using the error function approximation
normal_cdf = normal_xs.map(x => ({
  x,
  y: 0.5 * (1 + erf((x - normal_mu) / (normal_var * Math.sqrt(2))))
}))

// Helper: error function approximation
function erf(x) {
  // Abramowitz and Stegun formula 7.1.26
  const sign = x >= 0 ? 1 : -1
  const a1 = 0.254829592,
        a2 = -0.284496736,
        a3 = 1.421413741,
        a4 = -1.453152027,
        a5 = 1.061405429,
        p = 0.3275911

  const t = 1 / (1 + p * Math.abs(x))
  const y = 1 - (((((a5 * t + a4) * t + a3) * t + a2) * t + a1) * t) * Math.exp(-x * x)
  return sign * y
}

Show the code

Plot.plot({
  title: `PDF of Normal(${normal_mu}, ${normal_var})`,
  marks: [
    Plot.line(normal_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.max(...normal_pdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

Plot.plot({
  title:  `CDF of Normal(${normal_mu}, ${normal_var})`,
  marks: [
    Plot.line(normal_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...normal_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomNormal(normal_mu, normal_var ** (0.5))})).plot()

2.2 Standardisation

Without a computer, the only way to evaluate the probability is using a normal distribution table. However, such table only provides information about the standard normal distribution. Therefore, it becomes important to use standardisation to change any r.v. following normal distribution to the standard normal distribution.

Theorem 2 (Standardisation Normal Technique) If \(Y \sim \mathcal{N}(a, b^2)\), then \(Z = \frac{Y - a}{b} \sim \mathcal{N}(0, 1)\).

We say that \(Y\) has been standardised, and that \(Z\) is the standardised version of \(Y\).

Note that this technique is used not only for finding the probability in the exam with normal table, this can also be used to normalise the training data for a machine learning model¹.

3 Gamma Distribution

Definition 3 (Gamma Distribution Pdf) A random variable \(Y\) has the gamma distribution with parameters \(a\) and \(b\) if its pdf is of the form \[ f(y) = \frac{y^{a-1}e^{-y/b}}{b^a \Gamma(a)}, \quad y > 0 \quad (a, b > 0) \]

We write \(Y \sim \text{Gam}(a, b)\).

3.1 Mysterious \(\Gamma(\cdot)\)

\(\Gamma(\cdot)\) here is the gamma function, defined by \(\Gamma(k) = \int_0^\infty t^{k-1}e^{-t}\, dt\).

Some properties:

\(\Gamma(k) = (k - 1) \Gamma(k - 1)\) if \(k > 1\)
\(\Gamma(k) = (k - 1)!\) if \(k\) is a positive integer (e.g. \(\Gamma (4) = 3! = 6\))
\(\Gamma(1/2) = \sqrt{\pi}\)

Show the code

function gamma(z) {
  const g = 7
  const p = [
    0.99999999999980993,
    676.5203681218851,
   -1259.1392167224028,
    771.32342877765313,
   -176.61502916214059,
    12.507343278686905,
   -0.13857109526572012,
    9.9843695780195716e-6,
    1.5056327351493116e-7
  ]

  if (z < 0.5) {
    return Math.PI / (Math.sin(Math.PI * z) * gamma(1 - z))
  } else {
    z -= 1
    let x = p[0]
    for (let i = 1; i < g + 2; i++) {
      x += p[i] / (z + i)
    }
    const t = z + g + 0.5
    return Math.sqrt(2 * Math.PI) * t**(z + 0.5) * Math.exp(-t) * x
  }
}

// Generate values for plotting
gam_xs = Array.from({ length: 200 }, (_, i) => 0.01 + i / 199 * (8 - 0.01))
gamma_data = gam_xs.map(x => ({ x, y: gamma(x) }))

// Plot
Plot.plot({
  title: `Gamma Function Γ(x), from x = 0.01 to ${8}`,
  marks: [
    Plot.line(gamma_data, { x: "x", y: "y", stroke: "purple" }),
    Plot.ruleY([0])
  ],
  x: { label: "x", domain: [0, 8] },
  y: { label: "Γ(x)" },
  width: 600,
  height: 300
})

3.2 Mode of Gamma Distribution

Theorem 3 (Mode of Gamma Distribution) \[ Mode(Y) = \begin{cases} b(a-1) & a \geq 1 \\ 0 & a < 1 \end{cases} \]

3.3 Interactive Widget

Show the code

viewof gamma_a = Inputs.range([0.4, 5], { label: "a", step: 0.1, value: 1 })
viewof gamma_b = Inputs.range([0.1, 5], { label: "b", step: 0.1, value: 1 })

function gammainc_lower(x, a) {
  // Lower regularized incomplete gamma function P(a, x)
  // using a simple series expansion
  let sum = 1 / a
  let value = 1 / a
  for (let n = 1; n < 100; n++) {
    value *= x / (a + n)
    sum += value
    if (value < 1e-8) break
  }
  return sum * Math.exp(-x + a * Math.log(x)) / gamma(a)
}

function gammaCDF(x, alpha, theta) {
  if (x <= 0) return 0
  return gammainc_lower(x / theta, alpha)
}

function gammaPDF(x, a, t) {
  if (x < 1e-6) return a < 1 ? Infinity : 0
  return (1 / (t ** a * gamma(a))) * x ** (a - 1) * Math.exp(-x / t)
}

gamma_xs = [
  ...Array.from({ length: 100 }, (_, i) =>
    1e-6 + Math.exp(Math.log(1e-6) + (i / 100) * Math.log(0.5 / 1e-6))  // log scale from 1e-6 to ~0.5
  ),
  ...Array.from({ length: 100 }, (_, i) => 0.5 + i * 0.1)  // linear from 0.5 to 10.4
]


// PDF of Normal Distribution
gamma_pdf = gamma_xs.map(x => ({
  x,
  y: gammaPDF(x, gamma_a, gamma_b)
}))

// CDF of Normal Distribution using the error function approximation
gamma_cdf = gamma_xs.map(x => ({
  x,
  y: gammaCDF(x, gamma_a, gamma_b)
}))

Show the code

Plot.plot({
  title: `PDF of Gamma(${gamma_a}, ${gamma_b})`,
  marks: [
    Plot.line(gamma_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...gamma_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})

Show the code

Plot.plot({
  title:  `CDF of Gamma(${gamma_a}, ${gamma_b})`,
  marks: [
    Plot.line(gamma_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...gamma_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

// @title: Histogram of Samples
// import { Plot, binX, ruleY, line } from "@observablehq/plot"

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomGamma(gamma_a, gamma_b)})).plot()

3.4 Conclusion on the Gamma Distribution

Now, it is clear that the gamma distribution is very expressive with two model parameters. In fact, we can define many specific distribution by using this gamma distribution by fixing some of the model parameters.

4 The Chi² Distribution

Being a special case of the gamma distribution.

Definition 4 (Chi-Square Distribution) If \(Y \sim \text{Gam}(n/2, 2)\), we say that \(Y\) has the chi-square distribution with parameter \(n\).

Denote as \(Y \sim \chi^2(n)\).

Definition 5 (Chi-Square Degree of Freedom) \(n\) in the above formulation is the degrees of freedom (DOF).

Theorem 4 (Mode of Chi-Square Distribution) The mode of \(Y\) is \(n - 2\) if \(n \geq 2\), and it is 0 if \(n \leq 2\).

4.1 Interactive Widget

Show the code

viewof chi_dof = Inputs.range([1, 5], { label: "dof", step: 1, value: 1 })

// PDF of Normal Distribution
chi_pdf = gamma_xs.map(x => ({
  x,
  y: gammaPDF(x, chi_dof / 2, 2)
}))

// CDF of Normal Distribution using the error function approximation
chi_cdf = gamma_xs.map(x => ({
  x,
  y: gammaCDF(x, chi_dof / 2, 2)
}))

Show the code

// @title: Gamma PDF 

Plot.plot({
  title: `PDF of Chi(${chi_dof})`,
  marks: [
    Plot.line(chi_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...chi_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})

Show the code

// @title: Normal CDF

Plot.plot({
  title:  `CDF of Chi(${chi_dof})`,
  marks: [
    Plot.line(chi_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...chi_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

// @title: Histogram of Samples

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomGamma(chi_dof / 2, 2)})).plot()

5 Exponential Distribution

Another special case of the gamma distribution

Definition 6 (Exponential Distribution PDF) If \(Y \sim \text{Gam}(1, b)\), then \(Y\) has the exponential distribution with parameter \(b\).

We write \(Y \sim \text{Exp}(b)\) with the corresponding pdf being, \[ f(y) = \frac{1}{b}e^{-y/b}, \quad y > 0 \]

By using Theorem 3, we obtain the following corollary.

Corollary 1 (Mode of Exponential Distribution) \[ Mode(Y) = 0 \]

Now, we can establish the following connection with all the other ones have discovered. \[ \text{Exp}(2) = \text{Gam}(2/2, 2) = \chi^2(2) \]

This distribution is useful for modelling times until failure of components and times between successive arrivals in a queue.

5.1 Interactive Widget

Show the code

viewof exp_b = Inputs.range([0.1, 5], { label: "n", step: 0.1, value: 1 })

// PDF of Normal Distribution
exp_pdf = gamma_xs.map(x => ({
  x,
  y: gammaPDF(x, 1, exp_b)
}))

// CDF of Normal Distribution using the error function approximation
exp_cdf = gamma_xs.map(x => ({
  x,
  y: gammaCDF(x, 1, exp_b)
}))

Show the code

Plot.plot({
  title: `PDF of Exp(${exp_b})`,
  marks: [
    Plot.line(exp_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...exp_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})

Show the code

Plot.plot({
  title:  `CDF of Exp(${exp_b})`,
  marks: [
    Plot.line(exp_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...exp_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

// @title: Histogram of Samples

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomGamma(1, exp_b)})).plot()

5.2 Standard Exponential Distribution

Definition 7 (Standard Exponential Distribution) A special case of the exponential distribution.

If \(Y \sim \text{Exp}(1)\), we say that \(Y\) has the standard exponential distribution.

6 Beta Distribution

Definition 8 (Beta Distribution) A random variable \(Y\) has the beta distribution with parameters \(a\) and \(b\) if its pdf is of the form \[ f(y) = \frac{y^{a-1}(1-y)^{b-1}}{B(a, b)}, \quad o < y < 1 \; (a, b > 0) \]

We write \(Y \sim \text{Beta}(a, b)\).

Here, \(B(a, b) = \frac{\Gamma(a) \Gamma(b)}{\Gamma(a + b)}\) is the beta function.

6.1 Connection with Uniform Distribution

If \(a = b = 1\), then \(f(y) = 1\), \(0 < y < 1\). Thus \(\text{Beta}(1, 1) = U(0, 1)\).

It can be shown that \(Mode(Y) = (a - 1)/(a + b -2)\) if \(a > 1\) and \(b > 1\).

6.2 Interactive Widget

Show the code

viewof beta_alpha = Inputs.range([0.1, 10], { label: "alpha", step: 0.01, value: 1 })
viewof beta_beta = Inputs.range([0.1, 10], { label: "beta", step: 0.01, value: 1 })

// --- More points near edges ---
beta_xs = [
  ...Array.from({ length: 201 }, (_, i) => i / 200)
].filter(x => x <= 1)

// --- Beta PDF ---
function betaPDF(x, a, b) {
  if (x <= 0 || x >= 1) return 0
  const numerator = x ** (a - 1) * (1 - x) ** (b - 1)
  const denominator = gamma(a) * gamma(b) / gamma(a + b)
  return numerator / denominator
}

// Regularized incomplete beta function I_x(a, b)
// This is a simple continued fraction approximation for 0 < x < 1
// Based on the continued fraction form in NR, adapted for moderate values

function betainc(x, a, b) {
  if (x <= 0) return 0
  if (x >= 1) return 1

  const lnBeta = Math.log(gamma(a)) + Math.log(gamma(b)) - Math.log(gamma(a + b))
  const front = Math.exp(
    a * Math.log(x) + b * Math.log(1 - x) - lnBeta
  ) / a

  let f = 1, c = 1, d = 0
  for (let i = 1; i < 100; i++) {
    const m = i / 2
    const numerator = (i % 2 === 1)
      ? (b - m) * x / (a + 2 * m - 1)
      : -((a + m - 1) * (a + b + m - 1) * x) / ((a + 2 * m - 2) * (a + 2 * m - 1))
    d = 1 + numerator * d
    if (Math.abs(d) < 1e-30) d = 1e-30
    d = 1 / d
    c = 1 + numerator / c
    if (Math.abs(c) < 1e-30) c = 1e-30
    const delta = c * d
    f *= delta
    if (Math.abs(delta - 1) < 1e-8) break
  }

  return front * f
}

// --- PDF values ---
beta_pdf = beta_xs.map(x => ({
  x,
  y: betaPDF(x, beta_alpha, beta_beta)
}))

// Compute the CDF points using the approximation
beta_cdf = beta_xs.map(x => ({
  x,
  y: betainc(x, beta_alpha, beta_beta)
}))

Show the code

Plot.plot({
  title: `PDF of Beta(${beta_alpha}, ${beta_beta})`,
  marks: [
    Plot.line(beta_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...beta_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})

Show the code

Plot.plot({
  title: `CDF of Beta(${beta_alpha}, ${beta_beta})`,
  marks: [
    Plot.line(beta_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...beta_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})

Show the code

// @title: Histogram of Samples

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomBeta(beta_alpha, beta_beta)})).plot()

Footnotes

However, it is more common to use min-max normalisation.↩︎

1 Uniform Distribution

1.1 Interactive Widget

2 Normal Distribution

2.1 Interactive Widget

2.2 Standardisation

3 Gamma Distribution

3.1 Mysterious \(\Gamma(\cdot)\)

3.2 Mode of Gamma Distribution

3.3 Interactive Widget

3.4 Conclusion on the Gamma Distribution

4 The Chi2 Distribution

4.1 Interactive Widget

5 Exponential Distribution

5.1 Interactive Widget

5.2 Standard Exponential Distribution

6 Beta Distribution

6.1 Connection with Uniform Distribution

6.2 Interactive Widget

Footnotes

4 The Chi² Distribution