Known Continuous Distributions

Published

Sunday Jun 8, 2025

1 Uniform Distribution

Definition 1 Continuous random variable \(Y \sim U(a, b)\) if its pdf is, \[ f(y) = \frac{1}{b - a}, \qquad a < y < b (a < b) \]


Theorem 1 Suppose that \(Y \sim U(a, b)\). Find \(Y\)’s cdf.

\[ F(y) = \int_a^y \frac{1}{b-a} \, dt = \frac{y - a}{b - a}, \qquad a < y < b \]


1.1 Interactive Widget

Show the code
import { Inputs } from "@observablehq/inputs"
import { Plot } from "@observablehq/plot"

viewof ua = Inputs.range([0, 10], { label: "Minimum (a)", step: 0.1, value: 2 })
viewof ub = Inputs.range([0, 10], { label: "Maximum (b)", step: 0.1, value: 8 })

uniform_xs = Array.from({ length: 100 }, (_, i) => ua - 1 + (i / 99) * (ub - ua + 2))

uniform_pdf = uniform_xs.map(x => ({
  x,
  y: x >= ua && x <= ub ? 1 / (ub - ua) : 0
}))

uniform_cdf = uniform_xs.map(x => ({
  x,
  y: x < ua ? 0 : x > ub ? 1 : (x - ua) / (ub - ua)
}))
Show the code
Plot.plot({
  title: `PDF of Uniform(${ua}, ${ub})`,
  marks: [
    Plot.line(uniform_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.max(...uniform_pdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
Plot.plot({
  title:  `CDF of Uniform(${ua}, ${ub})`,
  marks: [
    Plot.line(uniform_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...uniform_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomUniform(ua, ub)})).plot()

2 Normal Distribution

Definition 2 (Normal Distribution) A random variable \(Y\) has the normal distribution with parameters \(a\) and \(b^2\) if its pdf is of the form

\[ f(y) = \frac{1}{b\sqrt{2\pi}} e^{-\frac{(y - a)^2}{2b^2}}, \quad -\infty < y < \infty \, (-\infty < a < \infty, b > 0) \]

We write \(Y \sim \mathcal{N}(a, b^2)\).


2.1 Interactive Widget

Show the code
viewof normal_mu = Inputs.range([-10, 10], { label: "mean", step: 0.1, value: 0 })
viewof normal_var = Inputs.range([0, 10], { label: "variance", step: 0.1, value: 1 })

normal_xs = Array.from({ length: 200 }, (_, i) => 
  - 4 * 4 + (i / 199) * 8 * 4  // from μ - 4σ to μ + 4σ
)

// PDF of Normal Distribution
normal_pdf = normal_xs.map(x => ({
  x,
  y: (1 / (normal_var * Math.sqrt(2 * Math.PI))) * Math.exp(-0.5 * ((x - normal_mu) / normal_var) ** 2)
}))

// CDF of Normal Distribution using the error function approximation
normal_cdf = normal_xs.map(x => ({
  x,
  y: 0.5 * (1 + erf((x - normal_mu) / (normal_var * Math.sqrt(2))))
}))

// Helper: error function approximation
function erf(x) {
  // Abramowitz and Stegun formula 7.1.26
  const sign = x >= 0 ? 1 : -1
  const a1 = 0.254829592,
        a2 = -0.284496736,
        a3 = 1.421413741,
        a4 = -1.453152027,
        a5 = 1.061405429,
        p = 0.3275911

  const t = 1 / (1 + p * Math.abs(x))
  const y = 1 - (((((a5 * t + a4) * t + a3) * t + a2) * t + a1) * t) * Math.exp(-x * x)
  return sign * y
}
Show the code
Plot.plot({
  title: `PDF of Normal(${normal_mu}, ${normal_var})`,
  marks: [
    Plot.line(normal_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.max(...normal_pdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
Plot.plot({
  title:  `CDF of Normal(${normal_mu}, ${normal_var})`,
  marks: [
    Plot.line(normal_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...normal_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomNormal(normal_mu, normal_var ** (0.5))})).plot()

2.2 Standardisation

Without a computer, the only way to evaluate the probability is using a normal distribution table. However, such table only provides information about the standard normal distribution. Therefore, it becomes important to use standardisation to change any r.v. following normal distribution to the standard normal distribution.

Theorem 2 (Standardisation Normal Technique) If \(Y \sim \mathcal{N}(a, b^2)\), then \(Z = \frac{Y - a}{b} \sim \mathcal{N}(0, 1)\).

We say that \(Y\) has been standardised, and that \(Z\) is the standardised version of \(Y\).

Note that this technique is used not only for finding the probability in the exam with normal table, this can also be used to normalise the training data for a machine learning model1.

3 Gamma Distribution

Definition 3 (Gamma Distribution Pdf) A random variable \(Y\) has the gamma distribution with parameters \(a\) and \(b\) if its pdf is of the form \[ f(y) = \frac{y^{a-1}e^{-y/b}}{b^a \Gamma(a)}, \quad y > 0 \quad (a, b > 0) \]

We write \(Y \sim \text{Gam}(a, b)\).

3.1 Mysterious \(\Gamma(\cdot)\)

\(\Gamma(\cdot)\) here is the gamma function, defined by \(\Gamma(k) = \int_0^\infty t^{k-1}e^{-t}\, dt\).

Some properties:

  • \(\Gamma(k) = (k - 1) \Gamma(k - 1)\) if \(k > 1\)
  • \(\Gamma(k) = (k - 1)!\) if \(k\) is a positive integer (e.g. \(\Gamma (4) = 3! = 6\))
  • \(\Gamma(1/2) = \sqrt{\pi}\)
Show the code
function gamma(z) {
  const g = 7
  const p = [
    0.99999999999980993,
    676.5203681218851,
   -1259.1392167224028,
    771.32342877765313,
   -176.61502916214059,
    12.507343278686905,
   -0.13857109526572012,
    9.9843695780195716e-6,
    1.5056327351493116e-7
  ]

  if (z < 0.5) {
    return Math.PI / (Math.sin(Math.PI * z) * gamma(1 - z))
  } else {
    z -= 1
    let x = p[0]
    for (let i = 1; i < g + 2; i++) {
      x += p[i] / (z + i)
    }
    const t = z + g + 0.5
    return Math.sqrt(2 * Math.PI) * t**(z + 0.5) * Math.exp(-t) * x
  }
}

// Generate values for plotting
gam_xs = Array.from({ length: 200 }, (_, i) => 0.01 + i / 199 * (8 - 0.01))
gamma_data = gam_xs.map(x => ({ x, y: gamma(x) }))

// Plot
Plot.plot({
  title: `Gamma Function Γ(x), from x = 0.01 to ${8}`,
  marks: [
    Plot.line(gamma_data, { x: "x", y: "y", stroke: "purple" }),
    Plot.ruleY([0])
  ],
  x: { label: "x", domain: [0, 8] },
  y: { label: "Γ(x)" },
  width: 600,
  height: 300
})

3.2 Mode of Gamma Distribution

Theorem 3 (Mode of Gamma Distribution) \[ Mode(Y) = \begin{cases} b(a-1) & a \geq 1 \\ 0 & a < 1 \end{cases} \]


3.3 Interactive Widget

Show the code
viewof gamma_a = Inputs.range([0.4, 5], { label: "a", step: 0.1, value: 1 })
viewof gamma_b = Inputs.range([0.1, 5], { label: "b", step: 0.1, value: 1 })

function gammainc_lower(x, a) {
  // Lower regularized incomplete gamma function P(a, x)
  // using a simple series expansion
  let sum = 1 / a
  let value = 1 / a
  for (let n = 1; n < 100; n++) {
    value *= x / (a + n)
    sum += value
    if (value < 1e-8) break
  }
  return sum * Math.exp(-x + a * Math.log(x)) / gamma(a)
}

function gammaCDF(x, alpha, theta) {
  if (x <= 0) return 0
  return gammainc_lower(x / theta, alpha)
}

function gammaPDF(x, a, t) {
  if (x < 1e-6) return a < 1 ? Infinity : 0
  return (1 / (t ** a * gamma(a))) * x ** (a - 1) * Math.exp(-x / t)
}

gamma_xs = [
  ...Array.from({ length: 100 }, (_, i) =>
    1e-6 + Math.exp(Math.log(1e-6) + (i / 100) * Math.log(0.5 / 1e-6))  // log scale from 1e-6 to ~0.5
  ),
  ...Array.from({ length: 100 }, (_, i) => 0.5 + i * 0.1)  // linear from 0.5 to 10.4
]


// PDF of Normal Distribution
gamma_pdf = gamma_xs.map(x => ({
  x,
  y: gammaPDF(x, gamma_a, gamma_b)
}))

// CDF of Normal Distribution using the error function approximation
gamma_cdf = gamma_xs.map(x => ({
  x,
  y: gammaCDF(x, gamma_a, gamma_b)
}))
Show the code
Plot.plot({
  title: `PDF of Gamma(${gamma_a}, ${gamma_b})`,
  marks: [
    Plot.line(gamma_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...gamma_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})
Show the code
Plot.plot({
  title:  `CDF of Gamma(${gamma_a}, ${gamma_b})`,
  marks: [
    Plot.line(gamma_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...gamma_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
// @title: Histogram of Samples
// import { Plot, binX, ruleY, line } from "@observablehq/plot"

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomGamma(gamma_a, gamma_b)})).plot()

3.4 Conclusion on the Gamma Distribution

Now, it is clear that the gamma distribution is very expressive with two model parameters. In fact, we can define many specific distribution by using this gamma distribution by fixing some of the model parameters.

4 The Chi2 Distribution

Being a special case of the gamma distribution.

Definition 4 (Chi-Square Distribution) If \(Y \sim \text{Gam}(n/2, 2)\), we say that \(Y\) has the chi-square distribution with parameter \(n\).

Denote as \(Y \sim \chi^2(n)\).

Definition 5 (Chi-Square Degree of Freedom) \(n\) in the above formulation is the degrees of freedom (DOF).

Theorem 4 (Mode of Chi-Square Distribution) The mode of \(Y\) is \(n - 2\) if \(n \geq 2\), and it is 0 if \(n \leq 2\).


4.1 Interactive Widget

Show the code
viewof chi_dof = Inputs.range([1, 5], { label: "dof", step: 1, value: 1 })

// PDF of Normal Distribution
chi_pdf = gamma_xs.map(x => ({
  x,
  y: gammaPDF(x, chi_dof / 2, 2)
}))

// CDF of Normal Distribution using the error function approximation
chi_cdf = gamma_xs.map(x => ({
  x,
  y: gammaCDF(x, chi_dof / 2, 2)
}))
Show the code
// @title: Gamma PDF 

Plot.plot({
  title: `PDF of Chi(${chi_dof})`,
  marks: [
    Plot.line(chi_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...chi_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})
Show the code
// @title: Normal CDF

Plot.plot({
  title:  `CDF of Chi(${chi_dof})`,
  marks: [
    Plot.line(chi_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...chi_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
// @title: Histogram of Samples

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomGamma(chi_dof / 2, 2)})).plot()

5 Exponential Distribution

Another special case of the gamma distribution

Definition 6 (Exponential Distribution PDF) If \(Y \sim \text{Gam}(1, b)\), then \(Y\) has the exponential distribution with parameter \(b\).

We write \(Y \sim \text{Exp}(b)\) with the corresponding pdf being, \[ f(y) = \frac{1}{b}e^{-y/b}, \quad y > 0 \]

By using Theorem 3, we obtain the following corollary.

Corollary 1 (Mode of Exponential Distribution) \[ Mode(Y) = 0 \]


Now, we can establish the following connection with all the other ones have discovered. \[ \text{Exp}(2) = \text{Gam}(2/2, 2) = \chi^2(2) \]


This distribution is useful for modelling times until failure of components and times between successive arrivals in a queue.


5.1 Interactive Widget

Show the code
viewof exp_b = Inputs.range([0.1, 5], { label: "n", step: 0.1, value: 1 })

// PDF of Normal Distribution
exp_pdf = gamma_xs.map(x => ({
  x,
  y: gammaPDF(x, 1, exp_b)
}))

// CDF of Normal Distribution using the error function approximation
exp_cdf = gamma_xs.map(x => ({
  x,
  y: gammaCDF(x, 1, exp_b)
}))
Show the code
Plot.plot({
  title: `PDF of Exp(${exp_b})`,
  marks: [
    Plot.line(exp_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...exp_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})
Show the code
Plot.plot({
  title:  `CDF of Exp(${exp_b})`,
  marks: [
    Plot.line(exp_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...exp_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
// @title: Histogram of Samples

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomGamma(1, exp_b)})).plot()

5.2 Standard Exponential Distribution

Definition 7 (Standard Exponential Distribution) A special case of the exponential distribution.

If \(Y \sim \text{Exp}(1)\), we say that \(Y\) has the standard exponential distribution.

6 Beta Distribution

Definition 8 (Beta Distribution) A random variable \(Y\) has the beta distribution with parameters \(a\) and \(b\) if its pdf is of the form \[ f(y) = \frac{y^{a-1}(1-y)^{b-1}}{B(a, b)}, \quad o < y < 1 \; (a, b > 0) \]

We write \(Y \sim \text{Beta}(a, b)\).

Here, \(B(a, b) = \frac{\Gamma(a) \Gamma(b)}{\Gamma(a + b)}\) is the beta function.

6.1 Connection with Uniform Distribution

If \(a = b = 1\), then \(f(y) = 1\), \(0 < y < 1\). Thus \(\text{Beta}(1, 1) = U(0, 1)\).

It can be shown that \(Mode(Y) = (a - 1)/(a + b -2)\) if \(a > 1\) and \(b > 1\).


6.2 Interactive Widget

Show the code
viewof beta_alpha = Inputs.range([0.1, 10], { label: "alpha", step: 0.01, value: 1 })
viewof beta_beta = Inputs.range([0.1, 10], { label: "beta", step: 0.01, value: 1 })

// --- More points near edges ---
beta_xs = [
  ...Array.from({ length: 201 }, (_, i) => i / 200)
].filter(x => x <= 1)

// --- Beta PDF ---
function betaPDF(x, a, b) {
  if (x <= 0 || x >= 1) return 0
  const numerator = x ** (a - 1) * (1 - x) ** (b - 1)
  const denominator = gamma(a) * gamma(b) / gamma(a + b)
  return numerator / denominator
}

// Regularized incomplete beta function I_x(a, b)
// This is a simple continued fraction approximation for 0 < x < 1
// Based on the continued fraction form in NR, adapted for moderate values

function betainc(x, a, b) {
  if (x <= 0) return 0
  if (x >= 1) return 1

  const lnBeta = Math.log(gamma(a)) + Math.log(gamma(b)) - Math.log(gamma(a + b))
  const front = Math.exp(
    a * Math.log(x) + b * Math.log(1 - x) - lnBeta
  ) / a

  let f = 1, c = 1, d = 0
  for (let i = 1; i < 100; i++) {
    const m = i / 2
    const numerator = (i % 2 === 1)
      ? (b - m) * x / (a + 2 * m - 1)
      : -((a + m - 1) * (a + b + m - 1) * x) / ((a + 2 * m - 2) * (a + 2 * m - 1))
    d = 1 + numerator * d
    if (Math.abs(d) < 1e-30) d = 1e-30
    d = 1 / d
    c = 1 + numerator / c
    if (Math.abs(c) < 1e-30) c = 1e-30
    const delta = c * d
    f *= delta
    if (Math.abs(delta - 1) < 1e-8) break
  }

  return front * f
}

// --- PDF values ---
beta_pdf = beta_xs.map(x => ({
  x,
  y: betaPDF(x, beta_alpha, beta_beta)
}))

// Compute the CDF points using the approximation
beta_cdf = beta_xs.map(x => ({
  x,
  y: betainc(x, beta_alpha, beta_beta)
}))
Show the code
Plot.plot({
  title: `PDF of Beta(${beta_alpha}, ${beta_beta})`,
  marks: [
    Plot.line(beta_pdf, { x: "x", y: "y" }),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "f(x)", domain: [0, Math.min(5, Math.max(...beta_pdf.map(d => d.y)) * 1.2)] },
  width: 600,
  height: 250
})
Show the code
Plot.plot({
  title: `CDF of Beta(${beta_alpha}, ${beta_beta})`,
  marks: [
    Plot.line(beta_cdf, {x: "x", y: "y"}),
    Plot.ruleY([0])
  ],
  x: { label: "x" },
  y: { label: "F(x)", domain: [0, Math.max(...beta_cdf.map(d => d.y)) * 1.2] },
  width: 600,
  height: 250
})
Show the code
// @title: Histogram of Samples

Plot.rectY({length: 10000}, Plot.binX({y: "count"}, {x: d3.randomBeta(beta_alpha, beta_beta)})).plot()

Footnotes

  1. However, it is more common to use min-max normalisation.↩︎