Without a computer, the only way to evaluate the probability is using a normal distribution table. However, such table only provides information about the standard normal distribution. Therefore, it becomes important to use standardisation to change any r.v. following normal distribution to the standard normal distribution.
Theorem 2 (Standardisation Normal Technique) If \(Y \sim \mathcal{N}(a, b^2)\), then \(Z = \frac{Y - a}{b} \sim \mathcal{N}(0, 1)\).
We say that \(Y\) has been standardised, and that \(Z\) is the standardised version of \(Y\).
Note that this technique is used not only for finding the probability in the exam with normal table, this can also be used to normalise the training data for a machine learning model1.
3 Gamma Distribution
Definition 3 (Gamma Distribution Pdf) A random variable \(Y\) has the gamma distribution with parameters \(a\) and \(b\) if its pdf is of the form \[
f(y) = \frac{y^{a-1}e^{-y/b}}{b^a \Gamma(a)}, \quad y > 0 \quad (a, b > 0)
\]
We write \(Y \sim \text{Gam}(a, b)\).
3.1 Mysterious \(\Gamma(\cdot)\)
\(\Gamma(\cdot)\) here is the gamma function, defined by \(\Gamma(k) = \int_0^\infty t^{k-1}e^{-t}\, dt\).
\(\Gamma(k) = (k - 1)!\) if \(k\) is a positive integer (e.g. \(\Gamma (4) = 3! = 6\))
\(\Gamma(1/2) = \sqrt{\pi}\)
Show the code
functiongamma(z) {const g =7const p = [0.99999999999980993,676.5203681218851,-1259.1392167224028,771.32342877765313,-176.61502916214059,12.507343278686905,-0.13857109526572012,9.9843695780195716e-6,1.5056327351493116e-7 ]if (z <0.5) {returnMath.PI/ (Math.sin(Math.PI* z) *gamma(1- z)) } else { z -=1let x = p[0]for (let i =1; i < g +2; i++) { x += p[i] / (z + i) }const t = z + g +0.5returnMath.sqrt(2*Math.PI) * t**(z +0.5) *Math.exp(-t) * x }}// Generate values for plottinggam_xs =Array.from({ length:200 }, (_, i) =>0.01+ i /199* (8-0.01))gamma_data = gam_xs.map(x => ({ x,y:gamma(x) }))// PlotPlot.plot({title:`Gamma Function Γ(x), from x = 0.01 to ${8}`,marks: [ Plot.line(gamma_data, { x:"x",y:"y",stroke:"purple" }), Plot.ruleY([0]) ],x: { label:"x",domain: [0,8] },y: { label:"Γ(x)" },width:600,height:300})
3.2 Mode of Gamma Distribution
Theorem 3 (Mode of Gamma Distribution)\[
Mode(Y) = \begin{cases}
b(a-1) & a \geq 1 \\
0 & a < 1
\end{cases}
\]
3.3 Interactive Widget
Show the code
viewof gamma_a = Inputs.range([0.4,5], { label:"a",step:0.1,value:1 })viewof gamma_b = Inputs.range([0.1,5], { label:"b",step:0.1,value:1 })functiongammainc_lower(x, a) {// Lower regularized incomplete gamma function P(a, x)// using a simple series expansionlet sum =1/ alet value =1/ afor (let n =1; n <100; n++) { value *= x / (a + n) sum += valueif (value <1e-8) break }return sum *Math.exp(-x + a *Math.log(x)) /gamma(a)}functiongammaCDF(x, alpha, theta) {if (x <=0) return0returngammainc_lower(x / theta, alpha)}functiongammaPDF(x, a, t) {if (x <1e-6) return a <1?Infinity:0return (1/ (t ** a *gamma(a))) * x ** (a -1) *Math.exp(-x / t)}gamma_xs = [...Array.from({ length:100 }, (_, i) =>1e-6+Math.exp(Math.log(1e-6) + (i /100) *Math.log(0.5/1e-6)) // log scale from 1e-6 to ~0.5 ),...Array.from({ length:100 }, (_, i) =>0.5+ i *0.1) // linear from 0.5 to 10.4]// PDF of Normal Distributiongamma_pdf = gamma_xs.map(x => ({ x,y:gammaPDF(x, gamma_a, gamma_b)}))// CDF of Normal Distribution using the error function approximationgamma_cdf = gamma_xs.map(x => ({ x,y:gammaCDF(x, gamma_a, gamma_b)}))
// @title: Histogram of Samples// import { Plot, binX, ruleY, line } from "@observablehq/plot"Plot.rectY({length:10000}, Plot.binX({y:"count"}, {x: d3.randomGamma(gamma_a, gamma_b)})).plot()
3.4 Conclusion on the Gamma Distribution
Now, it is clear that the gamma distribution is very expressive with two model parameters. In fact, we can define many specific distribution by using this gamma distribution by fixing some of the model parameters.
4 The Chi2 Distribution
Being a special case of the gamma distribution.
Definition 4 (Chi-Square Distribution) If \(Y \sim \text{Gam}(n/2, 2)\), we say that \(Y\) has the chi-square distribution with parameter \(n\).
Denote as \(Y \sim \chi^2(n)\).
Definition 5 (Chi-Square Degree of Freedom)\(n\) in the above formulation is the degrees of freedom (DOF).
Theorem 4 (Mode of Chi-Square Distribution) The mode of \(Y\) is \(n - 2\) if \(n \geq 2\), and it is 0 if \(n \leq 2\).
4.1 Interactive Widget
Show the code
viewof chi_dof = Inputs.range([1,5], { label:"dof",step:1,value:1 })// PDF of Normal Distributionchi_pdf = gamma_xs.map(x => ({ x,y:gammaPDF(x, chi_dof /2,2)}))// CDF of Normal Distribution using the error function approximationchi_cdf = gamma_xs.map(x => ({ x,y:gammaCDF(x, chi_dof /2,2)}))
// @title: Histogram of SamplesPlot.rectY({length:10000}, Plot.binX({y:"count"}, {x: d3.randomGamma(chi_dof /2,2)})).plot()
5 Exponential Distribution
Another special case of the gamma distribution
Definition 6 (Exponential Distribution PDF) If \(Y \sim \text{Gam}(1, b)\), then \(Y\) has the exponential distribution with parameter \(b\).
We write \(Y \sim \text{Exp}(b)\) with the corresponding pdf being, \[
f(y) = \frac{1}{b}e^{-y/b}, \quad y > 0
\]
By using Theorem 3, we obtain the following corollary.
Corollary 1 (Mode of Exponential Distribution)\[
Mode(Y) = 0
\]
Now, we can establish the following connection with all the other ones have discovered. \[
\text{Exp}(2) = \text{Gam}(2/2, 2) = \chi^2(2)
\]
This distribution is useful for modelling times until failure of components and times between successive arrivals in a queue.
5.1 Interactive Widget
Show the code
viewof exp_b = Inputs.range([0.1,5], { label:"n",step:0.1,value:1 })// PDF of Normal Distributionexp_pdf = gamma_xs.map(x => ({ x,y:gammaPDF(x,1, exp_b)}))// CDF of Normal Distribution using the error function approximationexp_cdf = gamma_xs.map(x => ({ x,y:gammaCDF(x,1, exp_b)}))
// @title: Histogram of SamplesPlot.rectY({length:10000}, Plot.binX({y:"count"}, {x: d3.randomGamma(1, exp_b)})).plot()
5.2 Standard Exponential Distribution
Definition 7 (Standard Exponential Distribution) A special case of the exponential distribution.
If \(Y \sim \text{Exp}(1)\), we say that \(Y\) has the standard exponential distribution.
6 Beta Distribution
Definition 8 (Beta Distribution) A random variable \(Y\) has the beta distribution with parameters \(a\) and \(b\) if its pdf is of the form \[
f(y) = \frac{y^{a-1}(1-y)^{b-1}}{B(a, b)}, \quad o < y < 1 \; (a, b > 0)
\]
We write \(Y \sim \text{Beta}(a, b)\).
Here, \(B(a, b) = \frac{\Gamma(a) \Gamma(b)}{\Gamma(a + b)}\) is the beta function.
6.1 Connection with Uniform Distribution
If \(a = b = 1\), then \(f(y) = 1\), \(0 < y < 1\). Thus \(\text{Beta}(1, 1) = U(0, 1)\).
It can be shown that \(Mode(Y) = (a - 1)/(a + b -2)\) if \(a > 1\) and \(b > 1\).
6.2 Interactive Widget
Show the code
viewof beta_alpha = Inputs.range([0.1,10], { label:"alpha",step:0.01,value:1 })viewof beta_beta = Inputs.range([0.1,10], { label:"beta",step:0.01,value:1 })// --- More points near edges ---beta_xs = [...Array.from({ length:201 }, (_, i) => i /200)].filter(x => x <=1)// --- Beta PDF ---functionbetaPDF(x, a, b) {if (x <=0|| x >=1) return0const numerator = x ** (a -1) * (1- x) ** (b -1)const denominator =gamma(a) *gamma(b) /gamma(a + b)return numerator / denominator}// Regularized incomplete beta function I_x(a, b)// This is a simple continued fraction approximation for 0 < x < 1// Based on the continued fraction form in NR, adapted for moderate valuesfunctionbetainc(x, a, b) {if (x <=0) return0if (x >=1) return1const lnBeta =Math.log(gamma(a)) +Math.log(gamma(b)) -Math.log(gamma(a + b))const front =Math.exp( a *Math.log(x) + b *Math.log(1- x) - lnBeta ) / alet f =1, c =1, d =0for (let i =1; i <100; i++) {const m = i /2const numerator = (i %2===1)? (b - m) * x / (a +2* m -1):-((a + m -1) * (a + b + m -1) * x) / ((a +2* m -2) * (a +2* m -1)) d =1+ numerator * dif (Math.abs(d) <1e-30) d =1e-30 d =1/ d c =1+ numerator / cif (Math.abs(c) <1e-30) c =1e-30const delta = c * d f *= deltaif (Math.abs(delta -1) <1e-8) break }return front * f}// --- PDF values ---beta_pdf = beta_xs.map(x => ({ x,y:betaPDF(x, beta_alpha, beta_beta)}))// Compute the CDF points using the approximationbeta_cdf = beta_xs.map(x => ({ x,y:betainc(x, beta_alpha, beta_beta)}))