probability distribution

Probability Distribution

A probability distributino over a discrete variable $X = {x_{1}, \dots, x_{n}}$ is a function $P : X \mapsto [0, 1] \subset R^{+}$ with $\sum_{i} P (x_{i}) = 1$ .

A probability distribution $p$ is a function that provides the probabilities of occurrence of different possible outcomes in an experiment / of a random variable $X$ at a particular value $x$ , given the parameters $θ$ .
In simpler terms, it is a model of the likelihood of an event/outcome/value, given some parameters that characterize a system:

X \sim P (X = x ∣ θ)

The sum of the probabilities in the distribution must be equal to 1:

i \sum P (X = x_{i}) = 1

Or for the continuous case (CDF, the integral over a PDF):

\int_{- \infty}^{\infty} f_{X} (x) d x = 1

rework this note after done with measure theory perspective (and delete it / merge it into distribution)

Wrong:
Probability distributions are classified into two main types:
probability density function (pdf) $f_{X} (x)$ for continuous variables
probability mass function (pmf) $P (X = x)$ for discrete variables.

Formally:

P_{X} (A) = {\sum_{x \in A} P (X = x) \int_{A} f_{X} (x) d x if X is discrete if X is continuous for any event A \subseteq R

So we get the probability density function/probability density function by taking the derivative of the cumulative distribution function:

Some common distributions:
bernoulli distributions model binary outcomes.
binomial distributions are used to model the number of successes in a fixed number of trials.
normal distributions arise in many natural processes; as you add up many random variables, their sum tends to follow a normal distribution, even if they don’t follow a normal distribution individually (Central Limit Theorem).
poisson distributions are used to model the number of events occurring in a fixed interval of time or space; rare events.
exponential distributions are model the probability of waiting times between poisson events.
multinomial distribution is a generalization of the binomial distribution to more than two categories.
geometric distribution
cauchy distribution
pareto distribution
…

machine learning is essentiall learning unknown probability distributions from data.

References

normal dist statquest
article

https://github.com/google-deepmind/distrax/tree/main/distrax/_src/distributions

Graph View

probability distribution

References

Backlinks