Limit theorems

Let us say, That there is some Huge random variable, that is very complicated and is not budging to be fit with any simple distribution and so on.. call it $X$, Then, Let us say we want to get the mean $E[X]$, without knowing the distribution $p_X(\cdot )$, then what we can do, hopefully, is to take samples $x_1,x_2,\dots x_n$ and hope that $\underset{n\to \mathrm{\infty }}{lim}\sum _{i=1}^m\frac{x_i}{n}=E[X]$.

Markov's inequality.

Let $X$ be a random variable that takes non-negative values. Then, $X$ can be partitioned as $X_{\ge a}$ and as $X_{<a}$ where one random variable is $X$ when it takes values bigger than or equal to $a$ and the other, the remaining part, for some constant $a$.

Using the total expectation theorem, $E[X]=E[X_{\ge a}]+E[X_{<a}]$, since expected value of any postiive (read non neagtive) random variable is positive, $E[X_{<a}]\ge 0$. Hence $$E[X] \ge \sum {x \ge a}xp(x) \ge a \sum_{x\ge a}p_{X}(x)$$

So the markov inequality states that for any constant $a$ in the range of a non negative random variable $X$, $$E[X] \ge a P(X\ge a)$$.

Applying the Markov inequality on $(X-E[X]{)}^2$ (on variance essentially)

$var(X)\ge a^{2}P(|X-\mu |\ge a)$ (where $\mu =E[X]$). (chebyshev inequality).

Now, these inequalities hold on continuous (non negative) random variables as well, just using integrals instead of sums.

using $\mu =E[X]$ and ${\sigma }^2=E[(X-\mu {)}^2]$, We have for any $a$, $$aP(X \ge a) \le \mu$$