> For the complete documentation index, see [llms.txt](https://sejkai.gitbook.io/academic/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sejkai.gitbook.io/academic/ncku-machine-learning/introduction.md).
# Introduction
## Mean $$\mu$$
就是求[平均值](https://www.khanacademy.org/math/ap-statistics/summarizing-quantitative-data-ap/measuring-center-quantitative/v/statistics-intro-mean-median-and-mode)。
$$
\begin{aligned}
&4, 3, 1, 6, 1, 7 \\
&\mu = \frac{4+3+1+6+1+7}{6} = 3.6
\end{aligned}
$$
## Variance $$\sigma$$
代表每個數值的分散程度，越大代表越分散
就是找到[所有值跟 mean 的差的平方，然後全部合起來求平均](https://www.khanacademy.org/math/statistics-probability/summarizing-quantitative-data/variance-standard-deviation-population/v/range-variance-and-standard-deviation-as-measures-of-dispersion)。
$$
\begin{aligned}
&4, 3, 1, 6, 1, 7 \\
&\sigma^2 = \frac{(4-3.6)^2+(3-3.6)^2+(1-3.6)^2+(6-3.6)^2+(1-3.6)^2+(7-3.6)^2}{6} = 5.2
\end{aligned}
$$
linearly variance
$$
if x, y independent\\
VAR\[x+y] = VAR\[x] + VAR\[y]\\
else\\
VAR\[x+y] = VAR\[x] + VAR\[y] + 2COV\[x, y]\\
Covariance = COV\[x, y] = E\[(xi-\mu x)(yi-\mu y)]\\
\mu\_x = E\[x]\\
\mu\_y = E\[y]\\
$$
$$
VAR\[x+x] = VAR\[x] + VAR\[x] + 2COV\[x,x] = 4VAR\[x]
$$
Pearsons correlation = $$\frac{COV\[x, y]}{\sqrt{VAR\[x]VAR\[y]}}$$
Spearman's rank correlation
## Standard Deviation $$\sigma$$
一樣是用來表達數值間的分散程度
但更好的表達**每個數值跟 mean 的平均距離有多遠**
只是把 variance 開根號就好了
$$
\begin{aligned}
&4, 3, 1, 6, 1, 7 \\
&\sigma = \sqrt{5.2} = 2.28
\end{aligned}
$$
## Expectation $$E$$
期望值其實只是[每個數值乘上出現的機率，得出來的總和而已](https://www.khanacademy.org/tag/expected-value)
期望值又可以表達成 $$E(X) = \mu\_X$$
例如一個骰子的機率是
| X | prob(X) |
| - | ------- |
| 1 | 1/6 |
| 2 | 1/6 |
| 3 | 1/6 |
| 4 | 1/6 |
| 5 | 1/6 |
| 6 | 1/6 |
所以期望值為
$$
E(X) = 1 \times \frac{1}{6} + 2 \times \frac{1}{6} + 3 \times \frac{1}{6} + 4 \times \frac{1}{6} + 5 \times \frac{1}{6} + 6 \times \frac{1}{6} = 3.5
$$
* [Expectation of binomial ditribution](https://www.khanacademy.org/math/in-in-grade-12-ncert/in-in-probability-of-events/copy-of-binomial-random-variables/v/expected-value-of-binomial-distribution)
linearly expectation
* expectation (p1)
$$
E\[f(x)] = \sum\_ipx\_if(x\_i)
$$
## Normal Distribution
指的是現實中常見的[常態分布](https://www.khanacademy.org/math/statistics-probability/modeling-distributions-of-data/normal-distributions-library/a/normal-distributions-review)
又稱作**高斯分布 (Gaussian distribution)**
* 是一個 bell 的形狀
* mean = median，且都在 distribution 的中央
* 有大約 68% 的數值在 1 standard deviation of the mean
* 有大約 95% 的數值在 2 standard deviation of the mean
* 有大約 99.7% 的數值在 3 standard deviation of the mean
* 上面三個數值的分布又稱為 **Empirical rule**
* The probability density of the normal distribution is
$$
f(x \mid \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{\frac{(x-\mu)^2}{2\sigma^2}}
$$
### Standard Normal Distribution
**標準常態分布**是 Normal distribution 的一種
他的平均在 0，且 variance = 1
$$\mu = 0, \sigma^2 = 1$$
* The probability density of the standard normal distribution is
$$
\varphi(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}x^2}
$$
## Entropy $$H$$
entropy (p32)
$$
H(p\_1 \cdots p\_k) = \sum\_i p\_i \log\frac{1}{p\_i} = E\[\log\frac{1}{p\_i}]
$$
entropy can be used to predict the least bits needs to be transfer a = 1/2, b = 1/2, h = 1 a = 2/3, b = 1/3, h = .92 => can use less than 1 bit to code (use huffman code)
## Probability
### Likelihood
### Prior Probability
prior odds P(A) = 20% P(-A) = 80% =1/4
### Posterior Probability
posterior odds
$$
\frac{P\[A|S]}{P\[-A|S]} = \frac{P\[A]}{P\[-A]} \frac{P\[S|A]}{P\[S|-A]}
$$
posterior = prior odds \* likelihood
is better than bayes law (P\[S] is not easy to get.)
### Bayes law
* odds form of bayes law (p29)
### Odds ratio
### Bent coin
beta integral with gamma function
$$
\forall k Prob\[k success] = \frac{1}{n+1}
$$
$$
F\_a + F\_b = F
$$
$$
\frac{F\_a!F\_b!}{(F\_a + F\_b + 1)!}
\=\frac{1}{(N+1)}\frac{F\_a!(F-F\_a)!}{F!} = \frac{1}{(F+1)\binom{F}{F\_a}}
$$
### 3.5 answer
$$
P\[P\_a\mid aba] = \frac{P\[P\_a]\times P\[aba|P\_a]}{P\[aba]}
$$
$$
P\[aba|P\_a] = P\_a^2(1-P\_a)
$$
$$
P\[aba] = \frac{1}{(3+1)\binom{3}{2}}
$$
$$
P\[P\_a\mid aba] =
\frac{P\_a(P\_a^2(1-P\_a))}{\frac{1}{12}}
$$
Course goal: Full understanding of inferrence of Pa
* binomial distribution inferrence
* chapter 3
## Binomial Distribution
## Maximum Likelihood
### gaussian mixture model
### Bayesian inference
### Belief update
### Maximum Likelihood Estimation
$$
\text{Given } f(x) = N(\mu, \sigma^2) \text{ with fixed } \sigma^2\\
\text{data }x\_1 \cdots x\_n\\
\text{What is the maximum likelihood estimation of }\mu
$$
## Random Variables
$$
I(X, Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)
$$
if x, y independent
$$
I(X, Y) = 0
$$
if X determines Y
$$
H(X|Y) = 0, I(X, Y) = H(X) = H(Y)
$$
hw : p-val = prob of data given hypothesis
### P-value
## Clustering
### k nearest neighbors
\=> K-NN NEAREST NEIGHBORS
### k-means clustering
\=> K-MEANS CLUSTERING
### soft k-mean clustering
p289
### Curse of demensionality
### Demension Reduction
feature selection, PCA, assume features are independent
## Beta Binomial Reasoning
* Conjugate
$$
\begin{aligned}
\text{prior} + \text{likelihood} &= \text{posterior form} \\
P\[P\_H] &\propto P\_H^a(1-P\_H)^b
\end{aligned}
$$
* 這個的微分為 beta integral
$$
Beta(x, a, b) \Rightarrow \frac{x^{a-1}(1-x)^{b-1}}{B(a, b)}= \frac{\Gamma(a)\Gamma(b)x^a(1-x)^b}{\Gamma(a+b)}
$$
* 當 a = 1, b = 1 時為 uniform prior
$$
\frac{x^0(1-x)^0\Gamma(1)\Gamma(1)}{\Gamma(2)} = 1 \mid \text{ where } \Gamma(i) = (i-1)!
$$
* beta distribution 不夠表達
* 可以加上 mixture model
* murphy p43
## Naive Bayes