0-Basic question
Linear regression considers $n$ samples $\{\mathbf{x}_1,\mathbf{x}_2,\cdots,\mathbf {x}_n\}$, each sample corresponds to $m+1$ dimensional feature$\mathbf{x}_i=\{x_{i0},x_{i1},x_{i2},\cdots,x_{im }\}$ (where $x_{i0}=1$), denoted as $\mathbf{X}_{n\times (m+1)}$, and each sample has a corresponding result output, denoted as $\mathbf{y}_{n}$. Need to find the parameter $\mathbf{w}_{m+1}$, construct the following model,
$$
\mathbf{y}=\mathbf{X}\mathbf{w}= \begin{pmatrix}\mathbf{x}_0\\ \mathbf{x}_1\\ \vdots\\ \mathbf{x}_m\end{pmatrix}\begin{pmatrix}w_0\ w_1\ \cdots\ w_m\ end{pmatrix},
$$
The error between the fitting result and the real target can be expressed as follows,
$$
J(\mathbf{ w})=\frac{1}{2}\left(\mathbf{X}\mathbf{w}-\mathbf{y}\right)^T\left(\mathbf{X}\mathbf{w}- \mathbf{y}\right),
$$
Among them, the parameter factor $\frac{1}{2}$ is added for the convenience of derivation.
1-Solving method
1.1-Least square method
1.2-MLE
1.3-Gradient descent method
< h3>2-Extension of linear regression
3-Regularization
3.1-L1 regularization (Lasso)
3.2-L2 regularization (Ridge, Ridge Return)
4-Reference Materials
https://www.cnblogs.com/huangyc/p/9782821.html
https://blog. csdn.net/ccnt_2012/article/details/81127117