instructions:Project
1. Abstract
2. Table of contents
3. introduction
4. Introduction of regression analysis.
a- Linear regression analysis: 1- simple linear regression (definition, formula), 2- multiple linear regression (definition, algorithms, formula),
b- Ordinary Least Squares OSL (definition, algorithms, goal of it, details, simple example, sum of squares, understand the error).
6- What regularization (definition, details, algorithms)
7- Bias and variance tradeoff ( error due to bias, error due to variance, details)
8- Shrinkage estimators (definitions, formulas, Degrees of freedom AIC, BIC, details)
9- multicollinearity(definition, details)
10- Variance inflation factor (VIF)(definition, algorithms, details)
11- Ridge regression (definition, algorithms, details, formula, ridge trace (explain, simple example), ridge bias constant , scale in ridge regression (details))
12- LASSO (definition, algorithms, details, formula)
13- The differences between ridge regression and LASSO (Constrained form, details)
14- References.

References:
SAS System for Regression: Third Edition
By Rudolf J. Freud, Ph.D., Ramon C. Littell, Ph.D.

http://www.holehouse.org/mlclass/07_Regularization.html
http://www.eecs.berkeley.edu/~russell/classes/cs194/f11/lectures/CS194%20Fall%202011%20Lecture%2004.pdf

https://www.coursera.org/learn/machine-learning/lecture/4VDlf/regularization-and-bias-variance

http://statweb.stanford.edu/~tibs/ftp/lassotalk.pdf
http://arxiv.org/pdf/math/0406456.pdf