Regression AnalysisRegression analysis is a set of statistical method to model the relationship between a response variable (Dependent) and one or more predictor variables (Independent). It is a way of statistically sorting out which of those variables does indeed have an impact. Regression analysis is used for three types of applications:
- Finding out the effect of independent variables on dependent variable.
- Finding out the change in dependent variable with respect to one or more independent variable.
- To find out upcoming trends.
- Simple linear regression
- Multiple linear regression
- Ordinary Least Squares regression
Simple linear regressionSimple Linear Regression is a type of Regression algorithms that allows you to summarize and study the relationships between a dependent variable (response variable) and a single independent variable (predictor variable). One variable, denoted by x, is regarded as the predictor independent variable and the other variable, denoted by y, is regarded as the response or dependent variable.
You can use simple linear regression for:
- How strong the relationship is between two variables (x and y) .
- The value of the dependent variable at a certain value of the independent variable.
After looking at the above scatter plot you can understand:
- The direction
- The strength
- The linearity
- y is the mean or expected value of y for a given value of x.
- ß0 is the y-intercept of the regression line.
- ß1 is the slope.
- x is the independent variable (the variable expect is influencing y).
- e is the error of the estimate.
Multiple linear regressionMultiple linear regression is an extension of simple linear regression . It is a statistical technique that uses several independent (predictor) variables to predict the outcome of a dependent(response) variable by fitting a linear equation to observed data. The Multiple linear regression enables analysts to determine the variation of the model and the relative contribution of each independent variable in the total variance. Example:
The height of a child can depend on the height of the mother, the height of the father, nutrition, and environmental factors.
Ordinary Least Squares regressionOrdinary least squares (OLS) regression is a regression analysis that estimates the relationship between one or more independent (predictor) variables and a dependent variable (response). This method estimates the relationship by minimizing the sum of the squares in the difference between the observed and predicted values of the dependent variable configured as a straight line.