Regression Analysis

Regression analysis is a set of statistical method to model the relationship between a response variable (Dependent) and one or more predictor variables (Independent). It is a way of statistically sorting out which of those variables does indeed have an impact. Regression analysis is used for three types of applications:
  1. Finding out the effect of independent variables on dependent variable.
  2. Finding out the change in dependent variable with respect to one or more independent variable.
  3. To find out upcoming trends.
Regression algorithms are essential for any Machine Learning problem that involves continuous numbers. It is fall under the family of Supervised Machine Learning algorithms which is a subset of machine learning algorithms. These algorithms may be linear as well as non-linear. We'll discuss them in detail in the following sections.
  1. Simple linear regression
  2. Multiple linear regression
  3. Ordinary Least Squares regression

Simple linear regression machine-learning

Simple linear regression

Simple Linear Regression is a type of Regression algorithms that allows you to summarize and study the relationships between a dependent variable (response variable) and a single independent variable (predictor variable). One variable, denoted by x, is regarded as the predictor independent variable and the other variable, denoted by y, is regarded as the response or dependent variable.

You can use simple linear regression for:

  1. How strong the relationship is between two variables (x and y) .
  2. The value of the dependent variable at a certain value of the independent variable.
A linear regression is where the relationships between your variables (x and y) can be represented by a straight line.
simple linear regression
After looking at the above scatter plot you can understand:
  1. The direction
  2. The strength
  3. The linearity
The equation that describes how y is related to x is known as the regression model . The simple linear regression model is represented by: y = ß0 +ß1x+e
  1. y is the mean or expected value of y for a given value of x.
  2. ß0 is the y-intercept of the regression line.
  3. ß1 is the slope.
  4. x is the independent variable (the variable expect is influencing y).
  5. e is the error of the estimate.

Multiple linear regression

Multiple linear regression is an extension of simple linear regression . It is a statistical technique that uses several independent (predictor) variables to predict the outcome of a dependent(response) variable by fitting a linear equation to observed data. The Multiple linear regression enables analysts to determine the variation of the model and the relative contribution of each independent variable in the total variance. Example:

The height of a child can depend on the height of the mother, the height of the father, nutrition, and environmental factors.

Ordinary Least Squares regression

Ordinary least squares (OLS) regression is a regression analysis that estimates the relationship between one or more independent (predictor) variables and a dependent variable (response). This method estimates the relationship by minimizing the sum of the squares in the difference between the observed and predicted values of the dependent variable configured as a straight line.




Related Topics
  1. Measures of Central Tendency
  2. Measures of the Spread of Data
  3. Relationship Between Variables
  4. Statistical Distributions
  5. Hypothesis Testing
  6. Classification Algorithms
  7. Model Evaluation and Model Validation