Linear Regression

Linear Regression

Regression is predictive modeling. It is a statistical method used in finance, investment, and other disciplines that attempts to determine the strength and character of the relationship between one dependent variable (usually denoted by Y) and a series of other variables (known as independent variables).
Regression helps investment and financial managers value assets and understand the relationships between variables, such as commodity prices and the stocks of businesses dealing in those commodities.

Description

Linear regression is a statistical and ML method to establish a linear relationship between the input variables (x) and a single output variable (y). The value of y can be calculated from a linear combination of variables x.

Why to use

To perform the Predictive Modeling for the dependent variable.

When to useWhen you want to predict a value depending upon single or multiple independent variables.When not to useOn textual data.

Prerequisites

There should not be any missing values in the data.
The output variable (y)  must be a continuous data type.

  • Linearity: The relationship between x and y is linear.
  • Homoscedasticity: Residual variance is the same for any value of x.
  • Independence: There is no correlation in the independent variables.
  • Normality: Residuals must be normally distributed.
Input
  • A single numeric Dependent variable
  • A set of Independent variables, a combination of numeric and categorical variables
OutputPredicted value.
Statistical Methods used
  • Regression
  • Mean
  • ANOVA
  • R square
  • Adjusted R square
  • Residual
  • Coefficient
  • Significance level (alpha)
  • Durbin Watson – Auto Correlation Factor
  • Variance Inflation Factor
  • p value
  • W Statistic
Limitations
  • It cannot be used on textual data.
  • It is sensitive to Outliers.
  • It is limited to linear



The equation for linear regression is, y=mx+c

  • y is the dependent variable (output)
  • m is the slope or weight
  • x is the independent variable (input)
  • c is the constant or intercept

The output variable is the dependent variable or scalar response, while the input variables are independent or explanatory. The output variable (y) must be a continuous data type.
There are two types of linear regression, simple and multiple. In simple linear regression, there is only one input variable (x), while in multiple linear regression, there are multiple input variables (x1,x2,x3,x4). As input variables increase, slopes or weights will also increase.
Linear regression aims to determine which predictors are significant in predicting the output variable, their efficiency in predicting the output variable, and how they impact the output variable.