Ridge Regression

Ridge Regression

Ridge Regression

Description

Predict and analyze data points as output for multiple regression data that suffer from multicollinearity by controlling the magnitude of coefficients to avoid over-fitting.

Why to use

Predictive Modeling

When to use

To regularize the regression, if the Sum of Squared Residuals is too high or too low.

When not to use

On Textual data.

Prerequisites

  • If the data contains any missing values, use Missing Value Imputation before proceeding with Ridge Regression.
  • If the input variable is of categorical type, use Label Encoder.
  • The output variable must be a continuous data type.
  • Linearity – The relationship between the dependent and independent variables is linear.
  • Independence – The variables should be independent of each other.
  • Normality – The variables should be normally distributed.
  • The Dependent variable (Y) vs. Residuals plot must not follow a pattern.
  • The errors should be normally distributed.

Input

Any continuous data

Output

The predicted value of the dependent variables.

Statistical Methods used

  • Fit Intercept
  • Dimensionality Reduction

Limitations

It cannot be used on textual data.


Regularization techniques are used to create simpler models from a dataset containing a considerably large number of features. Regularization solves the problem of over-fitting to a great extent and helps in feature selection.

Initially, L1 regularization (Lasso Regression) reduces the number of features by decreasing the coefficients of less important features to zero. After that, the L2 regularization, also called the Ridge Regression, introduces a penalty term to further reduce the magnitude of the remaining features' coefficients. The addition of penalty decreases the difference between the actual and the predicted observations.

Thus, Ridge regression solves the problem of multicollinearity in linear regression. Multicollinearity results when independent variables in a regression model are found to be correlated, and this can have a negative impact on the model fitting and interpretation of results.

Hence, when the magnitude of coefficients is pushed close to zero, the models work better on new datasets and are better optimized for prediction.

    • Related Articles

    • Regression

      Regression is predictive modeling. It is a statistical method, used in finance, investment, and other disciplines, that attempts to determine the strength and character of the relationship between one dependent variable (usually denoted by Y) and a ...

    • Poisson Regression

      Poisson Regression Description Poisson Regression is a type of linear regression used to model the countable data. Why to use For regression analysis of count data When to use For numerical variables When not to use For textual variables ...

    • Polynomial Regression

      Polynomial Regression Description Polynomial Regression is a supervised learning method in which the relationship between the independent and dependent variables is modeled as an nth degree polynomial. Why to use Predictive Modeling When to use When ...

    • Lasso Regression

      Lasso Regression Description Lasso Regression is used to penalize the regression method to select a subset of variables by imposing a constraint on model parameters. Why to use Predictive Modeling When to use For variables having high ...

    • Linear Regression

      Regression is predictive modeling. It is a statistical method used in finance, investment, and other disciplines that attempts to determine the strength and character of the relationship between one dependent variable (usually denoted by Y) and a ...