Decision Tree Regression

Decision Tree Regression

Decision Tree Regression

Description

Decision Tree Regressor builds a regression model in the form of a tree structure where each leaf node represented a class or a decision.

Why to use

When you want to predict a value depending on single or multiple independent variables. 

When to use

  • High accuracy is required
  • Reduced overfitting is required
  • Large Datasets

When not to use

  • On Small Datasets
  • When interpretability not needed
  • For Time-Series Forecasting

Prerequisites

If the data contains any missing values, use Missing Value Imputation before proceeding with Decision Tree Regression. 

Input

Any continuous large dataset

Output

  • Regression Key Performance Indicators (KPIs)
  • Residual vs Input
  • Y vs Standardized Residuals
  • Residuals Probability Chart

Statistical Method Used

  • Splitting Criteria
  • Variance Reduction
  • Root Mean Square Error
  • Mean Absolute Error
  • Maximum Depth and Minimum Samples Per leaf
  • Information Gain and Gini Impurity
  • Standard Error
  • Interpolation and Prediction
  • Bootstrap Sampling

Limitations

  • Overfitting of the data
  • High Variance
  • Instability
  • Not good for small datasets
  • Time and series limitations


    • Related Articles

    • Decision Tree

      Decision Tree Description Decision Tree builds a classification model in the form of a tree structure where each leaf node represented a class or a decision. Why to use Data Classification When to use When the data is categorized into Boolean values. ...

    • Random Forest Regression

      Random Forest Regression Description Random Forest Regression is an ensemble learning method that combines multiple decision trees to create a powerful predictive model for continuous target variables. It utilizes random feature selection to improve ...

    • Extreme Gradient Boost Regression (XGBoost)

      Extreme Gradient Boost Regression (XGBoost) Description Extreme Gradient Boost (XGBoost) Regression is a Decision tree-based ensemble algorithm that uses a gradient boosting framework. Why to use Predictive Modeling When to use When high execution ...

    • Regression

      Regression is predictive modeling. It is a statistical method, used in finance, investment, and other disciplines, that attempts to determine the strength and character of the relationship between one dependent variable (usually denoted by Y) and a ...

    • Support Vector Machine In Regression

      Support Vector Machine In Regression Description The Support Vector Machine (SVM) approach for regression is a machine learning technique that seeks to identify a hyperplane that closely resembles the regression function. It reduces deviations ...