Linear Regression is located under Machine Learning (  ) in Regression, in the task pane on the left. Use the drag-and-drop method to use the algorithm in the canvas. Click the algorithm to view and select different properties for analysis. Refer to Properties of Linear Regression.

Properties of Linear Regression

The available properties of Linear Regression are shown in the figure below.

The table below describes the different fields present in the properties of Linear Regression.

Stepwise Regression Methods

You can choose either of the following iterative regression methods in Machine Learning.

• Forward Selection
  • It starts with no variable in the model.
  • The system adds one variable in the model and checks whether the model is statistically significant.
  • If the model is statistically significant then the variable is added to the model.
  • The above steps are repeated till the system gets optimal results.
• Backward Elimination
  • It starts with a set of independent variables.
  • The system removes one variable from the model.
  • Then checks whether the model is statistically significant. If the model is not statistically significant then the variable is removed from the model.
  • The above steps are repeated till the system gets optimal results.
• Bidirectional Selection
  • It is a combination of the Forward selection and Backward elimination methods.
  • It adds and removes the independent variable combinations. Checks if the model is statistically significant.
  • Repeat the steps till it gets all the independent variables between the significant level.

Example of Linear Regression

The Linear Regression is applied to a Used credit card dataset in the example below. We select Income, Limit, Cards, Education, Age, and Balance as the independent variables and Rating as the dependent variable. The result of the XGBoost Regression is displayed in the figure below.

The Result page of Linear Regression is shown below.

The result page is segregated into the following sections.

1. Key Performance Indicator (KPI)
2. Regression Statistics
3. Graphical representation

Section 1: Key Performance Indicator (KPI)
The figure below displays various KPIs calculated in Linear Regression.

The table below describes multiple (KPIs) on the result page.

Section 2: Regression Statistics
Regression Statistics consists of ANOVA, Performance Metrices, Coefficient Summary, Durbin Watson Test - Auto Correlation, Variance Inflation Factor, and Shapiro-Wilk Normality Test for Residuals for each of the selected independent (predictor) variables.
The Observation displays the total number of rows considered in Linear Regression.

The Performance Metrices consist of all KPI Details.
The ANOVA test calculates the difference in independent variable means. It helps you distinguish the difference in the behavior of the independent variable. To know more about ANOVA, click here.
You see values like Degrees of Freedom (DF), Sum of Squares (SS), Mean Sum of Squares (MS), F statistic of ANOVA, and Significance F.
DF represents The number of independent values that can differ freely within the constraints imposed on them.
SS is the sum of the square of the variations. Variation is the difference (or spread) of each value from the mean.
MS is the value obtained by dividing the Sum of Squares by Degrees of Freedom
F statistic of ANOVA indicates whether the regression model provides a better fit than a model without independent variables.

The Coefficient Summary displays the coefficient correlation between a dependent variable and each independent variable. If the coefficient of correlation is less than zero then there is a negative correlation between the dependent and an independent variable. In this example, there is a negative correlation between Age and KM.
It also calculates the p value based on the t statistic. The null hypothesis is rejected if the p value is less than level of significance (alpha). In the given example, the p value is less than level of significance (alpha).

The Durbin Watson Test determines autocorrelation between the residuals. If the test value is less than 2 then there is a positive correlation in the sample.
To find out more details about the Durbin Watson test, Click here

The Variance Inflation Factor is used to detect multicollinearity in regression analysis. VIF is calculated by selecting one predictor and regressing it against all other predictors in the model.
The value of VIF is equal to or greater than 1. If VIF = 1, the predictors are not correlated. If VIF is between 1 and 5, they are moderately correlated. For any value greater than 5, the predictors are heavily correlated. The higher the value of VIF, the lower the reliability of the regression results, and ultimately the model.

The Shapiro-Wilk Normality Test for Residuals test checks if the input data comes from a normal distribution. If the p value is less than 0.05 then reject the null hypothesis. In the given example, the p value is less than 0.05; hence the null hypothesis is rejected. To find out more details about Shapiro-Wilk Normality Test, Click here

Section 3: Graphical representation

1. Residual vs. Input graph:

• If this graph creates a pattern, then the sample is improper.
• The graph is plotted with the Independent variable on the x-axis and Residual points on the y-axis.
• On the right side, there is an independent variable dropdown. In this example, independent variables like Age, KM, HP, and CC appear in the dropdown.
• The input variable is the same as in the dropdown. In the figure below, the input variable is Age, the same as in the dropdown.
• If this graph follows a pattern then the data is not normally distributed.
  
• 
  

2. The Y vs. Standardized Residuals graph plots the dependent variable, Price, on the x-axis and the Standardized Residual on the y-axis.

3. The Residuals Probability Plots the Independent variable on the x-axis and standardized Residuals on the y-axis. It helps to identify whether the error terms are normally distributed.

4. The Best Features displays the order of the Independent variable that provides the best result of R2. You have performed linear regression with the backward compatibility method in this example. The best sequence of the independent variable is Age, KM, HP, and CC. This order is based on the adjusted R2.