An Analysis of Variables’ Coefficients Effect on the ML Regression Models’ Accuracy

An Analysis of Variables’ Coefficients Effect on the ML Regression Models’ AccuracyR-squared (R²), Mean Squared Error (MSE), Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) are the most commonly used metrics to measure accuracy for continuous variables. In this post, we will observe the coefficient of variables (CoV) effect on the MAE, MSE, R², and accuracy. We will apply the same linear regression to 4 different data which has variables with different coefficients to explain how and why the MSE, MAE, R², and Accuracy are changing. First, while we keep the MSE and MAE fixed, we will observe the R² and accuracy with the change of coefficient of variables. Secondly, while we keep the R², and accuracy fixed, we will observe the MSE and MAE with the change of coefficient of variables.1. Constant MSE and Increasing R²We will create 4 different data (D1, D2, D3, D4) with different coefficient variables. First, while we keep the MSE and MAE fixed for D1, D2, D3 and D4, while the coefficient of variables values will be 0.1, 0.5, 1, 10 for D1, D2, D3 and D4 respectively.Data1 (D1)=0.1+0.1*X_1+0.1*X_2+0.1*X_3+(Random Error)While the constant number and coefficient of variables (CoV) are 0.1, we will add the random error which has the coefficient of error(CoE)=0.1. We will see how accuracy and R² will be affected with CoE=0.1/CoV=0.1 rate.Data2 (D2)=0.5+0.5*X_1+0.5*X_2+0.5*X_3+(Random Error)While the constant number and CoV are 0.5, and we will add the random error which has the CoE=0.1. Observation for the accuracy and R² with affect of the CoE=0.1/CoV=0.5 rate.Data3 (D3)=1+1*X_1+1*X_2+1*X_3+(Random Error)While the constant number and coefficient are 1, we will add the random error which has the CoE= 0.1.Data4 (D4)=10+10∗X1+10∗X2+10∗X3+(RandomError)Metrics Table:We kept the error fixed for D1, D2, D3, and D4, while the CoV values are 0.1, 0.5, 1, 10 for D1, D2, D3, and D4 respectively. We have applied the 0.1/0.1 (CoE/CoV) rate for D1 and the 0.1/10 (CoE/CoV) rate for D4. We have obtained constant MSE and MAE due to keeping fixed error rates as 0.1 for all the D1, D2, D3, and D4. On the other hand, CoE/CoV rate is changing from 1 to 0.01 for D1 to D4. The results show that the CoE/CoV rate is directly proportional with effect on the accuracy. Due to the directly proportional effect of CoE/CoV rate, the R² and accuracy are increasing from D1 to D4.2. Constant R² and Increasing MSEIn this part, We will again create 4 different data (D1, D2, D3, D4). We will apply CoE/CoV rate as 1 for the D1, D2, D3, D4. That is mean; The CoE and CoV values will be 0.1, 1, 10, 100 for D1, D2, D3, and D4 respectively.Data1 (D1)=0.1+0.1*X_1+0.1*X_2+0.1*X_3+(Random Error)While the constant number and CoV values are 0.1, and random error with te CoE=0.1 will be added.Data2 (D2)=1+1*X_1+1*X_2+1*X_3+(Random Error)While the constant number and CoV values are 1, we will add the random error which has CoE=1.Data3 (D3)=10+10*X_1+10*X_2+10*X_3+(Random Error)While the constant number and CoV values are 10, we will add the random error which has the CoE=10.Data4 (D4)=100+100*X_1+100*X_2+100*X_3+(Random Error)While the constant number and CoV values are 100, we will add the random error which has the CoE=100.Metrics Table:It can be seen from figure while MSE and MAE are increasing R² and Accuracy are constant. We applied CoE/CoV rate as 1 for the D1, D2, D3, and D4. While CoE values for random error were increased, the R² and Accuracy did not change due to same CoE/CoV=1 for the D1, D2, D3, and D4.3. ConclusionA detailed analysis of Metrics has been made to observe variables’ coefficients effect on the ML Regression Models’ accuracy. The MSE and accuracy values were obtained as MSE=0.09 and accuracy=73% by applying a linear regression model to D1. And, the same linear regression model has been applied to D4 and we have obtained MSE and accuracy as MSE=9405 and accuracy=73%, respectively.The analysis results showed that the R² and Accuracy values were in strong function of CoE/CoV rate and there is no certain limitation for MSE to evaluate the regression models.The Jupiter notebook can be found on Github.

Leave a Reply