One of the major steps involved in training a model is to evaluate its performance which can be done using the testing dataset and a few evaluation metrics. The regression models can be evaluated using the regression metrics which we have discussed here. These Evaluation Metrics for Regression Models are very powerful, with several advantages above the others, and the explanation of these metrics is provided here.
Mean Absolute Error (MAE):
Mean absolute error is calculated as an average of the absolute differences between the predicted value of the model and the actual values. It can be given by the equation:
Where,
- N is the total number of data points,
- yi is the actual value, and
- Ŷi is the predicted value.
If the value of MAE obtained is small, it means that the model performs great at prediction and a large MAE value suggests the model is having trouble in several areas and doesn’t perform well. Moreover, if an MAE score of 0 is obtained, then it is a perfect prediction.
Mean Squared Error (MSE):
Mean squared error, or MSE, is the parameter to determine the efficiency of a regression model, which is done by calculating the squared differences between the predicted values and the actual values. The lower the value of MSE is obtained, the better the regression model performs. The value of MSE can be calculated by using the equation:
Where,
- N is the total number of data points,
- yi is the actual value, and
- Ŷi is the predicted value.
The mean squared errors are not robust to outliers, and it penalizes the outliers most, resulting in a bigger value of MSE being calculated. Hence, MAE is at an advantage with this factor.
Root Mean Squared Error (RMSE):
It is a value that is obtained by taking the square root of the mean squared error (MSE) and is the average root squared difference calculated between the actual and the predicted value. To determine the efficiency of the prediction of the regression model, the value of RMSE is used where the lower value of RMSE denotes that the model is performing better and a higher value of RMSE determines a large deviation between the actual and the predicted values. RMSE can be calculated using the formula:
Where,
- N is the total number of data points,
- yi is the actual value, and
- Ŷi is the predicted value.
Max Error:
The max error metric is used as an evaluation metric for the regression model and is considered the worst-case error between the true value and the predicted value. It looks at the quantiles of the distribution of the absolute percentage errors. Max-error can be employed as an alternative to the RMSE score since it can be hard to interpret.
R² score aka coefficient of determination:
It is a coefficient that determines the extent to which the variance of one variable explains the variance of another variable. Simply put, the R2 score measures the amount of variance of the dependent variable being explained through the independent variable. The R2 score can be calculated using the following equation:
Where SSE is determined as the sum of the squares of the difference between the predicted value and the actual value, the SST term determines the total sum of the squares of the difference between the mean of actual value and the actual value. The SSE and SST can be determined using the following equations:
Where,
- M represents the number of observations,
- y-bar is the mean value,
- yi is the observed target value, and
- Ŷi is the predicted value.
Adjusted R-Square:
The adjusted R2 evaluation metric is similar to the standard R2, where the adjusted one is calculated by penalizing the models when additional features are added. It counters the problems faced by the standard R2 metric by penalizing, which adds more independent variables and does not increase the explanatory power of the regression model. The value of adjusted R2 can be calculated by using the formula:
Where,
- N represents the number of data points, and
- K is the number of independent variables present in the model.
Implementation of Evaluation Metrics for Regression Models
The first step is to import all the required libraries
import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error, max_error, r2_score
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.datasets import fetch_california_housingSecond step is to load the Boston housing data and define the input features and the target variables.
# Load Boston Housing dataset
data = fetch_california_housing()
X = data.data
y = data.targetThird step is to split the dataset into training and testing 20% of the dataset goes in testing.
# Split into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)Then implement the linear regression model and fit it on the training data
model = LinearRegression()
model.fit(X_train, y_train)
After that you have to make the prediction on the test.
y_pred = model.predict(X_test)At last evaluate the model using various regression metrics like mean absolute error, mean squared error. root mean squared error, max error, r2 score, and adjusted r2 score.
# Mean Absolute Error
mae = mean_absolute_error(y_test, y_pred)
# Mean Squared Error
mse = mean_squared_error(y_test, y_pred)
# Root Mean Squared Error
rmse = np.sqrt(mse)
# Max Error
max_err = max_error(y_test, y_pred)
# R2 Score
r2 = r2_score(y_test, y_pred)
# Adjusted R2 Score
n = X_test.shape[0] # number of samples
p = X_test.shape[1] # number of features
adj_r2 = 1 - (1 - r2) * ((n - 1) / (n - p - 1))
After that print the results.
print(f"Mean Absolute Error (MAE): {mae:.2f}")
print(f"Mean Squared Error (MSE): {mse:.2f}")
print(f"Root Mean Squared Error (RMSE): {rmse:.2f}")
print(f"Max Error: {max_err:.2f}")
print(f"R² Score: {r2:.2f}")
print(f"Adjusted R² Score: {adj_r2:.2f}")
Conclusion
In this blog we have learned how to evaluate the performance of the regression model using various metrics. You can implement them using the scikit-learn library.
If you like the article and would like to support me, make sure to:
