Responsible Artificial Intelligence in Management Decision-Making: A Comparative Evaluation of Predictive Accuracy, Fairness, and Explainability

Abstract

The growing use of machine learning (ML) algorithms for human-capital management purposes, ranging from recruitment to performance assessment and retention strategies, raises critical questions regarding fairness and transparency of such systems. This paper aims to operationalize the concept of responsible AI (RAI) by developing a three-dimensional evaluation framework that includes predictive accuracy, fairness, and explainability. To this end, six classifiers – Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, Support Vector Machine (with RBF kernel), and Multilayer Perceptron – were trained on the IBM HR Employee Attrition dataset (n = 1,470). Predictive performance of the models was assessed using accuracy, precision, recall, F1-score, and ROC-AUC metrics. Algorithmic fairness was evaluated using demographic disparity and opportunity metrics for three sensitive features (Gender, Age Group, Marital Status). Finally, model explainability was determined using permutation feature importance and in-built model interpretability methods. The results demonstrate that all models exhibit a trade-off between predictive accuracy and fairness. Remarkably, the two most accurate algorithms (in terms of ROC-AUC), namely Logistic Regression (0.810) and SVM-RBF (0.799), showed the highest degree of unfairness for Marital Status and Age Group, while the fairest model (MLP) had an unacceptable level of recall (1.7%). Finally, the analysis revealed that Overtime, Job Level, Monthly Income, Job Satisfaction, and Stock Option Level were the most important predictors of employee attrition. Overall, the results demonstrate that no algorithm performed consistently superior on all three dimensions of RAI.