Explainable Phishing Website Detection Using Comparative Machine Learning and SHAP

Abstract

Phishing attacks distributed through fraudulent URLs remain one of the most damaging cyber threats, including in Indonesia where malicious links spread widely through messaging applications and e-mail. Blacklist-based approaches cannot recognize newly created phishing URLs, while accurate machine learning models are often difficult to interpret. This study aims to compare six machine learning algorithms, namely XGBoost, Random Forest, Support Vector Machine, K-Nearest Neighbors, Logistic Regression, and Naive Bayes, for phishing website detection based on lexical URL, page content, and external reputation features, while providing model transparency through Explainable Artificial Intelligence (XAI). Experiments were conducted on the Web Page Phishing Detection Dataset containing 11,430 URLs with 87 features and a balanced class distribution. The research stages include exploratory data analysis, feature selection analysis using Chi-Square, Mutual Information, and Recursive Feature Elimination, an 80:20 data split, model training, hyperparameter optimization using RandomizedSearchCV, and interpretation of the best model using SHapley Additive exPlanations (SHAP). The results show that XGBoost delivers the best performance with 96.50% accuracy, 96.26% precision, 96.76% recall, 96.51% F1-score, and an AUC of 0.9943. SHAP analysis identifies google_index, page_rank, and nb_hyperlinks as the most influential features, dominated by external reputation-based features. Under this experimental setting, the findings indicate that an accurate phishing detection model can be equipped with interpretable explanations of its feature contributions. The main contribution of this study is an integrated comparative evaluation that combines six-algorithm benchmarking, leakage-free hyperparameter optimization, and SHAP-based interpretation on a public phishing dataset, offering practical guidance for security analysts.