Improving Fraud Detection in Credit Cards using Ensemble Learning with a Boosting Approach and Deep Learning Models

Abstract

Advances in technology have led to the emergence of digital payment. Since credit card transactions are the most common payment method and advanced methods have emerged for fraudsters, detecting fraudulent transactions is crucial due to the loss it causes to both cardholders and financial institutions. Existing studies suffer from limited interpretability and insufficient focus on behavioral and temporal feature engineering. This thesis aims to build models using state-of-the-art Ensemble Learning (EL) and Deep Learning (DL) algorithms for detecting credit card fraudulent transactions to decrease the losses for both cardholders and financial institutions. To achieve the objectives, the study highlights the importance of feature engineering by creating new features that include behavioral and temporal features aimed at enhancing model performance. Nine models were developed and evaluated; five of them follow a boosting approach in EL, namely Gradient Boosting Machine (GBM), Histogram-based Gradient Boosting (HGBM), Light Gradient Boosting Machine ( LightGBM), eXtreme Gradient Boosting (XGBoost), and Categorical Boosting (CatBoost). The other four models follow DL, namely Artificial Neural Network (ANN), Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). The study used European credit card transactions dataset in 2023, comprising 568,630 transactions. The models were trained on the European dataset twice, once with only parameters adjustment and once with parameters adjustment and feature engineering. In addition, understanding the features that most influence a model's decision to classify a transaction as fraudulent is crucial in the financial sector to enhance the reliability of models. Therefore, Shapley Additive Explanations (SHAP) method was applied to the EL and DL models. The results demonstrate the efficiency of both EL and DL models in detecting fraudulent transactions in credit cards. LightGBM and XGBoost models achieved the highest accuracy of 0.9998 on the original dataset. While in the dataset with new features, the HGBM, LightGBM and XGBoost models achieved the highest accuracy of 0.9997. In DL models, the highest accuracy was 0.9995 for the MLP, LSTM, and GRU models on the dataset with new features. The proposed models achieved the highest recall, ranging between 0.9989 and 1.0000, demonstrating their ability to detect fraudulent transactions. Meanwhile, precision ranged between 0.9838 and 0.9998, indicating that the models significantly reduce false alarms. Furthermore, F1 score values ranged between 0.9918 and 0.9998, indicating that the models achieved an excellent balance between recall and precision. The result highlights the ability of LSTM and GRU models to capture the temporal and behavioral sequences of transactions, where they are primarily designed to deal with time series. The findings highlight the reliability and robustness of the proposed models and their ability to accurately and efficiently detect fraudulent transactions. Furthermore, the proposed models may support financial institutions in improving credit card fraud detection decisions.