A novel stacking-based ensemble machine learning model for accurate user story effort estimation in scrum

Abstract

Effort estimation remains a persistent challenge in agile software development, particularly at the user story level where iterative delivery is emphasized. Inaccurate estimations can result in misallocated resources, budget overruns, and project delays. This research introduces a novel stacking-based ensemble Machine Learning (ML) model designed to enhance the accuracy of user story effort estimation. To address the scarcity of granular data, a high-fidelity dataset comprising 160 user stories from 36 professional scrum-based projects was developed, incorporating 13 industry-validated effort drivers. Adopting the Design Science Research (DSR) methodology, a two-tier hierarchical ensemble model was implemented. In this architecture, Extra Trees, XGBoost, and Random Forest function as Tier-1 base learners, while Linear Regression is employed as the Tier-2 meta-learner to synthesize predictive outputs and optimize the final estimate. Furthermore, a functional web-based prototype interface was developed, operationalizing the model for real-time decision support. Model’s performance was assessed using standard regression metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE). The proposed stacking ensemble model outperformed all standalone individual models, achieving MAE = 0.51, MSE = 0.49, and RMSE = 0.70. These results demonstrate the model’s effectiveness in improving estimation precision, thereby facilitating superior sprint planning and resource optimization within Scrum teams. Future research will explore the integration of unstructured textual data through Natural Language Processing (NLP) techniques and the adoption of deep learning architectures to further expand the model’s predictive capabilities.