Conclusion Stability on Performance of Analogy-Based Software Effort Estimation
Passakorn Phannachitta (1301021)
Analogy-based estimation (ABE) is one of the most successful software effort estimation methods. For over two decades, researchers have continually proposed new approaches, combined multiple approaches, and tailored them to ABE models to further improve its accuracy. To date, it has been reported that over thousands of combinations of approaches exist. However, to the best of our knowledge, no one has successfully determined the consistently best combination of approaches or knew whether the best combination of approaches does actually exist. This is mainly because if ones read through the literature on software effort estimation, conflicting research conclusions were considered frequently encountered. Therefore, we begin the studies of this thesis with an adoption of a stable ranking method to produce a trustworthy performance ranking of ABE models being tailored with numerous possible combinations of approaches, commonly adopted with the models in practice. Leveraged by this stable ranking method, we can successfully discover a stable ranking of those approaches and determine the generally best combination of approaches. Further study and analyses of results allow us to propose a new successful solution adaptation technique, one of the most imperative components of the ABE models. Composing all the findings from the studies of this thesis together, we compare the ABE model, tailoring with the generally best combination of approaches, with 7 other common machine-learning effort models (NNet, LReg, SWReg, PCReg, PLSReg, CART(yes), and CART(no) ). The results of this comparison are conclusive that this ABE model outperforms the other 7 effort models by all means of overall performance, generalized performance, stability, and robustness. Hence, we strongly recommend this ABE model to be a standard benchmark software effort estimation model for research community, and the effort model of choice for the software industries.