Performance analysis of six meta-heuristic algorithms over automated test suite generation for path coverage-based optimization

Abstract

There exists a direct need to automate the process of test suite generation to get the most optimal results as testing accounts for more than 40% of total cost. One method to solve this problem is the use of meta-heuristic algorithms which iteratively improve the test data to reach the most optimized test suites. This study focuses on the performance evaluation of six meta-heuristic algorithms namely: hill-climbing algorithm (HCA), particle swarm optimization (PSO), firefly algorithm (FA), cuckoo search algorithm (CS), bat algorithm (BA) and artificial bee colony algorithm (ABC) by using their standard implementation to optimize the path coverage and branch coverage produced by the test data. The goal of the study was to find the best-suited algorithm to narrow down the future research in the field of test automation for path coverage-based optimization approaches. Each algorithm was first implemented to automatically generate test suites based on the program under test. This was followed by the performance evaluation of each algorithm for five programs written in Java. The algorithms were compared using process metrics: average time, best time, worst time and product metrics: path coverage & objective function values of the generated test suites. Results indicated ABC as the best-suited algorithm as it gave the most optimal test suites in reasonable time. BA was found to be the fastest but produced less optimal results. FA was found to be the slowest algorithm, while CS, PSO and HCA performed in between. These results show the relative performance of the six algorithms for this scenario and may be used by the future researchers to narrow down and improve the best performing algorithms for path coverage-based optimization approaches.