Two stage grid search accepted as a promising heuristic search technique, involves a search performed in two stages. In the first stage a search is performed in coarse grain/low resolution to identify the optimal region and, in the second stage, a fine grain/high resolution search is performed in the neighborhood of the optimal region to identify the optimal parameters. Performing a search in two stages considerably reduces the computational complexity when compared to the basic grid search algorithm. However, an exhaustive search is to be carried out in the subspace during the second stage which may again be a computationally expensive task. The main contribution of this paper is to develop a new heuristic search technique which explores the discrete parameter space dimension wise recursively. The time complexity of the proposed algorithm is less than that of the two-stage grid search. The performance of the proposed algorithm in terms of required number of probes and time for optimal model selection, compared with the two-stage grid search, is verified for correctness and efficiency.