Neural Architecture Search (NAS) for Vision Tasks

NAS is one of the recent techniques that help in the automation of the process of coming up with the best neural network architecture for a given vision task and has been enhanced by using various methods to the architecture to enhance its performance. In this regard, this article provides the background information on the NAS, the approaches used in the NAS, the current issues and the future of the NAS. Several NAS techniques have been proposed in this work, which includes RL and DARTS, and that have enhanced the efficiency and accuracy of the neural network architecture search. However, there are some problems, which significantly influence NAS. Some of the issues that remain include; the fact that the process is computationally expensive since search and training of numerous candidate architectures is a very time consuming and requires a lot of computational power. In addition, the search space defined in advance can hinder the discovery of new architectures; problems of generalization are related to the question of how the obtained results can be applied to other tasks and datasets. The problem with NAS results is that it is not always clear why specific architectures perform well and this requires future work to consider the following; better search algorithms, the dynamic nature of the search space, and a better understanding of how NAS results can be transferred across different domains and easily explained. Further work could be done to use NAS with other optimization algorithms or other features of the problem or the domain that can improve the solutions found. It means, that NAS has the unique potential for increasing the automation level of the developing neural networks, and has many open issues which should be solved to reach the defined goal.