Removal of noise and restoration of images has been one of the most interesting topics in the field of image processing in the past few years. Existing filter-based methods can remove image noise; however, they cannot preserve image quality and information such as lines and edges. In this article, various classifiers and spatial filters are combined to achieve desirable image restoration. Meanwhile, the time adaptive self-organizing map (TASOM) classifier is more emphasized in our feature extraction and dimensionality reduction approaches to preserve the details during the process, and restore the images from noise. The TASOM was compared with the self-organizing map (SOM) network, and a suitable noise reduction method for images was attempted. As a result, we achieved an optimum method to reduce impulsive noise. In addition, by using this neural network, better noise suppression was achieved. Experimental results show that the proposed method effectively removes impulse noise and maintains color information as well as image details.