Identification and classification of signals which are heard by underwater microphones (hydrophones) can be used extensively in harbor traffic management, especially in economical harbors. However, automatic identification and classification of acoustic signals which are received by passive sonar system is a challenging problem, because of variation in temporal and frequency characteristics of signals (even they are received from a same source). In this paper, a novel method for classification of acoustic signals is presented, based on DWT as preprocessing, a diverse range of feature extraction methods (principal component analysis and its variations (6 methods) and discriminant analysis and its variations (3 methods)), and 4 ensemble learning methods with 3 classifiers (multilayer perceptron (MLP), probabilistic neural network (PNN) and support vector machine (SVM)). Performing a diverse range of performance tests, the performances of different methods are assessed and the best ones are chosen for the proposed method. The proposed method is used to extract features and classify acoustic signals of 8 ships. Using the proposed method, some real signals and their noisy version are classified. The accuracy of the proposed method in classification of test signals with Gaussian white noise with -5, -10 and -15 signal-to-noise ratio is obtained as 99.83%, 97.06% and 83.56%, respectively.