The Unscented Kalman filter (UKF) is the popular approach to estimate the recursive parameter of nonlinear dynamical system corrupted with Gaussian and white noises. Also, it has been applied to train the weights of the multi-layered neural network (MNN) models. The Group method of data handling (GMDH)-type neural network is one of the most widely used neural networks which has high capacity in modeling of the complex data. In many researches, different approaches are used in training of neural networks in terms of associated weights or coefficients, such as singular value decomposition, and genetic algorithms. In this paper, the unscented Kalman filter is used to train the parameters of GMDH-type neural network when the experimental data are deterministic. The effectiveness of GMDH-type neural network with UKF algorithm is demonstrated by the modeling of the using a table of the multi input-single output experimental data. The simulation result shows that the UKF-based GMDH algorithm perform well in modeling of nonlinear systems in comparison with the results of using traditional GMDH-type neural network and is more robust against the model and measurement uncertainty.