In this paper, modeling and tow type of nonlinear controller for trajectory tracking of a novel five-rotor UAV (Unmanned Aerial Vehicle) is developed. Because of the very simple structure and high maneuverability, quadrotors are one of the most preferred types of UAVs but the main problem of them is their small payload. In the proposed novel model, one propeller is added to the center of vehicle to improve the ability of lifting heavier payloads, and to excel anti-crosswind capability of quadrotor. The dynamic model is obtained via Newton Euler approach. The model is under actuated, nonlinear, and has strongly coupled terms. Also, two types of nonlinear controllers are presented. First one is a conventional input-output feedback linearization controller which involves high-order derivative terms and turns out to be quite sensitive to sensor noise as well as modeling uncertainty. Second controller is a BackStepping controller based on the hierarchical control strategy that yields easier controller. The obtained simulation results confirm that the performance of BackStepping controller is convenient in terms of stability, position tracking and it is robust in presence of disturbance.