Machining vibration is one of the most important constraints on productivity. This vibration may cause increase in machining costs, lower accuracy of products, and decrease tool life. Active control is one of the conventional methods for dealing with vibration in machining, but designing an optimized controller for machining process due to unknown parameters in the system is challenging. DVF control method with low computational costs and high capability in increasing the performance of the cutting tool is an effective method, but due to increasing in actuator control input, it can cause actuator saturation; thus, it is not an efficient control method. The aim of this research is implementation of a nonlinear fractional PID controller for increasing effectiveness and improving performance of active vibration control on a boring bar. The results of impact control tests indicate that nonlinear PID control algorithm has good performance in reducing vibrations and increasing the damping of the structure. Using the controller performance criteria, the optimal fractional can be chosen for the nonlinear PID controller, which in addition to increasing the damping of the tool, can reduce the power consumption and, thus, prevent the actuator saturation. The results of the cutting tests also show that the nonlinear PID controller reduces control voltage and actuator power with respect to the DVF controller, which results in improving the boundaries of stable machining. Moreover, during impacts in machining process, such as the initial engagement of the tool, the proposed controller results in a significant reduction in the control voltage peak.