Control of seismic shake table in order to track the predefined earthquake profile is a key concern in design of seismic shake tables. This paper proposes a vision-based real time displacement measurement system using image processing techniques to control a laboratory-scale seismic shake table. The shake table is controlled via a fuzzy-supervisory controller, an inner PID loop and a Fuzzy outer one, whose feedback is provided by the vision-based measurement system. To minimize tracking errors, the fuzzy controller uses displacement and acceleration responses as its feedbacks. For this purpose, a camera and an image processing application are utilized to measure the motions directly in real time. Results are sent to a host PC through a network as the controller feedback. Proposed system performance is compared with an alternative system which utilizes a linear encoder as displacement sensor and controller feedback. Test results prove effectiveness of the proposed fuzzy system in cutting back the tracking errors. In addition, the vision-based system uses a very low cost camera to measure the displacement directly. It has appropriate accuracy, works in real time, and doesn't need any contact with the table, comparing to the encoder version.