The goal of video stabilization is to remove the unwanted camera motion and obtain stable versions. Theoretically, a good stabilization algorithm should remove the unwanted motion without the loss of image qualities. However, due to the lack of ground-truth video frames, the accurate performance evaluation of different algorithms is hard. Most existing evaluation techniques usually synthesize stable videos from shaking ones, but they are not effective enough. Different from previous methods, in this paper we propose a novel method which synthesize shaking videos from stable frames. Based on the synthetic shaking videos, we perform preliminary video stabilization performance assessment on three stabilization algorithms. Our shaking video synthesis method can not only give a benchmark for full-reference video stabilization performance assessment, but also provide a basis for exploring the theoretical bound of video stabilization which may help to improve existing stabilization algorithms.