In this paper, a novel color face recognition method is proposed based on Local Binary Patterns (LBP) of Quaternionic Gabor features (QGF). By introducing Quaternion Gabor analysis into image representation, we make full use of the interrelationship among different color channels to enhance the performance of the face recognition system. Moreover, the QGF are used to encode the positions and attributes of the face elements. Non-parametric transformation is then imposed on these QGF using LBP method to obtain the robustness against variations of pose, illumination and facial expressions. Compared with the monochromatic face recognition systems, which nowadays dominate the marketplace and research field, this approach materializes the strong potential use of color face recognition system by establishing invariant quaternion wavelet features of color images. The experimental results on the open face database testify the validity of the proposed method under severe noise corruption and distinct variations of scale, illumination and facial expressions.