High Efficiency Video Coding (HEVC) achieves high efficiency by introducing a new coding structure in adoption of coding unit (CU), prediction unit (PU) and transform unit (TU). However, it also imposes great computation burden on the mode decision of encoders. In this paper, we propose a fast CU depth decision scheme to reduce the encoder complexity for HEVC. Firstly, the relationship between rate-distortion (R-D) cost and CU depth is explored carefully with Mean Squared Error (MSE) and Number of Encoded Bits (NEB) metrics. Then CU splitting is modeled as a binary classification problem and resolved by an offline trained Support Vector Machine (SVM) model. The experimental results show that the proposed algorithm achieves up to 59% running-time reduction with negligible loss in terms of PSNR and bit rate.