In High Efficiency Video Coding (HEVC), the computational complexity has been increased so that an Early SKIP mode decision method is proposed by using coded block flag of an inter prediction unit (PU) to speed up mode decision with BD-bitrate increase. In this paper, we propose a fast SKIP mode decision algorithm to speed up PU mode decision for HEVC in a rate-distortion (RD) optimization sense. It is based on an adaptive linear predictor for PU mode type and available neighboring SKIP mode units on the quad-tree partition structure in HEVC for adaptive RD optimization. Moreover, the proposed method consists of both local (neighboring PUs) and global (previous PUs in current frame) level schemes on each coding unit (CU) level. In this way, it is possible to exploit high correlations between the RD cost of SKIP mode and spatial characteristics of video sequences by using the distribution of SKIP mode and a linear RD cost predictor. Experimental results show that the encoding computational complexity can be reduced by an average of 36 % with threshold RD cost, when compared to that of the HM 12 reference software.