This paper presents a generic video coding framework with the texture abstraction and completion, inspired by a strong grouping bias of local elements in Gestalt psychology. Abstracting imagery by grouping perceptual salience from anisotropic diffusion, it decomposes video images into two layers composing of semantic components and residual detail. The similarity between textures of abstraction layer is motivated to infer the restoration of missing detail, under the spatio-temporal variation regularity. Through a motion and spatial context of moton, hence, a group of pictures (GOP) is divided into key frames and abstracted frames to form the final compressed data. An abstraction refinement is tuned to improve matching of detail restoration based on bilateral filtering. The proposed approach is more generic without incurring any specific side information, and achieves up to 20% bit saving versus standard H.264 at similar visual quality levels.