Small group people behavior analysis has attracted much attention in recent years. How to detect, track and analyze the group behavior of related people is a challenging problem. In this paper, a framework for small group people behavior analysis is proposed in video surveillance applications, which is based on temporal recursive trajectory identification. According to real applications, a specific surveillance scene is divided into several zones and correspondent zone connectivity relations are obtained. After that, people counting methods are used to get numbers of people in each zone. Together with zone connectivity relation, a temporal recursive method is applied to identify trajectory that represents the movement of each person between zones. From these determined motion trajectories, groups of people are detected and distributed and their behaviors are analyzed. Experiments on Shanghai World Expo 2010 video surveillance database are conducted to show the effectiveness of the proposed framework.