Computer Vision problems applied to visual surveillance have been studied for several years with the aim of finding accurate and efficient solutions, which are required to allow the execution of surveillance systems in real environments. The main goal of such systems is to analyze the scene focusing on the detection and recognition of suspicious activities performed by humans in the scene, so that the security personnel can pay closer attention to these preselected activities. To accomplish that, several problems have to be solved first, for instance background subtraction, person detection, tracking and re-identification, face recognition, and action recognition. Even though each of these problems has been researched in the past decades, they are hardly considered in a sequence, each one is usually solved individually. However, in real surveillance scenarios, the aforementioned problems have to be solved in sequence considering only videos as the input.