Earlier this month, Sense Ph.D. student Antonio Carlos de Nazare Junior presented his novel approach for multiple camera alignment based on accelerometer data at the 15th IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS). AVSS is the premier conference in the field of video and signal-based surveillance and was held 27-30 November in Auckland, New Zealand, hosted by the Auckland University of Technology.

Video alignment is an important task for the surveillance of environments with multiple distributed cameras. Alignment consists in establishing a temporal correspondence between frames captured by different video cameras, allowing, for instance, verification of the exact instant that an event happened from different views. In his paper Content-Based Multi-Camera Video Alignment using Accelerometer Data, Antonio proposed a novel approach for this task using an auxiliary accelerometer sensor.

Many methods have been proposed to solve video alignment when the cameras present overlapping of Field of View (FOV). In his work, however, Antonio presented an approach to perform video alignment for cameras without overlapping FOV. The method employs the sensor data generated by a smartphone (synchronized to a time server), to align multiple videos and find a temporal match between the ones that captured a person in the scene, as well as the signal of the smartphone’s accelerometer carried by this individual.

Regarding the method, Antonio pointed out that although there is the restriction that a person carries an accelerometer in the scene, this is an important contribution since, to his knowledge, this is the first attempt to synchronize videos without requiring the cameras to be located close to each other or to have an overlapping field of view.

The proposed approach was able to align multiple videos with a length of 30 minutes with errors as low as 160 ms. The experiments performed reported small temporal misalignment between the videos and the accelerometer, which demonstrates the robustness of the proposed synchronization method.

Future work will include experiments employing more cameras and people for data capture. It will also include new methods for synchronization using multiple movement sensors, such as accelerometer, gyroscope and magnetometer. Another research direction is to extend the approach to handle videos with variable frame-rate. Moreover, Antonio intends to extend the proposed method to identify the individual who is carrying the accelerometer, a promising strategy, since this task is very challenging in crowded scenes due to visual ambiguity, a problem that is mitigated by the use of sensor data.

The research had support from the National Council for Scientific and Technological Development (CNPq), the Minas Gerais State Agency for Research and Development (FAPEMIG), the Coordination for the Improvement of Higher Education Personnel (CAPES) through the DeepEyes Project, and Petrobras.

The researcher

Antonio Carlos de Nazere Junior has a degree in Computer Science from the Federal University of Ouro Preto (UFOP) and a Master’s degree in Computer Science from the Federal University of  Minas Gerais (UFMG), with the thesis A Scalable and Versatile Framework for Smart Video Surveillance. He is currently a technological development fellow of the National Council for Scientific and Technological Development (CNPq). He has experience in high performance computing applied to computer vision and his work focuses on intelligent surveillance systems, video analysis, parallel and distributed computing and SIMD architectures.

The publication

Led by Antonio, the research had contributions from Filipe de Oliveira Costa, from the Image and Speech Processing Management of the Center for Research and Development in Telecommunications (CPqD). It was advised by Professor William Robson Schwartz, coordinator of the Smart Sense Laboratory.

AVSS 2018

Sponsored by the Institute of Electrical and Electronics Engineers (IEEE), the International Conference on Advanced Video and Signal-based Surveillance focuses on underlying theory, methods, systems, and applications of surveillance. In its 15th edition, AVSS is the premier conference in the field of video and signal-based surveillance and brings together experts from academia, industry, and government to advance theories, methods, systems, and applications related to surveillance. Since its creation, the conference has already been based on nine countries and four continents. Taipei will host AVSS 2019.

Follow updates of the Smart Sense Laboratory through our Facebook and Twitter.