A method proposed by Sense members won 2nd place on the “Surveillance Imagery Search” task from the “Semantic Person Retrieval in Surveillance Using Soft Biometrics” challenge, held at the Conference on Advanced Video and Signal-based Surveillance, AVSS 2018. Gabriel Resende Gonçalves led the work, developed in collaboration with the postgraduate students Antonio Carlos de Nazare Junior, Matheus Alves Diniz and the undergraduate student Luiz Eduardo Lima. AVSS was held 27-30 November in Auckland, New Zealand, hosted by the Auckland University of Technology.

In parallel to its lectures, workshops and tutorials, AVSS has launched two challenges, the “Advanced Traffic Monitoring” and the “Semantic Person Retrieval in Surveillance Using Soft Biometrics”, each of them divided into two tasks. The Sense team participated on the latter challenge, on the “Surveillance Imagery Search” task.

A common goal in surveillance and security is to locate a subject of interest purely from a semantic description, such as an offender description form handed to a law enforcement agency. These tasks are primarily undertaken by operators either by manually searching the premises or by visual inspection of hours of video footage. A promising alternative is performance of an automatic semantic search.

Semantic search is of primary interest since it does not require registering the subject on a gallery of known individuals, as several others approaches do, instead, it searches video footage based on a textually supplied target query that specifies, for instance, gender, and clothing type and color . The main goal of the challenge addressed by the Sense team can be seen as analogous to a person re-identification task, but with text queries instead of probe images.

The awarded approach proposed by Sense researchers addresses this problem with a multi-task learning approach that assumes that the attributes available for the query are highly related and can be learned together in the same network as different tasks.

The approach
The method proposed for the challenge is a result of the research Soft Biometric Retrieval Using Deep Multi-Task Network, led by Gabriel Resende Goncalves. The work had contributions from the postgraduate students Antonio Carlos de Nazare Junior and Matheus Alves Diniz and the undergraduate student Luiz Eduardo Lima. Professor William Robson Schwartz, coordinator of Smart Sense Laboratory, guided the research.

AVSS 2018
In its 15th edition, the International Conference on Advanced Video and Signal-based Surveillance focuses on underlying theory, methods, systems, and applications of surveillance. Sponsored by the Institute of Electrical and Electronics Engineers (IEEE), AVSS is the premier conference in the field of video and signal-based surveillance and brings together experts from academia, industry, and government to advance theories, methods, systems, and applications related to surveillance. Since its creation, the event has already been based on four continents.

Follow updates of the Smart Sense Laboratory through our Facebook and Twitter.