Jessica Sena defended, this afternoon (18), her dissertation titled “Human Activity Recognition based on Wearable Sensors using Multiscale DCNN Ensemble”, obtaining her Master’s degree from the Graduate Program in Computer Science of the Federal University of Minas Gerais (UFMG). The main results of her research were presented two months ago at the European Signal Processing Conference (EUSIPCO), in Rome, Italy. Sena was recently accepted in the Ph.D. program in Computer Science at the UFMG, starting early 2019.

Sensor-based Human Activity Recognition (HAR) has attracted interest in several areas such as medical, military and security applications. Recently, wearable devices have gained space in activity recognition mainly due to their wide consumer adoption, ease of data collection and variety of sensor modalities, the latter of which provides rich and complementary sources of information.

However, there are two main aspects with wearable sensors that must be addressed, namely, the heterogeneity of the data from multiple sensors and its temporal nature. To address the first aspect, the approach proposed by Jessica processes each sensor separately, learning features for each sensor and performing the classification before fusing with the classification output of the other sensors. To address the second aspect, patterns are extracted from multiple temporal scales.

Extraction of features from multiple temporal scales is performed using an ensemble of Deep Convolutional Neural Networks (DCNN). In this ensemble, convolutional kernels with different heights are used for each DCNN. Since the rows in the sensor data represent the data captured over time, each kernel height extracts features at a specific temporal scale.

Jessica’s approach is able to extract data from simple movement patterns such as a wrist twist when picking up a spoon, to complex movements, such as the human gait. Her multimodal and multi-temporal method outperforms previous state-of-the-art works in seven important datasets using two different protocols.

Thesis committee

Professor William Robson Schwartz, coordinator of the Smart Sense Laboratory, supervised Sena’s research. The thesis committee was composed by Professors Marcos Andre Gonçalves and Leonardo Antonio Torres, from the Departments of Computer Science and Electronic Engineering of UFMG, respectively.

Researcher curriculum

One of Sense’s oldest members, Jessica Sena holds a degree in Information Systems from UFMG, with the senior thesis, also supervised by Professor Schwartz, “Improvements to Real-Time Update of Subject Galleries in Multi-camera Settings”. Her main research topics include biometrics, pattern recognition, surveillance and computer vision. She also participates in R&D projects conducted by Sense with private companies.