Importance Annotation for VIP and UIUC Pascal Sentence Datasets

The experimental results for the paper Assigning Relative Importance to Scene Elements in SIBGRAPI’2017 (link to the research page) were obtained using two datasets: VIP dataset and UIUC Pascal Sentence. Both datasets are associated to importance assignment researches and present a wide range of images containing multiple objects per image.

Data

To use both datasets on the paper Assigning Relative Importance to Scene Elements, it was necessary to generate importance annotations, since they are not provided along with dataset images. In addition, the VIP dataset does not provide element (people) bounding boxes and so, it was necessary to annotate these elements before using this dataset. VIP boxes annotations are available in: vip_boxes. To allow the reproduction of the experiments, we also provide the boxes of the UIUC Pascal sentence dataset, available in: uiuc_boxes.

After that, users were asked to generate importance annotations for images of the VIP, UIUC and Pascal Sentence Databases. These annotations are available in: Importance Annotations

References

You should cite the following paper if you use this dataset in your work.

Igor Leonardo Oliveira Bastos; William Robson Schwartz: Assigning Relative Importance to Scene Elements. In: Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 1-8, 2017. (Type: Inproceedings | Links | BibTeX)

Related Publications

Carlos Antônio Caetano Júnior

Motion-Based Representations for Activity Recognition PhD Thesis

Universidade Federal of Minas Gerais, 2020.

Abstract | BibTeX

@phdthesis{CarlosCaetano:2020:PhD,

title = {Motion-Based Representations for Activity Recognition},

author = {Carlos Antônio Caetano Júnior},

year  = {2020},

date = {2020-01-27},

school = {Universidade Federal of Minas Gerais},

abstract = {In this dissertation we propose four different representations based on motion information for activity recognition. The first is a spatiotemporal local feature descriptor that extracts a robust set of statistical measures to describe motion patterns. This descriptor measures meaningful properties of co-occurrence matrices and captures local space-time characteristics of the motion through the neighboring optical flow magnitude and orientation. The second, is the proposal of a compact novel mid-level representation based on co-occurrence matrices of codewords. This representation expresses the distribution of the features at a given offset over feature codewords from a pre-computed codebook and encodes global structures in various local region-based features. The third representation, is the proposal of a novel temporal stream for two-stream convolutional networks that employs images computed from the optical flow magnitude and orientation to learn the motion in a better and richer manner. The method applies simple non-linear transformations on the vertical and horizontal components of the optical flow to generate input images for the temporal stream. Finally, the forth is a novel skeleton image representation to be used as input of convolutional neural networks (CNNs). The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Moreover, the representation has the advantage of combining the use of reference joints and a tree structure skeleton, incorporating different spatial relationships between the joints and preserving important spatial relations. The experimental evaluations carried out on challenging well-known activity recognition datasets (KTH, UCF Sports, HMDB51, UCF101, NTU RGB+D 60 and NTU RGB+D 120) demonstrated that the proposed representations achieved better or similar accuracy results in comparison to the state of the art, indicating the suitability of our approaches as video representations.},

keywords = {},

pubstate = {published},

tppubtype = {phdthesis}

}

In this dissertation we propose four different representations based on motion information for activity recognition. The first is a spatiotemporal local feature descriptor that extracts a robust set of statistical measures to describe motion patterns. This descriptor measures meaningful properties of co-occurrence matrices and captures local space-time characteristics of the motion through the neighboring optical flow magnitude and orientation. The second, is the proposal of a compact novel mid-level representation based on co-occurrence matrices of codewords. This representation expresses the distribution of the features at a given offset over feature codewords from a pre-computed codebook and encodes global structures in various local region-based features. The third representation, is the proposal of a novel temporal stream for two-stream convolutional networks that employs images computed from the optical flow magnitude and orientation to learn the motion in a better and richer manner. The method applies simple non-linear transformations on the vertical and horizontal components of the optical flow to generate input images for the temporal stream. Finally, the forth is a novel skeleton image representation to be used as input of convolutional neural networks (CNNs). The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Moreover, the representation has the advantage of combining the use of reference joints and a tree structure skeleton, incorporating different spatial relationships between the joints and preserving important spatial relations. The experimental evaluations carried out on challenging well-known activity recognition datasets (KTH, UCF Sports, HMDB51, UCF101, NTU RGB+D 60 and NTU RGB+D 120) demonstrated that the proposed representations achieved better or similar accuracy results in comparison to the state of the art, indicating the suitability of our approaches as video representations.

Victor Hugo Cunha de Melo; Jesimon Barreto Santos; Carlos Antonio Caetano Junior; Jessica Sena; Otavio A B Penatti; William Robson Schwartz

Object-based Temporal Segment Relational Network for Activity Recognition Inproceedings

In: Conference on Graphic, Patterns and Images (SIBGRAPI), pp. 1-8, 2018.

BibTeX

Carlos Antonio Caetano Junior; Jefersson A dos Santos; William Robson Schwartz

Statistical Measures from Co-occurrence of Codewords for Action Recognition Inproceedings

In: VISAPP 2018 - International Conference on Computer Vision Theory and Applications, pp. 1-8, 2018.

Links | BibTeX

Igor Leonardo Oliveira Bastos; Larissa Rocha Soares; William Robson Schwartz

Pyramidal Zernike Over Time: A spatiotemporal feature descriptor based on Zernike Moments Inproceedings

In: Iberoamerican Congress on Pattern Recognition (CIARP 2017), pp. 77-85, 2017.

Links | BibTeX

Carlos Antonio Caetano Junior; Victor Hugo Cunha de Melo; Jefersson Alex dos Santos; William Robson Schwartz

Activity Recognition based on a Magnitude-Orientation Stream Network Inproceedings

In: Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 1-8, 2017.

Links | BibTeX

Carlos Antonio Caetano Junior; Jefersson A dos Santos; William Robson Schwartz

Optical Flow Co-occurrence Matrices: A Novel Spatiotemporal Feature Descriptor Inproceedings

In: IAPR International Conference on Pattern Recognition (ICPR), pp. 1-6, 2016.

Links | BibTeX

Importance Annotation for VIP and UIUC Pascal Sentence Datasets

<img loading="lazy" class="alignnone size-full wp-image-1018" src="http://smartsenselab.dcc.ufmg.br/wp-content/uploads/2018/11/database-1.png" alt="Data" width="32" height="32" /> Data

References

Related Publications

Data