A Continuous Learning Framework for Activity Recognition Using Deep Hybrid Feature Models

Most of the research on human activity recognition has focused on learning a static model, considering that all the training instances are labeled and present in advance, while in streaming videos new instances continuously arrive and are not labeled. Moreover, these methods generally use applicatio...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2015-11, Vol.17 (11), p.1909-1922
Main Authors: Hasan, Mahmudul, Roy-Chowdhury, Amit K.
Format: Article
Language:English
Subjects:
Active learning
activity recognition
Adult entertainment
autoencoder
Computational modeling
Data models
Feature extraction
Feature recognition
Human
hybrid feature model
incremental learning
Labeling
Learning
Machine learning
Multimedia
Networks
Recognition
Training
Transaction processing
Videos
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Most of the research on human activity recognition has focused on learning a static model, considering that all the training instances are labeled and present in advance, while in streaming videos new instances continuously arrive and are not labeled. Moreover, these methods generally use application- specific hand-engineered and static feature models, which are not suitable for continuous learning. Some recent approaches on activity recognition use deep-learning-based hierarchical feature models, but the large size of these networks constrain them from being used in continuous learning scenarios. In this work, we propose a continuous activity learning framework for streaming videos by intricately tying together deep hybrid feature models and active learning. This allows us to automatically select the most suitable features and take the advantage of incoming unlabeled instances to improve the existing model incrementally. Given the segmented activities from streaming videos, we learn features in an unsupervised manner using deep hybrid networks, which have the ability to take the advantage of both the local hand-engineered features and the deep model in an efficient way. Additionally, we use active learning to train the activity classifier using a reduced amount of manually labeled instances. Retraining the models with a huge amount of accumulated examples is computationally expensive and not suitable for continuous learning. Hence, we propose a method to select the best subset of these examples to update the models incrementally. We conduct rigorous experiments on four challenging human activity datasets to demonstrate the effectiveness of our framework.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2015.2477242