HEAL DSpace

Activity Recognition from Visual Cues in and beyond the Visual Spectrum

DSpace/Manakin Repository

Show simple item record

dc.contributor.author Μπάκαλος, Νικόλαος el
dc.contributor.author Bakalos, Nikolaos en
dc.date.accessioned 2023-01-13T08:51:21Z
dc.date.available 2023-01-13T08:51:21Z
dc.identifier.uri https://dspace.lib.ntua.gr/xmlui/handle/123456789/56669
dc.identifier.uri http://dx.doi.org/10.26240/heal.ntua.24367
dc.rights Αναφορά Δημιουργού 3.0 Ελλάδα *
dc.rights.uri http://creativecommons.org/licenses/by/3.0/gr/ *
dc.subject Deep learning, computer vision, analysis of visual cues, supervised and unsupervised learning, adaptive neural network architectures, data fusion, tensor-based learning en
dc.title Activity Recognition from Visual Cues in and beyond the Visual Spectrum en
dc.contributor.department Φωτογραμμετρία el
heal.type doctoralThesis
heal.secondaryTitle Αναγνώριση δραστηριοτήτων από οπτικές πηγές εντός και πέραν του ορατού φάσματος με χρήση βαθιά μηχανικής μάθησης. el
heal.classification Machine Learning en
heal.classification Computer Vision el
heal.language en
heal.access free
heal.recordProvider ntua el
heal.publicationDate 2022-09-20
heal.abstract Activity recognition from optical cues is an arduous task that has recently received a lot of attention in the research community, due to the performance of deep learning architectures in the analysis of such kinds of data. However, these analyses have to take into account both the types of input data as well as statistical particularities and a priori knowledge over the types of activities captured. This dissertation focuses on the development of deep machine learning methods to classify actions recorded in datasets consisting of capturings inside and outside the visible spectrum. Two main application scenarios are studied. The first application scenario includes recordings of traditional dance choreographies, where the dataset consists of a predefined set of actions (motion primitives), i.e. the steps that compose the specific dance choreography. The problem then takes the form a mutli-class classification task. Two deep learning classifiers are presented. For the data outside the visual spectrum, in this case recordings of infrared depth sensors, an optimised Long Short Term Memory (LSTM) neural network is presented. This classifier manages to capture both short-term dependencies, by using a short memory window before its input layer, as well as take into account non-causality during classification, by using the bidirectional variant of LSTM networks. For the data inside the visible spectrum, a hybrid architecture is presented. This architecture puts into use the feature extraction capabilities of Convolutional Neural Networks (CNN), as well as the ability of LSTM networks to map temporal correlations. Autoregressive and Moving Average capabilities are added to the architecture, while an adaptive weight control scheme is also employed. Finally, for the first application scenario, a tensor based classifier is presented that manages to classify choreographic motion primitives with similar performance, while requiring significantly less trainable parameters, allowing for increased performance even when a small set of training data is available. The second application scenario focuses on datasets where there is no a priori knowledge of the actions captured. Instead we study ways to “map” the normal state, and employ techniques for binary classification of the normal and the abnormal state. Initially, a supervised approach is presented, employing an adaptive NARMA filter, based on a CNN architecture. Data fusion from other sensors is also used to inform the classification step and increase performance. Additionally, unsupervised techniques, based on convolutional autoencoders are employed. Finally, a stack autoencoder method is presented where the feature extraction of convolutional spatiotemporal autoencoders is used in combination with a tensor-based autoencoder to model the normal state in datasets with large numbers of actions, and then perform outlier detection. en
heal.advisorName Doulamis, Anastasios
heal.committeeMemberName Georgopoulos, Andreas
heal.committeeMemberName Varvarigou, Theodora
heal.committeeMemberName Ioannides, Charalambos
heal.committeeMemberName Veskoukis, Vasileios
heal.committeeMemberName Karantzalos, Konstantinos
heal.committeeMemberName Voulodimos, Athanasios
heal.academicPublisher Σχολή Αγρονόμων και Τοπογράφων Μηχανικών el
heal.academicPublisherID ntua
heal.numberOfPages 123
heal.fullTextAvailability false


Files in this item

The following license files are associated with this item:

This item appears in the following Collection(s)

Show simple item record

Αναφορά Δημιουργού 3.0 Ελλάδα Except where otherwise noted, this item's license is described as Αναφορά Δημιουργού 3.0 Ελλάδα