Activity Recognition from Visual Cues in and beyond the Visual Spectrum

Μπάκαλος, Νικόλαος; Bakalos, Nikolaos

dc.contributor.author	Μπάκαλος, Νικόλαος	el
dc.contributor.author	Bakalos, Nikolaos	en
dc.date.accessioned	2023-01-13T08:51:21Z
dc.date.available	2023-01-13T08:51:21Z
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/56669
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.24367
dc.rights	Αναφορά Δημιουργού 3.0 Ελλάδα	*
dc.rights.uri	http://creativecommons.org/licenses/by/3.0/gr/	*
dc.subject	Deep learning, computer vision, analysis of visual cues, supervised and unsupervised learning, adaptive neural network architectures, data fusion, tensor-based learning	en
dc.title	Activity Recognition from Visual Cues in and beyond the Visual Spectrum	en
dc.contributor.department	Φωτογραμμετρία	el
heal.type	doctoralThesis
heal.secondaryTitle	Αναγνώριση δραστηριοτήτων από οπτικές πηγές εντός και πέραν του ορατού φάσματος με χρήση βαθιά μηχανικής μάθησης.	el
heal.classification	Machine Learning	en
heal.classification	Computer Vision	el
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2022-09-20
heal.abstract	Activity recognition from optical cues is an arduous task that has recently received a lot of attention in the research community, due to the performance of deep learning architectures in the analysis of such kinds of data. However, these analyses have to take into account both the types of input data as well as statistical particularities and a priori knowledge over the types of activities captured. This dissertation focuses on the development of deep machine learning methods to classify actions recorded in datasets consisting of capturings inside and outside the visible spectrum. Two main application scenarios are studied. The first application scenario includes recordings of traditional dance choreographies, where the dataset consists of a predefined set of actions (motion primitives), i.e. the steps that compose the specific dance choreography. The problem then takes the form a mutli-class classification task. Two deep learning classifiers are presented. For the data outside the visual spectrum, in this case recordings of infrared depth sensors, an optimised Long Short Term Memory (LSTM) neural network is presented. This classifier manages to capture both short-term dependencies, by using a short memory window before its input layer, as well as take into account non-causality during classification, by using the bidirectional variant of LSTM networks. For the data inside the visible spectrum, a hybrid architecture is presented. This architecture puts into use the feature extraction capabilities of Convolutional Neural Networks (CNN), as well as the ability of LSTM networks to map temporal correlations. Autoregressive and Moving Average capabilities are added to the architecture, while an adaptive weight control scheme is also employed. Finally, for the first application scenario, a tensor based classifier is presented that manages to classify choreographic motion primitives with similar performance, while requiring significantly less trainable parameters, allowing for increased performance even when a small set of training data is available. The second application scenario focuses on datasets where there is no a priori knowledge of the actions captured. Instead we study ways to “map” the normal state, and employ techniques for binary classification of the normal and the abnormal state. Initially, a supervised approach is presented, employing an adaptive NARMA filter, based on a CNN architecture. Data fusion from other sensors is also used to inform the classification step and increase performance. Additionally, unsupervised techniques, based on convolutional autoencoders are employed. Finally, a stack autoencoder method is presented where the feature extraction of convolutional spatiotemporal autoencoders is used in combination with a tensor-based autoencoder to model the normal state in datasets with large numbers of actions, and then perform outlier detection.	en
heal.advisorName	Doulamis, Anastasios
heal.committeeMemberName	Georgopoulos, Andreas
heal.committeeMemberName	Varvarigou, Theodora
heal.committeeMemberName	Ioannides, Charalambos
heal.committeeMemberName	Veskoukis, Vasileios
heal.committeeMemberName	Karantzalos, Konstantinos
heal.committeeMemberName	Voulodimos, Athanasios
heal.academicPublisher	Σχολή Αγρονόμων και Τοπογράφων Μηχανικών	el
heal.academicPublisherID	ntua
heal.numberOfPages	123
heal.fullTextAvailability	false