Semantic segmentation for ADL in assistive robotics and comparison of RGB/RGB-D input and Single/Multiple viewpoints

Οικονόμου, Αικατερίνη Μαρία; Oikonomou, Aikaterini Maria

dc.contributor.author	Οικονόμου, Αικατερίνη Μαρία	el
dc.contributor.author	Oikonomou, Aikaterini Maria	en
dc.date.accessioned	2021-04-19T14:43:28Z
dc.date.available	2021-04-19T14:43:28Z
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/53356
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.21054
dc.description	Εθνικό Μετσόβιο Πολυτεχνείο--Μεταπτυχιακή Εργασία. Διεπιστημονικό-Διατμηματικό Πρόγραμμα Μεταπτυχιακών Σπουδών (Δ.Π.Μ.Σ.) “Συστήματα Αυτοματισμού”
dc.rights	Default License
dc.subject	Semantic segmentation	en
dc.subject	Computer vision	en
dc.title	Semantic segmentation for ADL in assistive robotics and comparison of RGB/RGB-D input and Single/Multiple viewpoints	en
heal.type	masterThesis
heal.classification	Computer vision	en
heal.language	el
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2021-02-01
heal.abstract	Computer vision, the field of artificial intelligence that develops computational algorithms to interpret and understand the visual world, has been studied in many perspectives. It expands from raw data recordings into techniques and ideas combining digital image processing, pattern recognition, machine learning and computer graphics. As its extensive usage has attracted many scholars to integrate with many disciplines and fields, in assistive robotics, computer vision aims to develop systems to help robots understand images in a similar way humans do. In this thesis, we focus on semantic segmentation, the process of labelling each area–or pixel– of a digital image according to a representation class. We make use of deep learning methods, which, over the last years, have been shown to outperform previous state-of-the-art machine learning techniques in several fields, with computer vision being one of the most prominent cases. Deep learning models can be trained to identify and classify objects in an image, and, in particular, Convolutional Neural Networks (CNN), a class of deep neural networks most commonly used in visual imagery, has been shown to be able to extract important visual features that resemble the ones generated in the visual cortex of humans. In particular, we develop a novel convolutional neural network architecture for semantic segmentation in the framework of Activity of Daily Life (ADL), with focus on semantic segmentation of images in a breakfast scenario. We use RGB and RGB-D images as input signals (images with 4 channels: RGB colors and Depth) and quantify the benefits of the additional information of depth in object classification. As a next step, we evaluate the performance of our deep neural network when trained with data from multiple viewpoints. We make use of two cameras pointing on the table with a different field of view. The first camera is placed on a headset (user’s point of view) and the second on the table.Furthermore, for this project we create a new dataset using two Intel RealSense camera sensors, in the apparatus mentioned above. This dataset is used to train three different convolutional neural networks for semantic segmentation. The first two CNNs are trained with the dataset collected from the headset (one with RGB and one with RGB-D information). The third CNN is trained with the dataset collected from the second camera, and the last one with the dataset from both camera views. Quantifying the importance of multiple viewpoints and depth information in semantic segmentation will help in developing robust, fast, and more precise perception algorithms for object recognition in ADL applications. This is an important step towards the final goal of creating assistive robot agents (e.g. robotic arms, wheelchairs,and exoskeletons) that can help people with severe motor impairments enjoy their first meal of the day.	en
heal.advisorName	Kyriakopoulos, Kostas J.	en
heal.advisorName	Graeser, Axel	en
heal.committeeMemberName	Κυριακόπουλος, Κωνσταντίνος Ι.	el
heal.committeeMemberName	Τζαφέστας, Κωνσταντίνος Σ.	el
heal.committeeMemberName	Παπαδόπουλος, Ευάγγελος Γ.	el
heal.academicPublisher	Εθνικό Μετσόβιο Πολυτεχνείο. Σχολή Μηχανολόγων Μηχανικών	el
heal.academicPublisherID	ntua
heal.numberOfPages	46 σ.	el
heal.fullTextAvailability	false