Scene graph retrieval using contrastive learning in graph neural networks

Boufalis, Odyssefs Dimitrios; Μπούφαλης, Οδυσσεύς Δημήτριος

dc.contributor.author	Boufalis, Odyssefs Dimitrios	en
dc.contributor.author	Μπούφαλης, Οδυσσεύς Δημήτριος	el
dc.date.accessioned	2024-06-19T08:18:25Z
dc.date.available	2024-06-19T08:18:25Z
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/59746
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.27442
dc.rights	Default License
dc.subject	Graphs	en
dc.subject	Graph Kernels	en
dc.subject	Graph Neural Networks	en
dc.subject	Contrastive Learning	en
dc.subject	Scene Graph Similarity	en
dc.subject	Γράφοι	el
dc.subject	Πυρήνες Γράφων	el
dc.subject	Νευρωνικά Δίκτυα Γράφων	el
dc.subject	Αντιθετική Μάθηση	el
dc.subject	Ομοιότητα Γράφων Σκηνής	el
dc.title	Scene graph retrieval using contrastive learning in graph neural networks	en
heal.type	bachelorThesis
heal.classification	Machine Learning, Deep Learning	el
heal.language	el
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2024-03-26
heal.abstract	Graph Neural Networks (GNNs) have emerged as a transformative paradigm in various domains due to their remarkable ability to model complex relationships inherent in graph-structured data. The representation power of GNNs extends across diverse fields such as social network analysis, bioinformatics, recommendation systems and molecular sciences among others. Traditionally, in order to tackle the well known graph similarity problem, algorithms approximating Graph Edit Distance (GED) as well as Graph Kernels have been widely used. Recently, the advancement of deep learning techniques for graph-structured data has given rise to graph based neural approaches for the graph similarity problem. In this context, GNNs have been proven to be particularly potent, demonstrating the capability to capture intricate structural patterns and semantic relationships within graphs. This diploma thesis delves into the representation power of Graph Neural Networks (GNNs) trained within the Contrastive Learning Framework for scene graph retrieval, a task pivotal for comprehensive scene understanding. Leveraging the capabilities of GNNs in capturing complex relationships, the study employs well-established unsupervised contrastive learning techniques to produce high quality and distance preserving graph embeddings. Additionally, a rank aware weak supervised contrastive learning loss is introduced to further enhance the retrieval metrics of these models. Ground truth for evaluation is established using approximate Graph Edit Distance (GED) algorithms, with a focus on the bipartite matching algorithm. The experimental results showcase the superior performance of the proposed contrastive learning models in approximating the GED ground truth compared to well known Graph Kernels, validating the effectiveness of Contrastive GNNs in capturing both subtle relationships and the semantic contents of scene graphs. Given their superiority in producing high-quality embeddings, GNNs can be then used to provide Counterfactual Explanations by leveraging their adeptness in graph retrieval tasks. These models enable the extraction of the most similar scene graph from another class in response to a query scene graph. This capability serves as a powerful tool for semantically explaining the differential classification of the underlying pair of images from which the scene graphs have been generated. By uncovering and highlighting the subtle structural nuances within the graphs that contribute to dissimilar classifications, GNN-based counterfactual explanations offer valuable insights into the decision-making processes of the model, promoting a deeper understanding of the semantic disparities between images and enhancing interpretability in machine learning systems.	en
heal.advisorName	Βουλόδημος, Αθανάσιος	el
heal.committeeMemberName	Βουλόδημος, Αθανάσιος	el
heal.committeeMemberName	Στάμου, Γεώργιος	el
heal.committeeMemberName	Σταφυλοπάτης, Ανδρέας Γεώργιος	el
heal.academicPublisher	Εθνικό Μετσόβιο Πολυτεχνείο. Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών	el
heal.academicPublisherID	ntua
heal.fullTextAvailability	false
heal.fullTextAvailability	false