Abstractive text summarization: enhancing sequence to sequence models using word sense disambiguation and semantic content generalization

Kouris, Panagiotis; Alexandridis, Georgios; Stafylopatis, Andreas

dc.contributor.author	Kouris, Panagiotis
dc.contributor.author	Alexandridis, Georgios
dc.contributor.author	Stafylopatis, Andreas
dc.date.accessioned	2022-12-15T11:56:13Z
dc.date.available	2022-12-15T11:56:13Z
dc.identifier.issn	0891-2017	el
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/56458
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.24156
dc.rights	Default License
dc.subject	Abstractive Text Summarization	el
dc.subject	Machine Learning	el
dc.subject	Deep Learning	el
dc.subject	Natural Language Processing	el
dc.title	Abstractive text summarization: enhancing sequence to sequence models using word sense disambiguation and semantic content generalization	en
heal.type	journalArticle
heal.classification	Automatic text summarization	el
heal.classification	Artificial intelligence	el
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2021-12-23
heal.bibliographicCitation	Panagiotis Kouris, Georgios Alexandridis, and Andreas Stafylopatis. Abstractive text summarization: enhancing sequence to sequence models using word sense disambiguation and semantic content generalization. In: Computational Linguistics, pp. 1-41, MIT Press, 2021.	en
heal.abstract	Nowadays, most research conducted in the field of abstractive text summarization focuses on neural-based models alone, without considering their combination with knowledge-based that could further enhance their efficiency. In this direction, this work presents a novel framework that combines sequence to sequence neural-based text summarization along with structure and semantic-based methodologies. The proposed framework is capable of dealing with the problem of out-of-vocabulary or rare words, improving the performance of the deep learning models. The overall methodology is based on a well defined theoretical model of knowledge-based content generalization and deep learning predictions for generating abstractive summaries. The framework is comprised of three key elements: (i) a pre-processing task, (ii) a machine learning methodology and (iii) a post-processing task. The pre-processing task is a knowledge-based approach, based on ontological knowledge resources, word-sense-disambiguation and named-entity recognition, along with content generalization, that transforms ordinary text into a generalized form. A deep learning model of attentive encoder-decoder architecture, which is expanded to enable a coping and coverage mechanism, as well as reinforcement learning and transformer-based architectures, is trained on a generalized version of text-summary pairs, learning to predict summaries in a generalized form. The post-processing task utilizes knowledge resources, word embeddings, word-sense disambiguation and heuristic algorithms based on text similarity methods in order to transform the generalized version of a predicted summary to a final, human-readable form. An extensive experimental procedure on three popular datasets evaluates key aspects of the proposed framework, while the obtained results exhibit promising performance, validating the robustness of the proposed approach.	en
heal.publisher	MIT Press	en
heal.journalName	Computational Linguistics	en
heal.journalType	peer-reviewed
heal.fullTextAvailability	false
dc.identifier.doi	https://doi.org/10.1162/coli_a_00417	el