Deep multimodal fusion of image and non-image data in identification of high-risk carotid atheromatous plaque

Stamos, Evangelos; Στάμος, Ευάγγελος

dc.contributor.author	Stamos, Evangelos	en
dc.contributor.author	Στάμος, Ευάγγελος	el
dc.date.accessioned	2026-02-26T06:55:24Z
dc.date.available	2026-02-26T06:55:24Z
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/63599
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.31294
dc.rights	Default License
dc.subject	Carotid atherosclerosis	en
dc.subject	Plaque vulnerability	en
dc.subject	Deep learning	en
dc.subject	Multimodal fusion	en
dc.subject	Stroke prevention	en
dc.title	Deep multimodal fusion of image and non-image data in identification of high-risk carotid atheromatous plaque	en
heal.type	masterThesis
heal.classification	Deep Learning	en
heal.classification	Carotid Atherosclerosis	en
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2025-07-04
heal.abstract	Cardiovascular diseases remain the leading cause of global mortality, accounting for over 19 million deaths annually according to the World Health Organization. Among these, carotid atherosclerosis, a pathological process characterized by plaque accumulation in the carotid arteries, is a major contributor to ischemic stroke, driven by plaque rupture and thromboembolic events. Traditional diagnostic approaches, which rely on single-modality imaging or clinical data, often fail to capture the complex interplay of morphological, biological, and hemodynamic factors that determine plaque vulnerability. This dissertation addresses this critical gap by developing advanced deep multimodal fusion frameworks to integrate heterogeneous data sources, enabling precise risk stratification of carotid atheromatous plaques. The primary objective of this work is to create an end-to-end trainable system that synergistically combines B-mode carotid ultrasound imaging with non-image clinical data, including biochemical markers, protein biomarkers, and patient demographics. Three fusion strategies were rigorously investigated: (1) Joint Attention-Based Fusion, which dynamically weights imaging and tabular data contributions through learned attention mechanisms; (2) Early Fusion, merging raw inputs at the feature level; and (3) AttentionGated Video Hybrid Fusion, a novel architecture designed to process spatiotemporal ultrasound frame sequences alongside clinical data. These models were trained and validated on a multimodal dataset comprising 96 DICOM ultrasound recordings and 73 curated clinical profiles from a cohort of 73 patients, stratified into high-risk (symptomatic with ≥ 50% stenosis or asymptomatic with ≥ 70% stenosis) and low-risk groups. The Joint Attention-Based Fusion model with an EfficientNet-B0 backbone achieved superior performance, yielding AUC: 86.07%, Balanced Accuracy: 73.28%, F1 Score: 78.42 %, Sensitivity: 81.67%, outperforming unimodal approaches (imaging-only AUC: 84.55%; tabular-only AUC: 64.61%). Attention weights highlighted the dominance of imaging data (69.4% contribution), while clinical biomarkers provided complementary risk context. The proposed Attention-Gated Video Hybrid Fusion framework demonstrated feasibility for dynamic plaque analysis but faced computational constraints in scaling 3D spatiotemporal convolutions. In conclusion, this thesis advances the field of precision vascular medicine by demonstrating that multimodal deep learning can substantially improve the accuracy and reliability of carotid atheromatous plaque risk assessment compared to conventional single-modality approaches. The proposed architectures, particularly the attention-based fusion paradigm, provide a scalable framework for integrating heterogeneous data sources. Additionally, a prototype Clinical Decision Support System (CDSS) was developed as a web-based interface, illustrating the translational potential of these models for future integration into clinical workflows. Together, these contributions highlight the promise of AI-driven methodologies in enhancing stroke prevention strategies and supporting individualized therapeutic decision-making.	en
heal.advisorName	Nikita, Konstantina	en
heal.committeeMemberName	Stamou, Giorgos	en
heal.committeeMemberName	Voulodimos, Athanasios	en
heal.academicPublisher	Εθνικό Μετσόβιο Πολυτεχνείο. Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών	el
heal.academicPublisherID	ntua
heal.numberOfPages	119 σ.	el
heal.fullTextAvailability	false