Performance monitoring and workload characterization of big data and cloud based applications on the Intel SCC Manycore Platform

Γεωργιάδης, Ανδρέας - Λάζαρος; Georgiadis, Andreas - Lazaros

dc.contributor.author	Γεωργιάδης, Ανδρέας - Λάζαρος	el
dc.contributor.author	Georgiadis, Andreas - Lazaros	en
dc.date.accessioned	2015-06-17T11:53:01Z
dc.date.available	2015-06-17T11:53:01Z
dc.date.issued	2015-06-17
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/40859
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.9245
dc.rights	Default License
dc.subject	Big data	en
dc.subject	Embedded systems	en
dc.subject	Distributed systems	en
dc.subject	Computer architecture	en
dc.title	Performance monitoring and workload characterization of big data and cloud based applications on the Intel SCC Manycore Platform	en
heal.type	bachelorThesis
heal.classification	Πληροφορική	en
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2015-04-27
heal.abstract	The scope of this Diploma Thesis is to explore several performance, power consumption and scalability aspects of the execution of Big Data and Cloud Based workloads on the Intel Single-chip Cloud Computer Manycore Platform, which differentiates from typical cluster topologies, since it integrates 48 cores on a single chip. The applications we study are implemented using the MapReduce framework on top of the Hadoop Distributed File System. For the purpose of this analysis we have developed a runtime monitoring infrastructure which utilizes Ganglia, a monitoring tool for large clusters. Chapter 1 initially states the importance of studying Cloud Computing and Big Data Applications and presents some basic aspects of the concepts this diploma thesis deals with. This chapter concludes with the contribution this thesis attempts to make in the field of scale-out applications and many-core systems. Chapter 2 describes recent research findings in the related fields of scale-out workloads and performance and power monitoring of the Intel SCC that have provided the background and inspiration for this diploma thesis. Chapter 3 describes the architecture of the Intel SCC in detail, emphasizing on aspects of the platform whose understanding is crucial for application behavior characterization. Chapter 4 presents a detailed analysis of the Hadoop Distributed File System and the MapReduce framework, by discussing key implementation aspects and providing guidelines of how to configure an HDFS cluster installation and tune the execution of MapReduce jobs. Chapter 5 provides a detailed description of the tools that have been used and developed so as to deploy and launch Hadoop Clusters on the Intel SCC. The Runtime Environment setup and the Hadoop Cluster installation processes are described and explained in detail. Chapter 6 presents the Runtime Monitoring Framework we have developed for the Intel SCC. The Ganglia Cluster topology we have configured for the Intel SCC is analyzed and the process of collecting, storing and visualizing runtime metrics is explained. Chapter 7 describes and explains the experimental analysis we have conducted for four MapReduce applications when they run on the Intel SCC. Our investigation is focused on the behavior of those applications for varying input sizes, HDFS cluster topologies and frequency settings for the cluster nodes. Chapter 8 concludes the findings of this diploma thesis and presents suggestions for future work.	en
heal.abstract	Σκοπός αυτής της διπλωματικής εργασίας είναι η μελέτη επίδοσης, κατανάλωσης ισχύος και κλιμακωσιμότητας Big Data και Cloud Based εφαρμογών στην πολυπύρηνη πλατφόρμα Intel SCC. Η πλατφόρμα αυτή διαφοροποιείται από τυπικές τοπολογίες cluster, καθώς ολοκληρώνει 48 πυρήνες πάνω στο ίδιο τσιπ. Οι εφαρμογές που μελετάμε έχουν υλοποιηθεί με Hadoop και MapReduce. Για τους σκοπούς της διπλωματικής έχουμε επίσης αναπτύξει ένα runtime monitoring μηχανισμό, χρησιμοποιώντας το Ganglia, ένα εργαλείο για monitoring μεγάλων clusters.	el
heal.advisorName	Σούντρης, Δημήτριος	el
heal.committeeMemberName	Κοζύρης, Νεκτάριος	el
heal.committeeMemberName	Πεκμετζή, Κιαμάλ	el
heal.academicPublisher	Εθνικό Μετσόβιο Πολυτεχνείο. Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών. Τομέας Τεχνολογίας Πληροφορικής και Υπολογιστών. Εργαστήριο Μικροϋπολογιστών και Ψηφιακών Συστημάτων VLSI	el
heal.academicPublisherID	ntua
heal.numberOfPages	295 σ.
heal.fullTextAvailability	true