Σχεδίαση και αξιολόγηση τεχνικών προσεγγιστικού υπολογισμού σε βαθιά νευρωνικά δίκτυα

Μακρής, Γεώργιος; Makris, Georgios

dc.contributor.author	Μακρής, Γεώργιος	el
dc.contributor.author	Makris, Georgios	en
dc.date.accessioned	2021-12-14T09:27:53Z
dc.date.available	2021-12-14T09:27:53Z
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/54166
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.21864
dc.rights	Αναφορά Δημιουργού-Μη Εμπορική Χρήση-Όχι Παράγωγα Έργα 3.0 Ελλάδα	*
dc.rights.uri	http://creativecommons.org/licenses/by-nc-nd/3.0/gr/	*
dc.subject	Προσεγγιστικοί υπολογισμοί	el
dc.subject	Approximate computing	en
dc.subject	Συνελικτικά νευρωνικά δίκτυα	el
dc.subject	Προσεγγιστικοί πολλαπλασιαστές	el
dc.subject	Εξοικονόμηση ενέργειας	el
dc.subject	Υπολογιστική ακρίβεια	el
dc.subject	Deep neural network	en
dc.subject	Approximate multipliers	en
dc.subject	Energy efficiency	en
dc.subject	Inference accuracy	en
dc.title	Σχεδίαση και αξιολόγηση τεχνικών προσεγγιστικού υπολογισμού σε βαθιά νευρωνικά δίκτυα	el
dc.title	Design and evaluation of approximation techniques using approximate multipliers on deep neural networks	en
heal.type	bachelorThesis
heal.classification	Προσεγγιστικοί υπολογισμοί και βαθιά νευρωνικά δίκτυα	el
heal.classification	Approximate computing and deep neural networks	en
heal.language	el
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2021-06-24
heal.abstract	Over the past decade Convolutional Neural Networks (CNNs) emerged as the state-of- the-art approach to tackle certain Computer Vision problems such as image classi fication and object detection. The state-of-the-art works clearly indicate that neural networks feature an intrinsic error-resilience property. Since they often process noisy or redundant data and their users are willing to accept certain errors in many cases, the principles of approximate computing can be employed in the design of their energy efficient implementations. In this thesis, we target the development of novel approximation techniques that pro- vide a good trade-off between between energy consumption, performance and error, by performing an in-depth design space exploration to nd optimal solutions. In particular, we exploited the open-source library [1] that extends Tensorflow by providing Approximate Convolutional layers i.e., layers with reduced precision implemented using approximate multipliers. We extended this library by designing and developing four new approximation techniques in an effort to fi nd the optimal solutions that achieve a good a good trade-off between between energy consumption and inference accuracy. In order to develop this techniques we followed an hierarchical order. More speci fically, in the first technique we followed a non-uniform structure per layer, by replacing the multiplications in various convolutional layers with diverse approximate components while maintaining the accurate multiplications in other layers. In the second technique, we split the number of fi lters in each layer into k equivalent parts and assign in each of these parts a different approximate multiplier. In the third technique, we performed approximations inside the filters by either replacing the multiplications i.e., partial products, with diverse approximate components or simply skipping this operations i.e, not executing them at all. In the fourth and final approximation technique we observed that the fi lter weights of each layer follow a normal distribution and based on this we proposed to execute only the multiplications that have filter weights that belong in either this range [μ-σ,μ+σ] or this [μ-2σ,μ+2σ]. The evaluation of our proposed techniques is performed in Tensorflow with Resnet-8 using the validation set from CIFAR-10 and three inexact multipliers with different perforation, by examining the inference accuracy and energy for the inference of one input image. The final results show that the third and second technique are the best since they provide a signifi cant energy saving up to 33:5% and 30:4% compared to the accurate implementation respectively with a negligible drop in the inference accuracy. Finally, we compared the inference accuracy and the energy when some approximate multipliers from the EvoApproxLib[2] were employed in the Resnet-8 with the inference accuracy and energy that are provided by are best approximation techniques. The comparisons showed that our solutions are far better in terms of both inference accuracy and energy.	en
heal.advisorName	Σούντρης, Δημήτριος	el
heal.committeeMemberName	Τσανάκας, Παναγιώτης	el
heal.committeeMemberName	Πνευματικάτος, Διονύσιος	el
heal.academicPublisher	Εθνικό Μετσόβιο Πολυτεχνείο. Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών. Τομέας Τεχνολογίας Πληροφορικής και Υπολογιστών. Εργαστήριο Μικροϋπολογιστών και Ψηφιακών Συστημάτων VLSI	el
heal.academicPublisherID	ntua
heal.numberOfPages	164 σ.	el
heal.fullTextAvailability	false