Algorithm Design for Reliable Machine Learning

Καλαβάσης, Αλβέρτος; Kalavasis, Alvertos

dc.contributor.author	Καλαβάσης, Αλβέρτος	el
dc.contributor.author	Kalavasis, Alvertos	en
dc.date.accessioned	2023-09-19T09:12:18Z
dc.date.available	2023-09-19T09:12:18Z
dc.identifier.uri	https://dspace.lib.ntua.gr/xmlui/handle/123456789/58060
dc.identifier.uri	http://dx.doi.org/10.26240/heal.ntua.25757
dc.rights	Αναφορά Δημιουργού-Μη Εμπορική Χρήση-Όχι Παράγωγα Έργα 3.0 Ελλάδα	*
dc.rights.uri	http://creativecommons.org/licenses/by-nc-nd/3.0/gr/	*
dc.subject	Theoretical Machine Learning	en
dc.title	Algorithm Design for Reliable Machine Learning	en
heal.type	doctoralThesis
heal.secondaryTitle	Σχεδιασμός Αλγορίθμων για Αξιόπιστη Μηχανική Μάθηση	el
heal.classification	Theoretical Computer Science	en
heal.language	en
heal.access	free
heal.recordProvider	ntua	el
heal.publicationDate	2023-06-14
heal.abstract	In this thesis we theoretically study questions in the area of Reliable Machine Learning in order to design algorithms that are robust to bias and noise (Robust Machine Learning) and satisfy societal desiderata such as privacy and reproducibility (Responsible Machine Learning). In the area of Robust Machine Learning, we design computationally efficient algorithms for problems in the fields of Truncated Statistics, Censored Statistics and Robust Statistics. In particular, we provide the first efficient methods for truncated distribution learning in discrete settings and perfect data sampling from truncated data. Next, we study the fundamental problem of learning from partial/coarse labels. Our main algorithmic result is that essentially any problem learnable from fine grained labels can also be learned efficiently when the coarse data are sufficiently informative. We obtain our result through a generic reduction for answering Statistical Queries (SQ) over fine grained labels given only coarse labels. We also study the central problem in Censored Statistics of Gaussian mean estimation from coarse data. Finally, we consider the problem of learning linear sorting functions in the presence of bounded noise, a problem that generalizes the problem of learning halfspaces with Massart noise. In the area of Responsible Machine Learning, we study the notion of replicability as an algorithmic property and introduce the notion of replicable policies in the context of stochastic bandits, one of the canonical problems in interactive learning. We show that not only do replicable policies exist, but also they achieve almost the same optimal (non-replicable) regret bounds in terms of the time horizon. Lastly, we establish information-theoretic equivalences between notions of algorithmic stability such as replicability and approximate differential privacy. We do so by focusing on the following question: When two different parties use the same learning rule on their own data, how can we test whether the distributions of the two outcomes are similar? We study the similarity of outcomes of learning rules through the lens of the Total Variation (TV) distance of distributions. We say that a learning rule is TV indistinguishable if the expected TV distance between the posterior distributions of its outputs, executed on two training data sets drawn independently from the same distribution, is small. We first investigate the learnability of hypothesis classes using TV indistinguishable learners. Our main results are information-theoretic equivalences between TV indistinguishability and existing algorithmic stability notions such as replicability and approximate differential privacy.	en
heal.advisorName	Φωτάκης, Δημήτριος
heal.committeeMemberName	Φωτάκης, Δημήτριος
heal.committeeMemberName	Τζάμος, Χρήστος
heal.committeeMemberName	Παγουρτζής, Αριστείδης
heal.committeeMemberName	Ιωαννίδης, Στρατής
heal.committeeMemberName	Αχλιόπτας, Δημήτριος
heal.committeeMemberName	Λουλάκης, Μιχαήλ
heal.committeeMemberName	Συμβώνης, Αντώνιος
heal.academicPublisher	Εθνικό Μετσόβιο Πολυτεχνείο. Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών. Τομέας Ηλεκτρομαγνητικών Εφαρμογών Ηλεκτροοπτικής και Ηλεκτρονικών Υλικών	el
heal.academicPublisherID	ntua
heal.fullTextAvailability	false