Regularization methods for high dimensional learning
PhD Program in Computer Science 
School in Sciences and Technologies 
for Information and Knowledge (STIC)
SLIPGURU-DISI-UNIGE IIT
Course at a Glance

We are glad to offer a summer graduate course on Computational Learning. The course covers the foundations as well as the recent advances in Computational Learning with particular emphasis on the analysis of high dimensional data and focusing on a set of core techniques, namely regularization methods. See the synopsis and the syllabus for more details.
The course is co-organized by the SLIPGURU group at the University of Genova and the IIT@MIT Lab, a joint lab between the Istituto Italiano di Tecnologia (IIT) the Massachusetts Institute of Technology (MIT).

Dates and registration

The course will be held on June, 6-10 2011

Registration is free and requires sending an e-mail to one of the instructors by April 30, 2011.

Instructors
Francesca Odone -- University of Genova, odone@disi.unige.it
Lorenzo Rosasco -- Istituto Italiano di Tecnologia (IIT) and Massachusetts Institute of Technology (MIT). , lrosasco@mit.edu
Venue

The course will be held in the Department of Computer Science (DISI) of the University of Genova (see here for travelling information).
Genova
is in the region of Liguria in the Italian Riviera (see here or here for some nice pics)

Click here for suggestions on possible accomodations in Genova.

Synopsis

Classes will be helUnderstanding how intelligence works and how it can be emulated in machines has been an elusive problem for decades and it is arguably one of the biggest challenges in modern science. Learning, its principles, and computational implementations are at the very core of this endeavor. Only recently we have been able, for the first time, to develop  artificial intelligence systems able to solve complex tasks that were considered out of reach for several decades. Modern cameras can recognize faces, and smart phones recognize people voice, car provided with cameras can detect pedestrians and ATM machines can automatically read checks.  In most cases at the root of these success stories there are machine learning algorithms, that is, softwares that  are trained rather than programmed to solve a task. 

Among the variety of approaches and ideas in modern computational learning, we focus on a core class of methods, namely regularization methods, which represent a  fundamental set of concepts and techniques allowing to treat in a unified way a huge  class of diverse approaches, while providing the tools to design new ones. Starting from classical notions of smoothness, shrinkage and margin,  we will cover state of the art techniques based on the concepts of geometry  (e.g. manifold learning), sparsity, low rank, allowing to design algorithm for tasks such as supervised learning,  feature selection, structured prediction, multitask learning and model selection.
Practical applications will be discussed, primarily from the field of computational vision. 
   
The classes will focus on algorithmic and methodological aspects, while trying to give an idea of the underlying theoretical underpinnings. Practical laboratory sessions will give the opportunity to have hands on experience.

Slides of the classes will be posted on this website and scribes of most classes, as well as other material, can be found on the 9.520 course webpage at MIT.

Syllabus

- each class is 90 min. no breaks -

class 1 (C1) Welcome. Introduction to Learning [part1] [part2]
class 2 (C2) RKHS and Tikhonov Regularization
class 3 (C3) Spectral Methods for Supervised Learning
class 4 (C4) Error Analysis and Parameter Choice
class 5 (C5) Lab 1 - Binary classification and model selection
class 6 (C6) Sparsity Based Learning and Variable Selection
class 7 (C7)
Regularization with multiple kernels
class 8 (C8) Lab 2 - Sparsity based methods
class 9 (C9) Manifold Regularization
class 10 (C10) 
Regularization for Multi-Output Learning
class 11 (C11) Lab 3 - Manifold regularization
class 12 (C12) Applications to high dimensional problems
class 13 (C13) Lab 4 - Applications

Course schedule and rooms

MON 6 TUE 7 WED 8 THU 9 FRI 10
9:30-11:00
-
C3
C6
C9
C12
11:30-13:00
-
C4
C7
C10
C13 (lab)
14:30-16:00
C1 - 
C5 (lab)
C8 (lab)
C11 (lab)
-
16:30-18:00
C2
-
-
-
-

- room 322 (sala conferenze) - DISI 3rd floor [NOTICE THERE HAS BEEN A CHANGE HERE!]
- lab SW2 - DISI 3rd floor

Prerequisites
Multivariate Calculus, Basic Probability Theory, Matlab.
Short reading list

General references are

  • Bousquet, O., S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning Lecture Notes in Artificial Intelligence 3176, 169-207. (Eds.) Heidelberg, Germany (2004)
  • F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002.
  • T. Evgeniou and M. Pontil and T. Poggio. Regularization Networks and Support Vector Machines. Advances in Computational Mathematics, 2000.
  • T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 2003
  • L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1997.
  • V. N. Vapnik. Statistical Learning Theory. Wiley, 1998.
  • T. Hastie, R. Tibshirani, J. H. Friedman. The Elements of Statistical Learning, Springer 2001.
  • I. Steinwart and A. Christmann. Support vector machines. Springer, New York, 2008.
  • Cucker, Felipe; Zhou, Ding-Xuan Learning theory: an approximation theory viewpoint. 
    With a foreword by Stephen Smale. Cambridge Monographs on Applied and Computational Mathematics.
    Cambridge University Press, Cambridge, 2007. xii+224 pp. 
SLIPGURU-DISI-UNIGE IIT