|
|
AIMS
In the last decade, ensemble methods have shown to be effective in many
application domains and constitute one of the main current directions in
Machine Learning research. This school will address from a theoretical
and empirical view point, several important questions concerning the combination
of Learning Machines. In particular, different approaches to the problem
which have been proposed in the context of Machine Learning, Neural Networks,
and Statistical Pattern Recognition will be discussed. Moreover, a special
stress will be given to theoretical and practical tools to develop ensemble
methods and evaluate their applications on real-world domains, such as
Remote Sensing, Bioinformatics and Medical field.
GENERALITIES
Recently, driven by application needs, multiple classifier combinations
have evolved into a practical and effective solution for real-world pattern
recognition tasks. The idea appears in various disciplines (including Machine
Learning, Neural Networks, Pattern Recognition, and Statistics) under several
names: hybrid methods, combining decisions, multiple experts, mixture of
experts, sensor fusion and many more. In some cases, the combination is
motivated by the simple observation that classifier performance is not
uniform across the input space and different classifiers excel in different
regions. Under a Bayesian framework, integrating over expert distribution
leads naturally to expert combination. The generalization capabilities
of ensembles of learning machines have been interpreted in the framework
of Statistical Learning Theory and in the related theory of Large Margin
Classifiers.
There are several ways to use more than one classifier in a classification
problem. A first "averaging" approach consists of generating multiple hypotheses
from a single or multiple learning algorithms, and combining them through
majority voting or different linear and non linear combinations. A "feature-oriented"
approach is based on different methods to build ensembles of learning machines
by subdividing the input space (e.g., random subspace methods, multiple
sensors fusion, feature transformation fusion). "Divide-and-conquer" approaches
isolate the regions in input space on which each classifier performs well,
and direct new input accordingly, or subdivide a complex learning problem
in a set of simpler subproblems, recombining them using suitable decoding
methods. A "sequential-resampling" approach builds multiple classifier
systems using bootstrap methods in order to reduce variance (bagging) or
jointly bias and unbiased variance (boosting).
There are fundamental questions that need to be addressed for a practical
use of this collection of approaches: What are the theoretical tools to
interpret possibly in a unified framework this multiplicity of ensemble
methods? What is gained and lost in a combination of experts, when is it
preferable to alternative approaches? What types of data are best suitable
to expert combination? What types of experts are best suited for combinations?
What are optimal training methods for experts which are expected to participate
in a collective decision? What combination strategies are best suited to
a particular problem and to a particular distribution of the data? What
are the statistical methods and the appropriate benchmark data to evaluate
multiclassifier systems?
The school will address some of the above questions from a theoretical
and empirical view point and will teach students about this exciting and
very promising field using current state of the art data sets for pattern
recognition, classification and regression.
The main goals of the school are:
1. Offer an overview of the main research issues of ensemble methods from
the different and complementary perspectives of Machine Learning, Neural
Networks, Statistics and Pattern Recognition.
2. Offer theoretical tools to analyze the diverse approaches, and critically
evaluate their applications.
3. Offer practical and theoretical tools to develop new ensemble methods
and analyze their application on real-world problems.
|
