Representation and Generalization Properties of Class-Entropy Networks
Sandro Ridella, Stefano Rovetta, and Rodolfo Zunino

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 1, JANUARY 1999
Pages: 31-47

Abstract -

Using conditional class entropy (CCE) as a cost function allows feedforward
networks to fully exploit classification-relevant information. CCE-based
networks arrange the data space into partitions, which are assigned unambiguous
symbols and are labeled by class information. By this labeling mechanism
the network can model the empirical data distribution at the local level.
Region labeling evolves with the network-training process, which follows
a plastic algorithm. The paper proves several theoretical properties about
the performance of CCE-based networks, and considers both convergence during
training and generalization ability at run-time. In addition, analytical criteria
and practical procedures are proposed to enhance the generalization performance
of the trained networks. Experiments on artificial and real-world domains
confirm the accuracy of this class of networks and witness the validity
of the described methods.


Index Terms—

Class-entropy networks, clustering methods, generalization, minimum entropy methods,
noise-injection, pruning