×

zbMATH — the first resource for mathematics

Information geometry of \(U\)-Boost and Bregman divergence. (English) Zbl 1102.68489
Summary: We aim at an extension of AdaBoost to \({\mathcal U}\)-Boost, in the paradigm to build a stronger classification machine from a set of weak learning machines. A geometric understanding of the Bregman divergence defined by a generic convex function \({\mathcal U}\) leads to the \({\mathcal U}\)-Boost method in the framework of information geometry extended to the space of the finite measures over a label set. We propose two versions of \({\mathcal U}\)-Boost learning algorithms by taking account of whether the domain is restricted to the space of probability functions. In the sequential step, we observe that the two adjacent and the initial classifiers are associated with a right triangle in the scale via the Bregman divergence, called the Pythagorean relation. This leads to a mild convergence property of the \({\mathcal U}\)-Boost algorithm as seen in the expectation-maximization algorithm. Statistical discussions for consistency and robustness elucidate the properties of the \({\mathcal U}\)-Boost methods based on a stochastic assumption for training data.

MSC:
68Q32 Computational learning theory
68T05 Learning and adaptive systems in artificial intelligence
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] DOI: 10.1016/0893-6080(95)00003-8 · Zbl 05479131
[2] DOI: 10.1109/18.256500 · Zbl 0818.68126
[3] DOI: 10.1214/aos/1016218223 · Zbl 1106.62323
[4] DOI: 10.1162/089976602760128045 · Zbl 1019.94506
[5] DOI: 10.1016/0893-6080(96)00000-7
[6] DOI: 10.1109/72.329683
[7] Schapire R. E., Machine Learning 5 pp 197– (1990)
[8] DOI: 10.1214/aos/1024691352 · Zbl 0929.62069
[9] DOI: 10.1162/089976604322860695 · Zbl 1097.68608
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.