# MULTI-CLASS FROM BINARY - Divide to conquer

### Anderson Rocha, Siome Goldenstein

#### Abstract

Several researchers have proposed effective approaches for binary classification in the last years. We can easily extend some of those techniques to multi-class. Notwithstanding, some other powerful classifiers (e.g., SVMs) are hard to extend to multi-class. In such cases, the usual approach is to reduce the multi-class problem complexity into simpler binary classification problems (divide-and-conquer). In this paper, we address the multi-class problem by introducing the concept of affine relations among binary classifiers (dichotomies), and present a principled way to find groups of high correlated base learners. Finally, we devise a strategy to reduce the number of required dichotomies in the overall multi-class process.

#### References

- Allwein, E., Shapire, R., and Singer, Y. (2000). Reducing multi-class to binary: A unifying approach for margin classifiers. JMLR, 1(1):113-141.
- Anand, R., Mehrotra, K., Mohan, C., and Ranka, S. (1995). Efficient classification for multi-class problems using modular neural networks. TNN, 6(1):117-124.
- Athisos, V., Stefan, A., Yuan, Q., and Sclaroff, S. (2007). Classmap: Efficient multiclass recognition via embeddings. In ICCV.
- Bishop, C. (2006). Pattern Recognition and Machine Learning. Springer, 1 edition.
- Clark, P. and Boswell, R. (1991). Rule induction with CN2: Some improvements. In EWSL, pages 151-163.
- Crammer, K. and Singer, Y. (2002). On the learnability and design of output codes for multi-class problems. JMLR, 47(2-3):201-233.
- Dietterich, T. and Bakiri, G. (1996). Solving multi-class problems via ECOC. JAIR, 2(1):263-286.
- Klautau, A., Jevtic, N., and Orlitsky, A. (2004). On nearestneighbor ECOC with application to all-pairs multiclass support vector machines. JMLR, 4(1):1-15.
- Moreira, M. and Mayoraz, E. (1998). Improved pairwise coupling classification with correcting classifiers. In ECML.
- Narasimhamrthy, A. (2005). Theoretical bounds of majority voting performance for a binary classification problem. TPAMI, 27(12):1988-1995.
- Passerini, A., Pontil, M., and Frasconi, P. (2004). New results on error correcting output codes of kernel machines. TNN, 15(1):45-54.
- Pedrajas, N. and Boyer, D. (2006). Improving multi-class pattern recognition by the combination of two strategies. TPAMI, 28(6):1001-1006.
- Platt, J., Christiani, N., and Taylor, J. (1999). Large margin dags for multi-class classification. In NIPS, pages 547-553.
- Pujol, O., Radeva, P., and Vitria, J. (2006). Discriminant ECOC: A heuristic method for application dependent design of ECOC. TPAMI, 28(6):1007-1012.
- Stehling, R., Nascimento, M., and Falca˜o, A. (2002). A compact and efficient image retrieval approach based on border/interior pixel classification. In CIKM, pages 102-109.
- Takenouchi, T. and Ishii, S. (2007). Multi-class classification as a decoding problem. In FOCI, pages 470-475.
- Windeatt, T. and Ghaderi, R. (2003). Coding and decoding strategies for multi-class learning problems. Information Fusion, 4(1):11-21.
- Young, C., Yen, C., Pao, Y., and Nagurka, M. (2006). Oneclass-at-time removal sequence planning method for multi-class problems. TNN, 17(6):1544-1549.

#### Paper Citation

#### in Harvard Style

Rocha A. and Goldenstein S. (2009). **MULTI-CLASS FROM BINARY - Divide to conquer** . In *Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)* ISBN 978-989-8111-69-2, pages 323-330. DOI: 10.5220/0001777803230330

#### in Bibtex Style

@conference{visapp09,

author={Anderson Rocha and Siome Goldenstein},

title={MULTI-CLASS FROM BINARY - Divide to conquer},

booktitle={Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)},

year={2009},

pages={323-330},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0001777803230330},

isbn={978-989-8111-69-2},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)

TI - MULTI-CLASS FROM BINARY - Divide to conquer

SN - 978-989-8111-69-2

AU - Rocha A.

AU - Goldenstein S.

PY - 2009

SP - 323

EP - 330

DO - 10.5220/0001777803230330