End of Discriminant Functions based on Variance-covariance Matrices

Shuichi Shinmura

2014

Abstract

Fisher proposed a linear discriminant function (LDF) based on the maximization of the variance ratio. If data satisfies Fisher’s assumption, the same LDF is easily derived from a variance-covariance matrix. When the variance-covariance matrices of two classes are not the same, a quadratic discriminant function (QDF) can be used. These discriminant functions have three problems. First, none of them can correctly discriminate between xi on the discriminant hyperplane (the unresolved problem of discriminant analysis). Second, LDF and QDF cannot always recognize linear separable data, and the number of misclassifications (NM) made by these functions is usually higher than that of logistic regression. Third, these functions are not obtained if the value of some independent variable is constant, because the inverse matrix cannot be calculated. These facts mean that LDF and QDF should not be used for important discriminations. On the contrary, a revised Optimal Linear Discriminant Function by Integer Programming (Revised IP-OLDF) based on the Minimum NM (MNM) criterion resolves these problems completely. In addition, the mean error rate of Revised IP-OLDF is often less than those of LDF, logistic regression, and Support Vector Machines (SVM) under 100-fold cross-validation.

References

  1. Edgar, A., 1935. The irises of the Gaspe Peninsula. Bulltin of the American Iris Society, 59, 2-5.
  2. Fisher, R. A., 1936. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7, 179- 188.
  3. Flury, B., Rieduyl, H., 1988. Multivariate Statistics: A Practical Approach. Cambridge University Press.
  4. Friedman, J. H., 1989 . Regularized Discriminant Analysis . Journal of the American Statistical Association,84/405, 165-175.
  5. Goodnight, J. H., 1978. SAS Technical Report - The Sweep Operator: Its Importance in Statistical Computing - (R-100). SAS Institute Inc.
  6. Ibaraki,T., Muroga, S., 1970. Adaptive linear classifier by linear programming. IEEE transaction On systems science and cybernetics, SSC-6, 53-62.
  7. Lachenbruch, P. A., Mickey, M. R., 1968. Estimation of error rates in discriminant analysis. Technometrics 10, 1-11.
  8. Liitschwager, J. M., Wang, C., 1978. Integer programming solution of a classification problem. Management Science, 24/14, 1515-1525.
  9. Sall, J. P., 1981. SAS Regression Applications. SAS Institute Inc. (Japanese version is translated by Shinmura, S.)
  10. Sall, J. P., Creighton, L., Lehman, A., 2004. JMP Start Statistics, Third Edition. SAS Institute Inc. (Japanese version is edited by Shinmura, S.)
  11. Schrage, L., 1991. LINDO-An Optimization Modeling System-. The Scientific Press. (Japanese version is translated by Shinmura, S., & Takamori, H.)
  12. Schrage, L., 2006. Optimization Modeling with LINGO. LINDO Systems Inc. (Japanese version is translated by Shinmura, S.)
  13. Shinmura, S., Miyake, A., 1979. Optimal linear discriminant functions and their application. COMPSAC79, 167-172.
  14. Shinmura, S., 1998. Optimal Linear Discrimrnant Functions using Mathematical Programming. Journal of the Japanese Society of Computer Statistics, 11 / 2 , 89-101.
  15. Shinmura, S., 2000. A new algorithm of the linear discriminant function using integer programming. New Trends in Probability and Statistics, 5, 133-142.
  16. Shinmura, S., 2004. New Algorithm of Discriminant Analysis using Integer Programming. IPSI 2004 Pescara VIP Conference CD-ROM, 1-18.
  17. Shinmura, S., 2007. Overviews of Discriminant Function by Mathematical Programming. Journal of the Japanese Society of Computer Statistics, 20/1-2, 59- 94.
  18. Shinmura, S., 2009. Improvement of CPU time of Revised IPLP-OLDF using Linear Programming. Journal of the Japanese Society of Computer Statistics, 22/1, 37- 57.
  19. Shinmura, S., 2010. The optimal linear discriminant function. Union of Japanese Scientist and Engineer Publishing (in Japanese).
  20. Shinmura, S., 2011a.Problems of Discriminant Analysis by Mark Sense Test Data . Japanese Society of Applied Statistics,40/3,157-172.
  21. Shinmura, S., 2011b. Beyond Fisher's Linear Discriminant Analysis - New World of Discriminant Analysis -. 2011 ISI CD-ROM, 1-6.
  22. Shinmura, S., 2013. Evaluation of Optimal Linear Discriminant Function by 100-fold cross-validation. 2013 ISI CD-ROM, 1-6.
  23. Stam, A., 1997. Nontraditinal approaches to statistical classification: Some perspectives on Lp-norm methods. Annals of Operations Research, 74, 1-36.
  24. Markowitz, H. M., 1959. Portfolio Selection, Efficient Diversification of Investment. John Wiley & Sons, Inc.
  25. Miyake, A., Shinmura, S., 1976. Error rate of linear discriminant function, F.T. de Dombal & F. Gremy editors 435-445, North-Holland Publishing Company.
  26. Vapnik, V., 1995. The Nature of Statistical Learning Theory.Springer-Verlag.
  27. Warmack, R. E., Gonzalez, R. C., 1973. An algorithm for the Optimal Solution of Linear Inequalities and its Application to Pattern Recognition. IEEE Transaction on computing, C-22/12, 1065-1075.
Download


Paper Citation


in Harvard Style

Shinmura S. (2014). End of Discriminant Functions based on Variance-covariance Matrices . In Proceedings of the 3rd International Conference on Operations Research and Enterprise Systems - Volume 1: ICORES, ISBN 978-989-758-017-8, pages 5-16. DOI: 10.5220/0004699000050016


in Bibtex Style

@conference{icores14,
author={Shuichi Shinmura},
title={End of Discriminant Functions based on Variance-covariance Matrices},
booktitle={Proceedings of the 3rd International Conference on Operations Research and Enterprise Systems - Volume 1: ICORES,},
year={2014},
pages={5-16},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004699000050016},
isbn={978-989-758-017-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Operations Research and Enterprise Systems - Volume 1: ICORES,
TI - End of Discriminant Functions based on Variance-covariance Matrices
SN - 978-989-758-017-8
AU - Shinmura S.
PY - 2014
SP - 5
EP - 16
DO - 10.5220/0004699000050016