# AN ALGORITHM FOR DECISION RULES AGGREGATION

### Adam Gudys, Marek Sikora

#### Abstract

Decision trees and decision rules are usually applied for the classification problems in which legibility and possibility of interpretation of the obtained data model is important as well as good classification abilities. Beside trees, rules are the most frequently used knowledge representation applied by knowledge discovery algorithms. Rules generated by traditional algorithms use conjunction of simple conditions, each dividing input space by a hyperplane parallel to one of the hyperplanes of the coordinate system. There are problems for which such an approach results in a huge set of rules that poorly models real dependencies in data, is susceptible for overtfitting and hard to understand by human. Generating decision rules containing more complicated conditions may improve quality and interpretability of a rule set. In this paper an algorithm taking a set of traditional rules and aggregating them in order to obtain a smaller set of more complex rules has been presented. As procedure uses convex hulls, it has been called Convex Hull-Based Iterative Aggregation Algorithm.

#### References

- Barakat, N., Bradley, A. B., 2006. Rule Extraction from Support Victor Machines: Measuring the Explanation Capability Using the Area under the ROC Curve. In 18th International Conference on Pattern Recognition. Hong Kong, 2006.
- Barber, C. B., Dobkin, D. P., Huh, H., 1996. The Quickhull algorithm for convex hulls. ACM Transactions on Mathematical Software, pp.469-83.
- Bennett, K. P., Blue, J. A., 1997. A support vector machine approach to decision trees. Department of Mathematical Sciences Math Report No. 97-100.
- Cohen, W. W., 1995. Fast effective rule induction. In International Conference on Machine Learning. Tahoe City, 1995. Morgan Kaufmann.
- Fayad, U. M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R., 1996. From data mining to knowledge discovery. In Advances in knowledge discovery and data mining. Cambridge, 1996. AAAI/MIT-Press.
- Ferguson, D. E., 1960. Fibonaccian searching. Communications of the ACM, December. p.648.
- Furnkranz, J., 1999. Separate-and-conquer rule learning. Artificial Intelligence Review, pp.3-54.
- Furnkranz, J., Flach, P. A., 2005. ROC 'n' Rule Learning - Towards a Better Understanding of Covering Algorithms. Machine Learning, pp.39-77.
- Guillet, F., Hamilton, H. J., 2007. Quality measures in data mining. New York: Springer-Verlag.
- Hall, M., Eibe, F., Holmes, G., Pfahringer, B., Reutemann, P., Witte, I. H., 2009. The WEKA Data Mining Software: An Update. SIGKDD Explorations.
- Latkowski, R., Mikolajczyk, M., 2004. Data Decomposition and Decision Rule Joining for Classification of Data with Missing Values. Transactions on Rough Sets I, pp.299-320.
- Martens, D., Baesens, B., Gestel, T. V., 2009. Decompositional Rule Extraction form Support Vector Machines by Active Learning. IEEE Transaction on Knowledge and Data Engineering, pp.178-91.
- Murthy, S. K., Kasif, S., Salzberg, S., 1994. A system for induction oblique decision trees. Journal of Artificial Intelligence Research, pp.1-31.
- Murthy, S. K., Kasif, S., Salzberg, S., 1994. A system for induction oblique decision trees. Journal of Artificial Intelligence Research 2, pp.1-31.
- Nunez, H., Angulo, C., Catalia, A., 2008. Rule extraction on support and prototype vectors. In J. Diederich, ed. Rule extraction form SVM. Springer. pp.109-33.
- Pindur, R., Susmaga, R., Stefanowski, J., 2004. Hyperplane aggregation of dominance decision rules. Fundamenta Informaticae, pp.117-37.
- Seunghyun, I., Ras, Z. W., 2008. Action Rule Extraction from a Decision Table: ARED. In ISMIS. Toronto, 2008. Lecture Notes in Computer Science.
- Sikora, M., 2005. An algorithm for generalization of decision rules by joining. Foundation on Computing and Decision Sciences, Vol. 30, No. 3, pp.227-39.
- Sikora, M., 2006. Rule Quality Measures in Creation and Reduction of Data Rule Models. In Lecture Notes in Artificial Intelligence Vol. 4259. Berlin Heidelberg: Springer-Verlag. pp.716-25.
- Sikora, M., 2010. Decision rules based data models using TRS and NetTRS - method and algorithms. Transactions on Rough Sets XI. Lecture Notes on Computer Sciences Vol.5946, pp.130-60.

#### Paper Citation

#### in Harvard Style

Gudys A. and Sikora M. (2010). **AN ALGORITHM FOR DECISION RULES AGGREGATION** . In *Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2010)* ISBN 978-989-8425-28-7, pages 216-225. DOI: 10.5220/0003089702160225

#### in Bibtex Style

@conference{kdir10,

author={Adam Gudys and Marek Sikora},

title={AN ALGORITHM FOR DECISION RULES AGGREGATION},

booktitle={Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2010)},

year={2010},

pages={216-225},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0003089702160225},

isbn={978-989-8425-28-7},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2010)

TI - AN ALGORITHM FOR DECISION RULES AGGREGATION

SN - 978-989-8425-28-7

AU - Gudys A.

AU - Sikora M.

PY - 2010

SP - 216

EP - 225

DO - 10.5220/0003089702160225