A simple statistical model and association rule filtering for classification

György J. Simon, Vipin Kumar, Peter W. Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Citations (Scopus)

Abstract

Associative classification is a predictive modeling technique that constructs a classifier based on class association rules (also known as predictive association rules; PARs). PARs are association rules where the consequence of the rule is a class label. Associative classification has gained substantial research attention because it successfully joins the benefits of association rule mining with classification. These benefits include the inherent ability of association rule mining to extract high-order interactions among the predictors-an ability that many modern classifiers lack-and also the natural interpretability of the individual PARs. Associative classification is not without its caveats. Association rule mining often discovers a combinatorially large number of association rules, eroding the interpretability of the rule set. Extensive effort has been directed towards developing interestingness measures, which filter (predictive) association rules after they have been generated. These interestingness measures, albeit very successful at selecting interesting rules, lack two features that are highly valuable in the context of classification. First, only few of the interestingness measures are rooted in a statistical model. Given the distinction between a training and a test data set in the classification setting, the ability to make statistical inferences about the performance of the predictive classification rules on the test set is highly desirable. Second, the unfiltered set of predictive assocation rules (PARs) are often redundant, we can prove that certain PARs will not be used to construct a classification model given the presence of other PARs. In this paper, we propose a simple statistical model towards making inferences on the test set about the various performance metrics of predictive association rules. We also derive three filtering criteria based on hypothesis testing, which are very selective (reduce the number of PARs to be considered by the classifier by several orders of magnitude), yet do not effect the performance of the classification adversely. In the case, where the classification model is constructed as a logistic model on top of the PARs, we can mathematically prove, that the filtering criteria do not significantly effect the classifier's performance. We also demonstrate empirically on three publicly available data sets that the vast reduction in the number of PARs indeed did not come at the cost of reducing the predictive performance.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Pages823-831
Number of pages9
DOIs
StatePublished - 2011
Event17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD'11 - San Diego, CA, United States
Duration: Aug 21 2011Aug 24 2011

Other

Other17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD'11
CountryUnited States
CitySan Diego, CA
Period8/21/118/24/11

Fingerprint

Association rules
Classifiers
Statistical Models
Logistics
Labels

ASJC Scopus subject areas

  • Software
  • Information Systems

Cite this

Simon, G. J., Kumar, V., & Li, P. W. (2011). A simple statistical model and association rule filtering for classification. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 823-831) https://doi.org/10.1145/2020408.2020550

A simple statistical model and association rule filtering for classification. / Simon, György J.; Kumar, Vipin; Li, Peter W.

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2011. p. 823-831.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Simon, GJ, Kumar, V & Li, PW 2011, A simple statistical model and association rule filtering for classification. in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 823-831, 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD'11, San Diego, CA, United States, 8/21/11. https://doi.org/10.1145/2020408.2020550
Simon GJ, Kumar V, Li PW. A simple statistical model and association rule filtering for classification. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2011. p. 823-831 https://doi.org/10.1145/2020408.2020550
Simon, György J. ; Kumar, Vipin ; Li, Peter W. / A simple statistical model and association rule filtering for classification. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2011. pp. 823-831
@inproceedings{08e21057e7b440a2872cfc9459e33cc3,
title = "A simple statistical model and association rule filtering for classification",
abstract = "Associative classification is a predictive modeling technique that constructs a classifier based on class association rules (also known as predictive association rules; PARs). PARs are association rules where the consequence of the rule is a class label. Associative classification has gained substantial research attention because it successfully joins the benefits of association rule mining with classification. These benefits include the inherent ability of association rule mining to extract high-order interactions among the predictors-an ability that many modern classifiers lack-and also the natural interpretability of the individual PARs. Associative classification is not without its caveats. Association rule mining often discovers a combinatorially large number of association rules, eroding the interpretability of the rule set. Extensive effort has been directed towards developing interestingness measures, which filter (predictive) association rules after they have been generated. These interestingness measures, albeit very successful at selecting interesting rules, lack two features that are highly valuable in the context of classification. First, only few of the interestingness measures are rooted in a statistical model. Given the distinction between a training and a test data set in the classification setting, the ability to make statistical inferences about the performance of the predictive classification rules on the test set is highly desirable. Second, the unfiltered set of predictive assocation rules (PARs) are often redundant, we can prove that certain PARs will not be used to construct a classification model given the presence of other PARs. In this paper, we propose a simple statistical model towards making inferences on the test set about the various performance metrics of predictive association rules. We also derive three filtering criteria based on hypothesis testing, which are very selective (reduce the number of PARs to be considered by the classifier by several orders of magnitude), yet do not effect the performance of the classification adversely. In the case, where the classification model is constructed as a logistic model on top of the PARs, we can mathematically prove, that the filtering criteria do not significantly effect the classifier's performance. We also demonstrate empirically on three publicly available data sets that the vast reduction in the number of PARs indeed did not come at the cost of reducing the predictive performance.",
author = "Simon, {Gy{\"o}rgy J.} and Vipin Kumar and Li, {Peter W.}",
year = "2011",
doi = "10.1145/2020408.2020550",
language = "English (US)",
isbn = "9781450308137",
pages = "823--831",
booktitle = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

}

TY - GEN

T1 - A simple statistical model and association rule filtering for classification

AU - Simon, György J.

AU - Kumar, Vipin

AU - Li, Peter W.

PY - 2011

Y1 - 2011

N2 - Associative classification is a predictive modeling technique that constructs a classifier based on class association rules (also known as predictive association rules; PARs). PARs are association rules where the consequence of the rule is a class label. Associative classification has gained substantial research attention because it successfully joins the benefits of association rule mining with classification. These benefits include the inherent ability of association rule mining to extract high-order interactions among the predictors-an ability that many modern classifiers lack-and also the natural interpretability of the individual PARs. Associative classification is not without its caveats. Association rule mining often discovers a combinatorially large number of association rules, eroding the interpretability of the rule set. Extensive effort has been directed towards developing interestingness measures, which filter (predictive) association rules after they have been generated. These interestingness measures, albeit very successful at selecting interesting rules, lack two features that are highly valuable in the context of classification. First, only few of the interestingness measures are rooted in a statistical model. Given the distinction between a training and a test data set in the classification setting, the ability to make statistical inferences about the performance of the predictive classification rules on the test set is highly desirable. Second, the unfiltered set of predictive assocation rules (PARs) are often redundant, we can prove that certain PARs will not be used to construct a classification model given the presence of other PARs. In this paper, we propose a simple statistical model towards making inferences on the test set about the various performance metrics of predictive association rules. We also derive three filtering criteria based on hypothesis testing, which are very selective (reduce the number of PARs to be considered by the classifier by several orders of magnitude), yet do not effect the performance of the classification adversely. In the case, where the classification model is constructed as a logistic model on top of the PARs, we can mathematically prove, that the filtering criteria do not significantly effect the classifier's performance. We also demonstrate empirically on three publicly available data sets that the vast reduction in the number of PARs indeed did not come at the cost of reducing the predictive performance.

AB - Associative classification is a predictive modeling technique that constructs a classifier based on class association rules (also known as predictive association rules; PARs). PARs are association rules where the consequence of the rule is a class label. Associative classification has gained substantial research attention because it successfully joins the benefits of association rule mining with classification. These benefits include the inherent ability of association rule mining to extract high-order interactions among the predictors-an ability that many modern classifiers lack-and also the natural interpretability of the individual PARs. Associative classification is not without its caveats. Association rule mining often discovers a combinatorially large number of association rules, eroding the interpretability of the rule set. Extensive effort has been directed towards developing interestingness measures, which filter (predictive) association rules after they have been generated. These interestingness measures, albeit very successful at selecting interesting rules, lack two features that are highly valuable in the context of classification. First, only few of the interestingness measures are rooted in a statistical model. Given the distinction between a training and a test data set in the classification setting, the ability to make statistical inferences about the performance of the predictive classification rules on the test set is highly desirable. Second, the unfiltered set of predictive assocation rules (PARs) are often redundant, we can prove that certain PARs will not be used to construct a classification model given the presence of other PARs. In this paper, we propose a simple statistical model towards making inferences on the test set about the various performance metrics of predictive association rules. We also derive three filtering criteria based on hypothesis testing, which are very selective (reduce the number of PARs to be considered by the classifier by several orders of magnitude), yet do not effect the performance of the classification adversely. In the case, where the classification model is constructed as a logistic model on top of the PARs, we can mathematically prove, that the filtering criteria do not significantly effect the classifier's performance. We also demonstrate empirically on three publicly available data sets that the vast reduction in the number of PARs indeed did not come at the cost of reducing the predictive performance.

UR - http://www.scopus.com/inward/record.url?scp=80052686087&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052686087&partnerID=8YFLogxK

U2 - 10.1145/2020408.2020550

DO - 10.1145/2020408.2020550

M3 - Conference contribution

SN - 9781450308137

SP - 823

EP - 831

BT - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

ER -