Discovering Predictive Association Rules

@inproceedings{MegiddoSrikant98, abstract = {Association rule algorithms can produce a very large number of output patterns. This has raised questions of whether the set of discovered rules "overfit" the data because all the patterns that satisfy some constraints are generated (the Bonferroni effect). In other words, the question is whether some of the rules are "false discoveries" that are not statistically significant. We present a novel approach for estimating the number of "false discoveries" at any cutoff level. Empirical evaluation...}, author = {Megiddo, Nimrod and Srikant, Ramakrishnan }, booktitle = {Knowledge Discovery and Data Mining}, citeulike-article-id = {1689846}, comment = {ooo. specifically addresses multiple comparisons problem. Here's the deal - postulates the count of a given item (or itemset) in the dataset is a Binomial r.v. with n=\# of baskets. p, is of course unknown. n is large, so can use poisson (or paper claims normal) approximation... oh so it puts both sides of the rule in the set and tests the hypothesis that they are less than the support level. zscore calculated as: obs proportion-chosen support/sqrt(supp(1-supp))}, keywords = {almaden, association, datamining, rules}, pages = {274--278}, posted-at = {2007-09-24 17:00:03}, priority = {0}, title = {Discovering Predictive Association Rules}, url = {http://citeseer.ist.psu.edu/megiddo98discovering.html}, year = {1998} }

TY - CONF ID - MegiddoSrikant98 L3 - citeulike-article-id:1689846 N2 - Association rule algorithms can produce a very large number of output patterns. This has raised questions of whether the set of discovered rules "overfit" the data because all the patterns that satisfy some constraints are generated (the Bonferroni effect). In other words, the question is whether some of the rules are "false discoveries" that are not statistically significant. We present a novel approach for estimating the number of "false discoveries" at any cutoff level. Empirical evaluation... EP - 278 TI - Discovering Predictive Association Rules T2 - Knowledge Discovery and Data Mining SP - 274 KW - almaden KW - association KW - datamining KW - rules N1 - ooo. specifically addresses multiple comparisons problem. Here's the deal - postulates the count of a given item (or itemset) in the dataset is a Binomial r.v. with n=# of baskets. p, is of course unknown. n is large, so can use poisson (or paper claims normal) approximation... oh so it puts both sides of the rule in the set and tests the hypothesis that they are less than the support level. zscore calculated as: obs proportion-chosen support/sqrt(supp(1-supp)) AU - Megiddo, Nimrod AU - Srikant, Ramakrishnan PY - 1998/// UR - http://citeseer.ist.psu.edu/megiddo98discovering.html ER -