Detecting group differences: Mining contrast sets. (English) Zbl 0982.68048

Bay, Stephen D.; Pazzani, Michael J.

Data Mining and Knowledge Discovery 5, No. 3, 213-246 (2001).

Summary: A fundamental task in data analysis is understanding the differences between several contrasting groups. These groups can represent different classes of objects, such as male or female students, or the same group over time, e.g. freshman students in 1993 through 1998. We present the problem of mining contrast sets: conjunctions of attributes and values that differ meaningfully in their distribution across groups. We provide a search algorithm for mining contrast sets with pruning rules that drastically reduce the computational complexity. Once the contrast sets are found, we post-process the results to present a subset that are surprising to the user given what we have already shown. We explicitly control the probability of Type I error (false positives) and guarantee a maximum error rate for the entire analysis by using Bonferroni corrections.

MSC:

68P05	Data structures
68M10	Network design and communication in computer systems
68U99	Computing methodologies and applications

Keywords:

data analysis