×

Interactive slice visualization for exploring machine learning models. (English) Zbl 07546455

Summary: Machine learning models fit complex algorithms to arbitrarily large datasets. These algorithms are well known to be high on performance and low on interpretability. We use interactive visualization of slices of predictor space to address the interpretability deficit; in effect opening up the black-box of machine learning algorithms, for the purpose of interrogating, explaining, validating and comparing model fits. Slices are specified directly through interaction, or using various touring algorithms designed to visit high-occupancy sections, or regions where the model fits have interesting properties. The methods presented here are implemented in the R package condvis2. Supplementary files for this article are available online.

MSC:

62-XX Statistics
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Asimov, D., “The Grand Tour: a Tool for Viewing Multidimensional data, Siam Journal on Scientific and Statistical Computing, 6, 128-143 (1985) · Zbl 0552.62052 · doi:10.1137/0906011
[2] Baniecki, H.; Biecek, P., The Grammar of Interactive Explanatory Model Analysis, CoRR, abs/2005.00497 (2020)
[3] Becker, R. A.; Cleveland, W. S.; Shyu, M.-J., “The Visual Design and Control of Trellis Display,”, Journal of Computational and Graphical Statistics, 5, 123-155 (1996)
[4] Bellman, R., Research Studies, Adaptive Control Processes: A Guided Tour, Rand Corporation (1961), Princeton, NJ: Princeton University Press, Princeton, NJ · Zbl 0103.12901
[5] Bischl, B.; Lang, M.; Kotthoff, L.; Schiffner, J.; Richter, J.; Studerus, E.; Casalicchio, G.; Jones, Z. M., “mlr: Machine Learning in R,”, Journal of Machine Learning Research, 17, 1-5 (2016) · Zbl 1392.68007
[6] Breiman, L., “Random Forests, Machine Learning, 45, 5-32 (2001) · Zbl 1007.68152 · doi:10.1023/A:1010933404324
[7] Britton, M., VINE: Visualizing Statistical Interactions in Black Box Models, arXiv 1904.00561 (2019)
[8] Chambers, J.; Cleveland, W.; Kleiner, B.; Tukey, P., Graphical Methods for Data Analysis (1983), Belmont, CA: Wadsworth, Belmont, CA · Zbl 0532.65094
[9] Chang, W.; Cheng, J.; Allaire, J.; Xie, Y.; McPherson, J., Shiny: Web Application Framework for R (2020), R package version 1.5.0
[10] Cook, D.; Buja, A.; Cabrera, J.; Hurley, C., “Grand Tour and Projection Pursuit,”, Journal of Computational and Graphical Statistics, 4, 155-172 (1995)
[11] Cook, D.; Swayne, D. F., Interactive and Dynamic Graphics for Data Analysis With R and GGobi (2007), New York: Springer Publishing Company, Incorporated, New York · Zbl 1154.62006
[12] De Cock, D., “Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project, Journal of Statistics Education, 19 (2011) · doi:10.1080/10691898.2011.11889627
[13] Earle, D.; Hurley, C. B., “Advances in Dendrogram Seriation for Application to Visualization, Journal of Computational and Graphical Statistics, 24, 1-25 (2015) · doi:10.1080/10618600.2013.874295
[14] Fanaee-T, H.; Gama, J., “Event Labeling Combining Ensemble Detectors and Background Knowledge,”, Progress in Artificial Intelligence, 1-15 (2013)
[15] Friedman, J. H., “Greedy Function Approximation: A Gradient Boosting Machine, The Annals of Statistics, 29, 1189-1232 (2001) · Zbl 1043.62034 · doi:10.1214/aos/1013203451
[16] Friedman, J. H.; Popescu, B. E., “Predictive Learning Via Rule Ensembles,”, Annals of Applied Statistics, 2, 916-954 (2008) · Zbl 1149.62051
[17] Furnas, G. W.; Buja, A., “Prosection Views: Dimensional Inference Through Sections and Projections,”, Journal of Computational and Graphical Statistics, 3, 323-353 (1994)
[18] Goldstein, A.; Kapelner, A.; Bleich, J.; Pitkin, E., “Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation, Journal of Computational and Graphical Statistics, 24, 44-65 (2015) · doi:10.1080/10618600.2014.907095
[19] Gower, J. C., “A General Coefficient of Similarity and Some of Its Properties, Biometrics, 27, 857-871 (1971) · doi:10.2307/2528823
[20] Hurley, C.; O’Connell, M.; Domijan, K., Condvis2: Conditional Visualization for Supervised and Unsupervised Models in Shiny (2020), R package version 0.1.1
[21] Hurley, C. B., “Model Exploration Using Conditional Visualization, WIREs Computational Statistics, 13, e1503 (2021) · doi:10.1002/wics.1503
[22] Inglis, A.; Parnell, A.; Hurley, C., Vivid: Variable Importance and Variable Interaction Displays (2020), R package version 0.1.0
[23] Kim, S.; Oh, S., “Development of Machine Learning Models for Diagnosis of Glaucoma, PLoS One, 5 (2017) · doi:10.1371/journal.pone.0177726
[24] Kuhn, M., Caret: Classification and Regression Training,, R package version 6, 0-84 (2019)
[25] Kuhn, M.; Vaughan, D., parsnip: A Common API to Modeling and Analysis Functions (2021), R package version 0.1.25
[26] Laa, U.; Cook, D.; Valencia, G., “A Slice Tour for Finding Hollowness in High-Dimensional Data, Journal of Computational and Graphical Statistics, 29, 681-687 (2020) · Zbl 07499307 · doi:10.1080/10618600.2020.1777140
[27] Lang, M.; Binder, M.; Richter, J.; Schratz, P.; Pfisterer, F.; Coors, S.; Au, Q.; Casalicchio, G.; Kotthoff, L.; Bischl, B., “mlr3: A Modern Object-Oriented Machine Learning Framework in R, Journal of Open Source Software, 4, 44, 1903 (2019) · doi:10.21105/joss.01903
[28] Lundberg, S. M.; Lee, S.-I.; Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; Garnett, R., Advances in Neural Information Processing Systems, 30), A Unified Approach to Interpreting Model Predictions, 4765-4774 (2017), New York: Curran Associates, Inc, New York
[29] Maechler, M.; Rousseeuw, P.; Struyf, A.; Hubert, M.; Hornik, K., cluster: Cluster Analysis Basics and Extensions (2019), R package version 2.1.0
[30] O’Connell, M., Conditional Visualisation for Statistical Models (2017), National University of Ireland: National University of Ireland, Maynooth
[31] O’Connell, M.; Hurley, C.; Domijan, K., Condvis: Conditional Visualization for Statistical Models (2016), R package version 0.1.1
[32] O’Connell, M.; Hurley, C.; Domijan, K., “Conditional Visualization for Statistical Models: An Introduction to the condvis Package in R, Journal of Statistical Software, 81, 1-20 (2017)
[33] Ribeiro, M. T.; Singh, S.; Guestrin, C., Why Should I Trust You?”: Explaining the Predictions of Any Classifier, 1135-1144 (2016)
[34] Salzberg, S. L., “C4.5: Programs for Machine Learning by J. Ross Quinlan. Morgan Kaufmann Publishers, Inc., 1993, Machine Learning, 16, 235-240 (1994) · doi:10.1007/BF00993309
[35] Staniak, M.; Biecek, P., “Explanations of Model Predictions With Live and Breakdown Packages, The R Journal, 10, 395-409 (2018) · doi:10.32614/RJ-2018-072
[36] Stuetzle, W., “Plot Windows, Journal of the American Statistical Association, 82, 466-475 (1987) · doi:10.1080/01621459.1987.10478449
[37] Tufte, E. R., The Visual Display of Quantitative Information (1986), Cheshire, CT: Graphics Press, Cheshire, CT
[38] Unwin, A., GDAdata: Datasets for the Book Graphical Data Analysis With R (2015), R package version 0.93
[39] Unwin, A.; Valero-Mora, P., “Ensemble Graphics, Journal of Computational and Graphical Statistics, 27, 157-165 (2018) · Zbl 07498975 · doi:10.1080/10618600.2017.1383264
[40] Urbanek, S.; Härdle, W.; Rönz, B., Compstat, Different Ways to See a Tree - KLIMT,”, 303-308 (2002), Heidelberg: Physica-Verlag HD, Heidelberg · Zbl 1439.62042
[41] Waddell, A.; Oldford, R. W., loon: Interactive Statistical Data Visualization (2020), r package version 1.3.1
[42] Wickham, H., ggplot2: Elegant Graphics for Data Analysis (2016), New York: Springer-Verlag, New York · Zbl 1397.62006
[43] Wilkinson, L., The Grammar of Graphics (Statistics and Computing (2005), Secaucus, NJ: Springer-Verlag New York, Inc, Secaucus, NJ · Zbl 1080.68107
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.