×

A new score for adaptive tests in Bayesian and credal networks. (English) Zbl 07542357

Vejnarová, Jiřina (ed.) et al., Symbolic and quantitative approaches to reasoning with uncertainty. 16th European conference, ECSQARU 2021, Prague, Czech Republic, September 21–24, 2021. Proceedings. Cham: Springer. Lect. Notes Comput. Sci. 12897, 399-412 (2021).
Summary: A test is adaptive when the sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. Graphical models, such as Bayesian networks, are used for adaptive tests as they allow to model the uncertainty about the questions and the skills in an explainable fashion, especially when coping with multiple skills. A better elicitation of the uncertainty in the question/skills relations can be achieved by interval probabilities. This turns the model into a credal network, thus increasing the inferential complexity of the queries required to select questions. This is especially the case for the information-theoretic quantities used as scores to drive the adaptive mechanism. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain. This makes considerably simpler the evaluation in the credal case, without significantly affecting the quality of the adaptive process. Numerical tests on synthetic and real-world data are used to support this claim.
For the entire collection see [Zbl 1487.68022].

MSC:

68T37 Reasoning under uncertainty in the context of artificial intelligence

Software:

CREMA
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Abellan, J.; Moral, S., Maximum of entropy for credal sets, Int. J. Uncertainty Fuzziness Knowl.-Based Syst., 11, 5, 587-597 (2003) · Zbl 1072.68099 · doi:10.1142/S021848850300234X
[2] Almond, RG; Mislevy, RJ, Graphical models and computerized adaptive testing, Appl. Psychol. Meas., 23, 3, 223-237 (1999) · doi:10.1177/01466219922031347
[3] Antonucci, A.; Piatti, A.; Godo, L.; Pugliese, A., Modeling unreliable observations in Bayesian networks by credal networks, Scalable Uncertainty Management, 28-39 (2009), Heidelberg: Springer, Heidelberg · doi:10.1007/978-3-642-04388-8_4
[4] Antonucci, A.; de Campos, CP; Huber, D.; Zaffalon, M.; van der Gaag, LC, Approximating credal network inferences by linear programming, Symbolic and Quantitative Approaches to Reasoning with Uncertainty, 13-24 (2013), Heidelberg: Springer, Heidelberg · Zbl 1390.68629 · doi:10.1007/978-3-642-39091-3_2
[5] Antonucci, A.; de Campos, CP; Huber, D.; Zaffalon, M., Approximate credal network updating by linear programming with applications to decision making, Int. J. Approximate Reasoning, 58, 25-38 (2015) · Zbl 1328.68225 · doi:10.1016/j.ijar.2014.10.003
[6] Bachrach, Y., Graepel, T., Minka, T., Guiver, J.: How to grade a test without knowing the answers-a Bayesian graphical model for adaptive crowdsourcing and aptitude testing. arXiv preprint arXiv:1206.6386 (2012)
[7] Badaracco, M.; Martínez, L., A fuzzy linguistic algorithm for adaptive test in intelligent tutoring system based on competences, Expert Syst. Appl., 40, 8, 3073-3086 (2013) · doi:10.1016/j.eswa.2012.12.023
[8] Badran, M.E.K., Abdo, J.B., Al Jurdi, W., Demerjian, J.: Adaptive serendipity for recommender systems: Let it find you. In: ICAART (2), pp. 739-745 (2019)
[9] Bolt, J.H., De Bock, J., Renooij, S.: Exploiting Bayesian network sensitivity functions for inference in credal networks. In: Proceedings of the Twenty-Second European Conference on Artificial Intelligence (ECAI), vol. 285, pp. 646-654. IOS Press (2016) · Zbl 1403.68289
[10] Bonesana, C., Mangili, F., Antonucci, A.: ADAPQUEST: a software for web-based adaptive questionnaires based on Bayesian networks. In: IJCAI 2021 Workshop Artificial Intelligence for Education (2021) · Zbl 1491.68241
[11] Chen, S.J., Choi, A., Darwiche, A.: Computer adaptive testing using the same-decision probability. In: BMA@ UAI, pp. 34-43 (2015)
[12] Conati, C.; Gertner, AS; VanLehn, K.; Druzdzel, MJ; Jameson, A.; Paris, C.; Tasso, C., On-line student modeling for coached problem solving using Bayesian networks, User Modeling, 231-242 (1997), Vienna: Springer, Vienna · doi:10.1007/978-3-7091-2670-7_24
[13] Cozman, FG, Credal networks, Artif. Intell., 120, 2, 199-233 (2000) · Zbl 0945.68163 · doi:10.1016/S0004-3702(00)00029-1
[14] Embretson, S.E., Reise, S.P.: Item Response Theory. Psychology Press, Hove (2013)
[15] Hájek, A.; Smithson, M., Rationality and indeterminate probabilities, Synthese, 187, 1, 33-48 (2012) · Zbl 1275.91023 · doi:10.1007/s11229-011-0033-3
[16] Huber, D., Cabañas, R., Antonucci, A., Zaffalon, M.: CREMA: a Java library for credal network inference. In: Jaeger, M., Nielsen, T. (eds.) Proceedings of the 10th International Conference on Probabilistic Graphical Models (PGM 2020). Proceedings of Machine Learning Research, PMLR, Aalborg, Denmark (2020)
[17] Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009) · Zbl 1183.68483
[18] Laitusis, CC; Morgan, DL; Bridgeman, B.; Zanna, J.; Stone, E., Examination of fatigue effects from extended-time accommodations on the SAT reasoning test, ETS Research Report Series, 2007, 2, i-13 (2007) · doi:10.1002/j.2333-8504.2007.tb02073.x
[19] Mangili, F.; Bonesana, C.; Antonucci, A.; Antonucci, A.; Cholvy, L.; Papini, O., Reliable knowledge-based adaptive tests by credal networks, Symbolic and Quantitative Approaches to Reasoning with Uncertainty, 282-291 (2017), Cham: Springer, Cham · Zbl 1491.68241 · doi:10.1007/978-3-319-61581-3_26
[20] Marchetti, S., Antonucci, A.: Reliable uncertain evidence modeling in Bayesian networks by credal networks. In: Brawner, K.W., Rus, V. (eds.) Proceedings of the Thirty-First International Florida Artificial Intelligence Research Society Conference (FLAIRS-31), pp. 513-518. AAAI Press, Melbourne, Florida, USA (2018)
[21] Mauá, DD; De Campos, CP; Benavoli, A.; Antonucci, A., Probabilistic inference in credal networks: new complexity results, J. Artif. Intell. Res., 50, 603-637 (2014) · Zbl 1366.68329 · doi:10.1613/jair.4355
[22] Piatti, A., Antonucci, A., Zaffalon, M.: Building knowledge-based expert systems by credal networks: a tutorial. In: Baswell, A. (ed.) Advances in Mathematics Research, vol. 11, chap. 2. Nova Science Publishers, New York (2010)
[23] Plajner, M., Vomlel, J.: Monotonicity in practice of adaptive testing. arXiv preprint arXiv:2009.06981 (2020) · Zbl 1491.68169
[24] Sawatzky, R., Ratner, P.A., Kopec, J.A., Wu, A.D., Zumbo, B.D.: The accuracy of computerized adaptive testing in heterogeneous populations: a mixture item-response theory analysis. PLoS ONE 11(3), e0150563 (2016)
[25] Vomlel, J., Bayesian networks in educational testing, Int. J. Uncertain. Fuzziness Knowl.-Based Syst., 12, supp01, 83-100 (2004) · Zbl 1101.68847 · doi:10.1142/S021848850400259X
[26] Vomlel, J., Building adaptive tests using Bayesian networks, Kybernetika, 40, 3, 333-348 (2004) · Zbl 1249.68174
[27] Wilcox, AR, Indices of qualitative variation and political measurement, Western Political Q., 26, 2, 325-343 (1973) · doi:10.1177/106591297302600209
[28] Xiang, G., Kosheleva, O., Klir, G.J.: Estimating information amount under interval uncertainty: algorithmic solvability and computational complexity. Technical report 158, Departmental Technical Reports (CS) (2006)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.