CLEVR swMATH ID: 42503 Software Authors: Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, Ross Girshick Description: CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning. When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings. Existing benchmarks for visual question answering can help, but have strong biases that models can exploit to correctly answer questions without reasoning. They also conflate multiple sources of error, making it hard to pinpoint model weaknesses. We present a diagnostic dataset that tests a range of visual reasoning abilities. It contains minimal biases and has detailed annotations describing the kind of reasoning each question requires. We use this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations. Homepage: https://arxiv.org/abs/1612.06890 Source Code: https://github.com/facebookresearch/clevr-dataset-gen Related Software: CLEVR dataset; VQA; Grad-CAM; t-SNE; DeepProbLog; NeurASP; AlexNet; GitHub; ImageNet; PyTorch; GPT-3; BERT; Tensor2Tensor; AQuA; Visual7W; YOLO; COVAREP; RUBi; OpenFace; VL-InterpreT Cited in: 7 Publications all top 5 Cited by 20 Authors 1 Chen, Chaofan 1 Chen, Zhi 1 Dash, Tirtharaj 1 Derczynski, Leon 1 Doran, Derek 1 Higuera, Nelson 1 Holzinger, Andreas 1 Huang, Haiyang 1 Jurewicz, Mateusz 1 Müller, Heimo 1 Oetsch, Johannes 1 Pritz, Michael 1 Ras, Gabrielle 1 Rudin, Cynthia 1 Semenova, Lesia 1 Srinivasan, Ashwin 1 van Gerven, Marcel A. J. 1 Xie, Ning 1 Yang, Zhun 1 Zhong, Chudi Cited in 5 Serials 2 The Journal of Artificial Intelligence Research (JAIR) 1 Artificial Intelligence 1 Machine Learning 1 Theory and Practice of Logic Programming 1 Statistics Surveys Cited in 2 Fields 7 Computer science (68-XX) 1 Statistics (62-XX) Citations by Year