×

3D grasp saliency analysis via deep shape correspondence. (English) Zbl 1505.65110

Summary: Grasp saliency map is an important analysis tool to explore human grasping skills and has many potential applications in visual and robotic fields. Currently, few works concentrate on the 3D grasp saliency detection for novel challenging instances, since the calculation of the grasp saliency map depends on the insufficient human grasping data and geometry of the 3D shapes. To address the above problem, we propose a novel grasp saliency map transferring technique via learning-based shape correspondence to augment saliency data. The robustness and accuracy of the proposed training model can be effectively improved with the augmented saliency data. Different from the crow-sourced data based approach, our method predicts grasp saliency map through classification deep neural network with global shape information. In addition, our approach can deal with the unknown object captured on-the-fly. Extensive experimental results show that our approach can achieve satisfactory grasp saliency maps for objects with complex shapes.

MSC:

65D17 Computer-aided design (modeling of curves and surfaces)
65D19 Computational issues in computer and robotic vision
68U07 Computer science aspects of computer-aided design
68T07 Artificial neural networks and deep learning
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Akizuki, Shuichi; Aoki, Y., Tactile logging for understanding plausible tool use based on human demonstration, (BMVC (2018)), 334
[2] Boscaini, D.; Masci, J.; Bronstein, M. M.; Cremers, D., Anisotropic diffusion descriptors, Comput. Graph. Forum, 35, 431-441 (2016)
[3] Boscaini, D.; Masci, J.; Melzi, S.; Bronstein, M. M.; Castellani, U.; Vandergheynst, P., Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks, Comput. Graph. Forum, 34, 13-23 (2015)
[4] Boscaini, D.; Masci, J.; Rodolà, E.; Bronstein, M. M., Learning Shape Correspondence with Anisotropic Convolutional Neural Networks (2016)
[5] Brahmbhatt, S.; Ham, C.; Kemp, C. C.; Hays, J., Contactdb: analyzing and predicting grasp contact via thermal imaging (2019), arXiv preprint
[6] Brock, A.; Lim, T.; Ritchie, J. M.; Weston, N., Generative and discriminative voxel modeling with convolutional neural networks, Comput. Sci. (2016)
[7] Charles, R. Q.; Hao, S.; Mo, K.; Guibas, L. J., Pointnet: deep learning on point sets for 3d classification and segmentation, (IEEE Conference on Computer Vision & Pattern Recognition (2017))
[8] Cheng, M. M.; Zhang, G. X.; Mitra, N. J.; Huang, X.; Hu, S. M., Global contrast based salient region detection, (Computer Vision and Pattern Recognition (2011))
[9] Ezuz, D.; Solomon, J.; Kim, V. G.; Ben-Chen, M., Gwcnn: a metric alignment layer for deep shape analysis, Comput. Graph. Forum, 36 (2017)
[10] Goferman, S.; Zelnikmanor, L.; Tal, A., Context-aware saliency detection, IEEE Trans. Pattern Anal. Mach. Intell., 34, 1915-1926 (2012)
[11] Guerrero, P.; Kleiman, Y.; Ovsjanikov, M.; Mitra, N. J., Pcpnet: learning local shape properties from raw point clouds, Comput. Graph. Forum, 37 (2017)
[12] Guo, Y.; Wang, F.; Xin, J., Point-wise saliency detection on 3d point clouds via covariance descriptors, Vis. Comput., 34, 1-14 (2017)
[13] Guy, G.; Medioni, G., Inference of surfaces, 3-d curves, and junctions from sparse 3-d data, (International Symposium on Computer Vision (1997))
[14] Hamer, H.; Gall, J.; Weise, T.; Gool, L. J.V., An object-dependent hand pose prior from sparse training data, (Computer Vision & Pattern Recognition (2010))
[15] Howard, P. I.; Rogers, B. J., Seeing in Depth, vol. 2 (2002), University of Toronto Press
[16] Jaderberg, M.; Simonyan, K.; Zisserman, A.; Kavukcuoglu, K., Spatial Transformer Networks (2015)
[17] Kan, G.; Zou, D.; Chen, X., 3d mesh labeling via deep convolutional neural networks, ACM Trans. Graph., 35, 1-12 (2015)
[18] Kwatra, V.; Schödl, A.; Essa, I.; Turk, G.; Bobick, A., Graphcut textures: image and video synthesis using graph cuts, ACM Trans. Graph., 22, 277-286 (2003)
[19] Kwok, T. H.; Wan, W.; Pan, J.; Wang, C. C.L.; Chen, Y., Rope caging and grasping, (IEEE International Conference on Robotics & Automation (2016))
[20] Lau, M.; Dev, K.; Shi, W.; Dorsey, J.; Rushmeier, H., Tactile mesh saliency, ACM Trans. Graph., 35, 52 (2016)
[21] Leal, E.; Sanchez, G.; Bedoya, J., Point cloud saliency detection via local sparse coding, DYNA, 86, 238-247 (2019)
[22] Lee, C. H.; Varshney, A.; Jacobs, D. W., Mesh saliency, ACM Trans. Graph., 24, 659-666 (2005)
[23] Li, Y.; Bu, R.; Sun, M.; Wu, W.; Di, X.; Chen, B., Pointcnn: Convolution on \(\mathcal{X} \)-Transformed Points (2018)
[24] Liang, Z.; Zhu, G.; Shen, P.; Song, J., Learning spatiotemporal features using 3dcnn and convolutional lstm for gesture recognition, (IEEE International Conference on Computer Vision Workshop (2017))
[25] Lippiello, V., Grasp the possibilities: anthropomorphic grasp synthesis based on the object dynamic properties, IEEE Robot. Autom. Mag., 22, 69-79 (2015)
[26] Masci, J.; Boscaini, D.; Bronstein, M. M.; Vandergheynst, P., Geodesic convolutional neural networks on Riemannian manifolds, (IEEE International Conference on Computer Vision Workshop (2015))
[27] Monti, F.; Boscaini, D.; Masci, J.; Rodola, E.; Bronstein, M. M., Geometric deep learning on graphs and manifolds using mixture model cnns, (IEEE Conference on Computer Vision & Pattern Recognition (2017))
[28] Pauly, M.; Keiser, R.; Gross, M., Multi-scale feature extraction on point-sampled surfaces, Comput. Graph. Forum, 22, 281-289 (2003)
[29] Plaisier, M. A.; Tiest, W. M.B.; Kappers, A. M.L., Salient features in 3-d haptic shape perception, Atten. Percept. Psychophys., 71, 421-430 (2009)
[30] Qi, C. R.; Yi, L.; Su, H.; Guibas, L. J., Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (2017)
[31] Ran, M.; Tal, A.; Zelnik-Manor, L., What makes a patch distinct?, (IEEE Conference on Computer Vision & Pattern Recognition (2013))
[32] Ran, S.; Liu, Y.; Martin, R. R.; Echavarria, K. R., Local-to-global mesh saliency, Vis. Comput., 34, 1-14 (2018)
[33] Ran, S.; Liu, Y.; Martin, R. R.; Rosin, P. L., Mesh saliency via spectral processing, ACM Trans. Graph., 33, 1-17 (2014) · Zbl 1288.68239
[34] Rodolà, E.; Bulò, S. R.; Windheuser, T.; Vestner, M.; Cremers, D., Dense non-rigid shape correspondence using random forests, (IEEE Conference on Computer Vision & Pattern Recognition (2014))
[35] Rodriguez, A.; Mason, M. T.; Ferry, S., From caging to grasping, (Robotics: Science & Systems VII (2011)), 886-900
[36] Shu, Z.; Xin, S.; Xin, X.; Liu, L.; Ladislav, K., Detecting 3d points of interest using multiple features and stacked auto-encoder, IEEE Trans. Vis. Comput. Graph., 25, 2583-2596 (2018)
[37] Sinha, A.; Jing, B.; Ramani, K., Deep Learning 3D Shape Surfaces Using Geometry Images (2016)
[38] Tao, P.; Cao, J.; Li, S.; Liu, X.; Liu, L., Mesh saliency via ranking unsalient patches in a descriptor space, Comput. Graph., 46, 264-274 (2015)
[39] Tasse, F. P.; Kosinka, J.; Dodgson, N. A., Cluster-based point set saliency, (IEEE International Conference on Computer Vision (2015))
[40] Wan, W.; Rui, F.; Shimosaka, M.; Sato, T.; Kuniyoshi, Y., A new “grasping by caging” solution by using eigen-shapes and space mapping, (2013 IEEE International Conference on Robotics and Automation. 2013 IEEE International Conference on Robotics and Automation, ICRA (2013))
[41] Wang, S.; Li, N.; Shuai, L.; Luo, Z.; Su, Z.; Hong, Q., Multi-scale mesh saliency based on low-rank and sparse analysis in shape feature space, Comput. Aided Geom. Des., 35-36, 206-214 (2015) · Zbl 1417.68257
[42] Wang, Y.; Liu, R.; Song, X.; Su, Z., A nonlocal \(l_0\) model with regression predictor for saliency detection and extension, Vis. Comput., 33, 1-16 (2017)
[43] Wei, Q.; Jing, H.; Yi, Z.; Bai, L., Saliency detection via Boolean and foreground in a dynamic Bayesian framework, Vis. Comput. Int. J. Comput. Graph., 33, 209-220 (2015)
[44] Wu, Z.; Song, S.; Khosla, A.; Yu, F.; Zhang, L.; Tang, X.; Xiao, J.; Wu, Z.; Song, S.; Khosla, A., 3d shapenets: a deep representation for volumetric shapes, (IEEE Conference on Computer Vision & Pattern Recognition (2015))
[45] Xiang, F.; Wanggen, W.; Da, X. R.Y.; Stuart, P.; Song, Z.; Zexin, L., A new mesh visual quality metric using saliency weighting-based pooling strategy, Graph. Models (2018)
[46] Zarubin, D.; Pokorny, F. T.; Toussaint, M.; Kragic, D., Caging Complex Objects with Geodesic Balls (2013)
[47] Zhang, X.; Chan, K. C.; Wang, C. C.L.; Wong, K. C.; Kumta, S. M., Computing stable contact interface for customized surgical jigs, (2015 IEEE International Conference Robotics and Automation. 2015 IEEE International Conference Robotics and Automation, ICRA (2015))
[48] Zhou, J.; Wang, M. J.; Mao, W. D.; Gong, M. L.; Liu, X. P., Siamesepointnet: a Siamese point network architecture for learning 3d shape descriptor, Comput. Graph. Forum (2019)
[49] Zhu, W.; Shuang, L.; Wei, Y.; Jian, S., Saliency optimization from robust background detection, (IEEE Conference on Computer Vision & Pattern Recognition (2014))
[50] Zhu, Z.; Wang, X.; Song, B.; Cong, Y.; Xiang, B., Deep learning representation using autoencoder for 3d shape retrieval, (International Conference on Security (2014))
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.