zbMATH — the first resource for mathematics

Temporal delay estimation of sparse direct visual inertial odometry for mobile robots. (English) Zbl 1437.93089
Summary: Most visual inertial odometry (VIO) system can get an ideal result on datasets, which typically use an external device to synchronize measurements between the camera and the inertial measurement unit (IMU). But for an inexpensive homemade cameras-IMU system, the accuracy of these algorithms usually degrades or even fails because of the temporal misalignment between the camera and the IMU. In this article, we focus on the time synchronization problem in direct VIO system, and we propose a method to calibrate the temporal delay between the camera and the IMU. To achieve this, the temporal delay parameter is added into the state variable of the extended Kalman filter (EKF), and the brightness error of the landmark is used as the error function to update the EKF filter. Finally, we construct a sparse direct VIO system with online temporal delay estimation, and make a comparison with the VINS-mono and the ROVIO. The experiments show that the brightness error of the landmark can be used to accurately calibrate the temporal delay between the image and the IMU measurements in direct VIO system, and the method considering the temporal delay will improve the performance of the direct VIO system.
93C85 Automated systems (robots, etc.) in control theory
93C43 Delay control/observation systems
93E11 Filtering in stochastic control theory
Full Text: DOI
[1] Basin, M. V.; Ramírez, P. C.R.; Guerra-Avellaneda, F., Continuous fixed-time controller design for mechatronic systems with incomplete measurements, IEEE/ASME Trans. Mechatron., 23, 1, 57-67 (2018)
[2] Karaboga, N., A new design method based on artificial bee colony algorithm for digital IIR filters, J. Frankl. Inst., 346, 4, 328-348 (2009) · Zbl 1166.93351
[3] Basin, M.; Maldonado, J. J., Optimal mean-square state and parameter estimation for stochastic linear systems with poisson noises, Inf. Sci., 197, 177-186 (2012) · Zbl 1256.93104
[4] Bian, Y.; Peng, J.; Han, C., Finite-time control for nonholonomic mobile robot by brain emotional learning-based intelligent controller, Int. J. Innov. Comput. Inf. Control, 14, 683-695 (2018)
[5] Wang, Y.; Yang, U.; Wang, S., Path tracking control of an indoor transportation robot utilizing future information of the desired trajectory, Int. J. Innov. Comput. Inf. Control, 14, 2, 561-572 (2018)
[6] Smith, R. C.; Cheeseman, P., On the representation and estimation of spatial uncertainty, Int. J. Robot. Res., 5, 4, 56-68 (1986)
[7] Xiong, X.; Adan, A.; Akinci, B.; Huber, D., Automatic creation of semantically rich 3d building models from laser scanner data, Autom. constr., 31, 325-337 (2013)
[8] Mehta, S. S.; Ton, C.; Kan, Z.; Curtis, J. W., Vision-based navigation and guidance of a sensorless missile, J. Frankl. Inst., 352, 12, 5569-5598 (2015) · Zbl 1395.93447
[9] Forster, C.; Zhang, Z.; Gassner, M.; Werlberger, M.; Scaramuzza, D., SVO: semidirect visual odometry for monocular and multicamera systems, IEEE Trans. Robot., 33, 2, 249-265 (2016)
[10] Davison, A. J.; Reid, I. D.; Molton, N. D.; Stasse, O., MonoSLAM: real-time single camera SLAM, IEEE Transactions on Pattern Analysis & Machine Intelligence, volume 6, 1052-1067 (2007)
[11] Mur-Artal, R.; Tardós, J. D., Orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras, IEEE Trans. Robot., 33, 5, 1255-1262 (2017)
[12] Sun, K.; Mohta, K.; Pfrommer, B.; Watterson, M.; Liu, S.; Mulgaonkar, Y.; Taylor, C. J.; Kumar, V., Robust stereo visual inertial odometry for fast autonomous flight, IEEE Robot. Autom. Lett., 3, 2, 965-972 (2018)
[13] Henry, P.; Krainin, M.; Herbst, E.; Ren, X.; Fox, D., RGB-d mapping: using kinect-style depth cameras for dense 3d modeling of indoor environments, Int. J. Robot. Res., 31, 5, 647-663 (2012)
[14] Mur-Artal, R.; Tardós, J. D., Visual-inertial monocular SLAM with map reuse, IEEE Robot. Autom. Lett., 2, 2, 796-803 (2017)
[15] Yang, Z.; Shen, S., Monocular visual-inertial state estimation with online initialization and camera-IMU extrinsic calibration, IEEE Trans. Autom. Sci. Eng., 14, 1, 39-51 (2016)
[16] Leutenegger, S.; Lynen, S.; Bosse, M.; Siegwart, R.; Furgale, P., Keyframe-based visual-inertial odometry using nonlinear optimization, Int. J. Robot. Res., 34, 3, 314-334 (2015)
[17] Forster, C.; Carlone, L.; Dellaert, F.; Scaramuzza, D., On-manifold preintegration for real-time visual-inertial odometry, IEEE Trans. Robot., 33, 1, 1-21 (2016)
[18] Abdel-Hafez, M. F., On the development of an inertial navigation error-budget system, J. Frankl. Inst., 348, 1, 24-44 (2011) · Zbl 1213.93010
[19] Engel, J.; Koltun, V.; Cremers, D., Direct sparse odometry, IEEE Trans. Pattern Anal. Mach. Intell., 40, 3, 611-625 (2017)
[20] Kelly, J.; Sukhatme, G. S., Visual-inertial sensor fusion: localization, mapping and sensor-to-sensor self-calibration, Int. J. Robot. Res., 30, 1, 56-79 (2011)
[21] Seetharaman, G.; Bao, H.; Shivaram, G., Calibration of camera parameters using vanishing points, J. Frankl. Inst., 331, 5, 555-585 (1994) · Zbl 0939.68907
[22] Hesch, J. A.; Kottas, D. G.; Bowman, S. L.; Roumeliotis, S. I., Consistency analysis and improvement of vision-aided inertial navigation, IEEE Trans. Robot., 30, 1, 158-176 (2013)
[23] Havangi, R., Unscented h-infinity filtering based simultaneous localization and mapping with evolutionary resampling, J. Frankl. Inst., 352, 11, 4801-4825 (2015) · Zbl 1395.93538
[24] Bloesch, M.; Burri, M.; Omari, S.; Hutter, M.; Siegwart, R., Iterated extended kalman filter based visual-inertial odometry using direct photometric feedback, Int. J. Robot. Res., 36, 10, 1053-1072 (2017)
[25] Kelly, J.; Sukhatme, G. S., A general framework for temporal calibration of multiple proprioceptive and exteroceptive sensors, Experimental Robotics, 195-209 (2014), Springer
[26] Choi, M.; Choi, J.; Park, J.; Chung, W. K., State estimation with delayed measurements considering uncertainty of time delay, 2009 IEEE International Conference on Robotics and Automation, 3987-3992 (2009), IEEE
[27] Merwe, R. V.D.; Wan, E.; Julier, S., Sigma-point kalman filters for nonlinear estimation and sensor-fusion: applications to integrated navigation, AIAA Guidance, Navigation, and Control Conference and Exhibit, 5120 (2004)
[28] Tungadi, F.; Kleeman, L., Time synchronisation and calibration of odometry and range sensors for high-speed mobile robot mapping, Proceedings of the Australasian Conference on Robotics and Automation (2008)
[29] Li, M.; Mourikis, A. I., 3-d motion estimation and online temporal calibration for camera-IMU systems, 2013 IEEE International Conference on Robotics and Automation, 5709-5716 (2013), IEEE
[30] Qin, T.; Shen, S., Online temporal calibration for monocular visual-inertial systems, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3662-3669 (2018), IEEE
[31] Mourikis, A. I.; Roumeliotis, S. I., A multi-state constraint kalman filter for vision-aided inertial navigation, Proceedings 2007 IEEE International Conference on Robotics and Automation, 3565-3572 (2007), IEEE
[32] Qin, T.; Li, P.; Shen, S., Vins-mono: a robust and versatile monocular visual-inertial state estimator, IEEE Trans. Robot., 34, 4, 1004-1020 (2018)
[33] Tedaldi, D.; Pretto, A.; Menegatti, E., A robust and easy to implement method for IMU calibration without external equipments, 2014 IEEE International Conference on Robotics and Automation (ICRA), 3042-3049 (2014), IEEE
[34] Li, M.; Mourikis, A. I., High-precision, consistent EKF-based visual-inertial odometry, Int. J. Robot. Res., 32, 6, 690-711 (2013)
[35] Rehder, J.; Nikolic, J.; Schneider, T.; Hinzmann, T.; Siegwart, R., Extending kalibr: calibrating the extrinsics of multiple IMUs and of individual axes, 2016 IEEE International Conference on Robotics and Automation (ICRA), 4304-4311 (2016), IEEE
[36] Burri, M.; Nikolic, J.; Gohl, P.; Schneider, T.; Rehder, J.; Omari, S.; Achtelik, M. W.; Siegwart, R., The euroc micro aerial vehicle datasets, Int. J. Robot. Res., 35, 10, 1157-1163 (2016)
[37] Bloesch, M.; Omari, S.; Hutter, M.; Siegwart, R., Robust visual inertial odometry using a direct EKFbased approach, 2015 IEEE/RSJ international Conference on Intelligent Robots and Systems (IROS), 298-304 (2015), IEEE
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.