×

An upgraded-YOLO with object augmentation: mini-UAV detection under low-visibility conditions by improving deep neural networks. (English) Zbl 1498.90021

Summary: Over the last few years, the manufacturing technology of mini-unmanned aerial vehicles (mini-UAVs), also known as mini-drones, has been experiencing a significant evolution. Thus, the early warning optical drone detection, as an important part of intelligent surveillance, is becoming a global research hotspot. In this article, the authors provide a prospective study to prevent any potential hazards that mini-UAVs may cause, especially those that can carry payloads. Subsequently, we regarded the problem of detecting and locating mini-UAVs in different environments as the problem of detecting tiny and very small objects from an aerial perspective. However, the accuracy and speed of existing detection algorithms do not meet the requirements of real-time detection. For solving this problem, we developed a mini-UAV detection model called Upgraded-YOLO based on the state-of-the-art object detection method of YOLOv5. The proposed model is able to perform real-time tiny/small flying object detection. The main contributions of this research are as follows: firstly, an air image dataset of mini-UAVs was built using a Dahua multisensor camera. Secondly, a strategy of instance augmentation is proposed, in which we added small appearance of mini-drones to samples of the custom air image dataset. Thirdly, in addition to hyperparameter tuning and optimization operations, shallow layers are added to improve the model’s ability to detect mini-UAVs. A comparative study with several contemporary object detectors proved that the Upgraded-YOLO performed better. Therefore, the proposed mini-UAV detection technology can be deployed in a monitoring center in order to protect a strategic installation even in low-visibility conditions.

MSC:

90B06 Transportation, logistics and supply chain management
68T05 Learning and adaptive systems in artificial intelligence
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] USAF (2009) Unmanned Aircraft Systems Flight Plan 2009-2047, Technical report, unclassified, United States Air Force, Washington DC, pp. 24-27
[2] Doyle DD (2013) Real-time, multiple, pan/tilt/zoom, computer vision tracking, and 3D position estimating system for small unmanned aircraft system metrology. DEPARTMENT OF THE AIR FORCE, AIR UNIVERSITY, Wright-Patterson Air Force Base, Ohio, USA. Jeffrey Maddalon
[3] Maddalon J, Hayhurst KJ, Koppen DM, Upchurch JM (2013) Perspectives on unmanned aircraft classification for civil airworthiness standards. Langley Research Center, Hampton, Virginia. NASA/TM-2013-217969
[4] Lykou G, Moustakas D, Gritzalis D (2020) Defending airports from UAS: a survey on cyber-attacks and counter-drone sensing technologies. Sensors 20(12):3537
[5] Official DJI website: https://www.dji.com/matrice600-pro/info (last Accessed on 23 Mar 2021)
[6] Seidaliyeva U, Akhmetov D, Ilipbayeva L, Matson ET (2020) Real-time and accurate drone detection in a video with a static background. Sensors 20(14):3856
[7] Official DJI website: https://www.dji.com/t16/info#downloads (Last Accessed on 23 Jun 2021)
[8] Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497
[9] He K, Gkioxari G, Dollár P, Girshick R (2018) Mask R-CNN. arXiv preprint arXiv:1703.06870
[10] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Alexander C (2015) Berg. SSD: single shot MultiBox detector. arXiv preprint arXiv:1512.02325
[11] Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.779-788
[12] Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. arXiv preprint arXiv:1612.08242
[13] Everingham M, Ali Eslami SM, Van Gool L, Williams CKI, Winn J, Zisserman A (2015) The Pascal Visual Object Classes challenge: a retrospective. Int J Comput Vis 111:98-136
[14] Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The PASCAL Visual Object Classes (VOC) challenge. Int J Comput Vis 88:303-338
[15] Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767
[16] Lawal MO (2021) Tomato detection based on modified YOLOv3 framework. Sci Rep Jan 14;11(1):1447. doi:10.1038/s41598-021-81216-5. PMID: 33446897; PMCID: PMC7809275
[17] Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Lawrence Zitnick C (2014) Microsoft COCO: common objects in context. in 13th European Conference on Computer Vision, pp. 740-755
[18] Kim D-H (2019) Evaluation of COCO Validation 2017 Dataset with YOLOv3. Journal of Multidisciplinary Engineering Science and Technology 6(7)
[19] Bochkovskiy A, Wang CY, Liao HYM (2020) YOLOv4: optimal speed and accuracy of object detection, arXiv preprint arXiv: 2004.10934
[20] Wang Z, Wu Y, Yang L, Thirunavukarasu A, Evison C, Zhao Y (2021) Fast personal protective equipment detection for real construction sites using deep learning approaches Sensors 21(10):3478
[21] Ultralytics YOLOv5 and Vision AI, Madrid, Spain. Available online: http://www.ultralytics.com (Last Accessed on 03 Aug 2021)
[22] Kharel S, Ahmed KR (2021) Potholes detection using deep learning and area estimation using image processing, Proceedings of SAI Intelligent Systems Conference, IntelliSys 2021: Intelligent Systems and Applications 296:373-388
[23] Wang X, Wei J, Liu Y, Li J, Zhang Z, Chen J, Jiang B (2021) Research on morphological detection of FR I and FR II radio galaxies based on improved YOLOv5. Universe 7(7):211
[24] Yan B, Fan P, Lei X, Liu Z, Yang F (2021) A real-time apple targets detection method for picking robot based on improved YOLOv5. Remote Sensing 13(9):1619
[25] Yang G, Feng W, Jin J, Lei Q, Li X, Gui G, Wang W (2020) Face mask recognition system with YOLOV5 based on image recognition. Proceedings of 2020 IEEE 6th International Conference on Computer and Communications, IEEE Xplore , pp. 1398-1404
[26] COCO dataset. Available online: https://cocodataset.org/#home (Last Accessed on 17 Sept 2021)
[27] Adibhatla VA, Chih H-C, Hsu C-C, Cheng J, Abbod MF, Shieh J-S (2021) Applying deep learning to defect detection in printed circuit boards via a newest model of you-only-look-once. Math Biosci Eng 18(4):4411-4428
[28] Agarwal S, Du Terrail JO, Jurie F (2018) Recent advances in object detection in the age of deep convolutional neural networks. arXiv preprint arXiv: 1809.03193
[29] Yao J, Qi J, Zhang J, Shao H, Yang J, Li X (2021) A real-time detection algorithm for kiwifruit defects based on YOLOv5. Electronics 10(14):1711
[30] Dahua Technology. Available online: https://www.dahuasecurity.com/products/All-Products/Thermal-Cameras/Wizmind-Series/TPC-8-Series/TPC-PT8621C (Last Accessed on 04 Aug 2021)
[31] Zhang Y, Yongliang S, Jun Z (2019) An improved tiny-yolov3 pedestrian detection algorithm. Digital Signal Processing 183:17-23
[32] NguyenN-D, Do T, Ngo TD, Le D-D (2020) An evaluation of deep learning methods for small object detection. J Electr Comput Eng 2020(3189691):18
[33] Delleji T, Fekih H, Chtourou Z (2020) Deep learning-based approach for detection and classification of micro/mini drones. In 2020 4th International Conference on Advanced Systems and Emergent Technologies (IC_ASET), pp. 332-337
[34] Kisantal M, Wojna Z, Murawski J, Naruniec J, Cho K (2020) Augmentation for small object detection. arXiv preprint arXiv: 1902.07296
[35] Tong K, Wu Y, Zhou F (2020) Recent advances in small object detection based on deep learning: a review. Image and Vision Computing, Science Direct, ELSEVIER 97
[36] Pang J, Li C, Shi J, Xu Z, Feng H (2019) R2-CNN: fast tiny object detection in large-scale remote sensing images. IEEE Trans Geosci Remote Sens 57(8)
[37] Zhang Y, Bai Y, Ding M, Ghanem B (2020) Multi-task generative adversarial network for detecting small objects in the wild. Int J Comput Vis pp. 1810-1828
[38] Chen C, Liu M-Y, Tuzel O, Xiao J (2016) R-CNN for small object detection. Asian Conference on Computer Vision ACCV, pp.214-230
[39] Du Z, Yin J, Yang J (2019) Expanding receptive field YOLO for small object detection. Journal of Physics. Conference Series, 3rd International Conference on Electrical, Mechanical and Computer Engineering, Guizhou, China 1314
[40] Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2018) Focal loss for dense object detection. IEEE Transactions on PatternAnalysis and Machine Intelligence 42(2):318-327
[41] Zheng Z, Wang P, Liu W, Ye R, Ren D (2020) Distance-IoU loss: faster and better learning for bounding box regression. AAAI Conference on Artificial Intelligence 34(07)
[42] Rezatofighi H, Tsoi N, Gwak JY, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. arXiv preprint arXiv: 1902.09630
[43] Madasamy K, Shanmuganathan V, Kandasamy V, Lee MY, Thangadurai M (2021) OSDDY: embedded system-based object surveillance detection system with small drone using deep YOLO. EURASIP Journal on Image and Video Processing 2021:19
[44] Wang, X.; Song, J., ICIoU: improved loss based on complete intersection over union for bounding box regression, IEEE Access, 9, 105686-105695 (2021) · doi:10.1109/ACCESS.2021.3100414
[45] Zhou J, Tian Y, Yuan C, Yin K, Yang G, Wen M (2019) Improved UAV opium poppy detection using an updated YOLOv3 model. Sensors 19(22):4851
[46] Wicaksono AS, Supianto AA (2018) Hyper parameter optimization using genetic algorithm on machine learning methods for online news popularity prediction. International Journal of Advanced Computer Science and Applications(IJACSA) 9(12)
[47] Chawla, S., Application of genetic algorithm and backpropagation neural network for effective personalize web search-based on clustered query sessions, International Journal of Applied Evolutionary Computation (IJAEC), 7, 1, 33-49 (2016) · doi:10.4018/IJAEC.2016010103
[48] Kingma DP, Ba JL (2014) A method for stochastic optimization. arXiv preprint arXiv: 1412.6980
[49] Nicolai W, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. arXiv preprint arXiv: 1703.07402
[50] Jiang N, Peng X, Yu X, Wang Q, Xing J, Li G, Zhao J, Guo G, Han Z (2021) Anti-UAV: a large multi-modal benchmark for UAV tracking. arXiv preprint arXiv: 2101.08466
[51] Zhao J, Wang G, Li J, Jin L, Fan N, Wang M, Wang X, Yong T, Deng Y, Guo Y, Ge S, Guo G (2021) The 2nd Anti-UAV Workshop & Challenge: methods and results. arXiv preprint arXiv: 2108.09909
[52] website Roboflow. PyTorch Object Detection, YOLOv5 is Here, https://models.roboflow.com/object-detection/yolov5 (Last Accessed on 03 Dec 2021)
[53] Yuxin F, Liao B, Wang X, Fang J, Qi J, Wu R, Niu J, Liu W (2021) You only look at one sequence: rethinking transformer in vision through object detection. arXiv preprint arXiv: 2106.00666
[54] Adibhatla VA, Chih H-C, Hsu C-C, Cheng J, Abbod MF, Shieh J-S (2021) Applying deep learning to defect detection in printed circuit boards via a newest model of you-only-look-once. Math Biosci Eng 18
[55] Sahin O, Ozer S (2021) YOLODrone: improved YOLO architecture for object detection in drone images. In 2021 44th International Conference on Telecommunications and Signal Processing (TSP), pp. 361-365. IEEE
[56] ImageNet. ImageNet Large Scale Visual Recognition Challenge 2017 (ILSVRC2017). Available online: https://image-net.org/challenges/LSVRC/2017/ (Last Accessed on 22 Jul 2022)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.