An Intelligent Vision System for Automatic Forest Fire Surveillance
Subject Areas : electrical and computer engineeringMohammad Sadegh Kayhanpanah 1 , Behrooz Koohestani 2 *
1 -
2 -
Keywords: forest fire, unmanned aerial vehicles, image processing, deep learning, convolutional neural networks,
Abstract :
Fighting forest fires to avoid their potential dangers as well as protect natural resources is a challenge for researchers. The goal of this research is to identify the features of fire and smoke from the unmanned aerial vehicle (UAV) visual images for classification, object detection, and image segmentation. Because forests are highly complex and nonstructured environments, the use of the vision system is still having problems such as the analogues of flame characteristics to sunlight, plants, and animals, or the smoke blocking the images of the fire, which causes false alarms. The proposed method in this research is the use of convolutional neural networks (CNNs) as a deep learning method that can automatically extract or generate features in different layers. First, we collect data and increase them according to data augmentation methods, and then, the use of a 12-layer network for classification as well as transfer learning method for segmentation of images is proposed. The results show that the data augmentation method used due to resizing and processing the input images to the network to prevent the drastic reduction of the features in the original images and also the CNNs used can extract the fire and smoke features in the images well and finally detect and localize them.
[1] C. Yuan, Y. M. Zhang, and Z. X. Liu, "A survey on technologies for automatic forest fire monitoring, detection, and fighting using unmanned aerial vehicles and remote sensing techniques," Canadian J. of Forest Research, vol. 45, no. 7, pp. 783-792, Jul. 2015.
[2] J. R. Martinez-de Dios, et al., "Computer vision techniques for forest fire perception," Image Vision Comput, vol. 26, no. 4, pp. 550-562, Apr. 2008.
[3] K. G. Hirsch and P. Fuglem, Canadian Wildland Fire Strategy: Background Syntheses, Analyses, and Perspectives, Canadian Council for Forest Ministries, Natural Resources Canada, Canadian Forest Service, Northern Forestry Centre, Edmonton, Alberta, 2006.
[4] H. Lin, Z. Liu, T. Zhao, and Y. Zhang, "Early warning system of forest fire detection based on video technology," in Proc. of the 9th Int. Symp. on Linear Drives for Industry Applications, vol. 3, pp. 751-758, Hangzhou, China, 7-10 Jul. 2013.
[5] E. Den Breejen, et al., "Autonomous forest fire detection," in Proc. of the 3rd Int. Conf. on Forest Fire Research, pp. 2003-2012, Luso, Portugal, 16-20 Nov. 1998.
[6] R. W. Beard, T. W. McLain, D. B. Nelson, D. Kingston, and D. Johanson, "Decentralized cooperative aerial surveillance using fixed-wing miniature UAVs," Proceedings of the IEEE, vol. 94, no. 7, pp. 1306-1324, Jul. 2006.
[7] H. Olsson, et al., "Current and emerging operational uses of remote sensing in Swedish forestry," in Proc. of the 5th Annual Forest Inventory and Analysis Symp. USDA Forest Service, pp. 39-46, Portland, ONn, USA, 3-6 Oct. 2005.
[8] J. A. J. Berni, P. J. Zarco-Tejada, L. Surez, and E. Fereres, "Thermal and narrowband multispectral remote sensing for vegetation monitoring from an unmanned aerial vehicle," IEEE Trans. Geosci. Remote Sens., vol. 47, no. 3, pp. 722-738, Mar. 2009.
[9] J. Everaerts, "The use of unmanned aerial vehicles (UAVs) for remote sensing and mapping," Proc. Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 37, Pt. B1, pp. 1187-1192, Beijing, China, 3-11 Jul. 2008.
[10] V. G. Ambrosia and T. Zajkowski, "Selection of appropriate class UAS/sensors to support fire monitoring: experiences in the United States," in Handbook of Unmanned Aerial Vehicles, Edited by K. P. Valavanis and G. J. Vachtsevano, Springer, Netherlands, pp. 2723-2754, 2015.
[11] R. Sporea, Color Space and Color Model, (http://www.opticallimits.com)
[12] M. Pietikainen, A. Hadid, G. Zhao, and T. Ahonen, "Local binary pattern for still images," In: Computer Vision Using Local Binary Patterns. Computational Imaging and Vision, vol 40, pp. 13-47, London: Springer, 2011.
[13] T. X. Truong and J. M. Kim, "Fire flame detection in video sequences using multi-stage pattern recognition techniques," Engineering Applications of Artificial Injelligence J., vol. 25, no. 7, pp. 1365-1372, Oct. 2012.
[14] G. W. Lindsay, "Convolutional neural networks as a model of the visual system: past, present, and future," J. of Cognitive Neuroscience, vol. 33, no. 10, pp. 1-15, Sept. 2020.
[15] S. Frizzi, et al., "Convolutional neural network for video fire and smoke detection," in Proc. IECON 42nd Annual Conf. of the IEEE Industrial Electronics Society, pp. 877-882, Florence, Italy, 23-26 Oct. 2016.
[16] Y. Zhao, J. Ma, X. Li, and J. Zhang, "Saliency detection and deep learning-based wildfire identification in UAV imagery," Sensors, vol. 18, no. 3, Article ID: 712, Feb. 2018.
[17] H. Pan, D. Badawi, and A. E. Cetin, "Computationally efficient wildfire detection method using a deep convolutional network pruned via Fourier analysis," Sensors, vol. 20, no. 10, Article ID: 2891, May 2020.
[18] https://github.com/tensorflow/models/tree/master/research/object_detection
[19] K. He, G. Gkioxari, P. Dollar, and R. Girshick, Mask R-CNN, In ICCV, 2017.
[20] T. Fawcet, "An introduction to ROC analysis," Pattern Recognition Letters, vol. 27, no. 8, pp. 861-874, Jun. 2006.
[21] D. M. W. Powers, "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness & correlation," J. of Machine Learning Technologies, vol. 2, no. 1, pp. 37-63, 2011.
[22] K. M. Ting, C. Sammut, and G. I. Webb, Encyclopedia of Machine Learning, Boston: Springer, 2011.
[23] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. of the 25th Int. Conf. on Neural Information Processing Systems, pp. 1097-1105, Lake Tahoe, NA, USA, 3-6 Dec. 2012.