THERMAL IMAGE-BASED OBJECT SEGMENTATION USING DEEP LEARNING (PRE-TRAINED RESNET 34)

Abstract

Technology is developing very rapidly in a pluralistic country, among these technologies are electric vehicles without human intervention, we are more familiar with this with autonomous electric vehicles. The purpose of this autonomous electric vehicle is to suppress human negligence when driving, besides being able to make it easier for the driver to travel without the need to drive it. Before all these things can operate properly, the autonomous electric vehicle needs a detection scheme that can distinguish objects, the segmentation method uses a thermal camera sensor based on Deep Learning that is trained with the required data set. This method uses a Fully Convolutional Network (FCN) with a Residual Network 34 (ResNet 34) architectural model with an image dimension of 640x512 pixels as its feature extraction. The advantage of ResNet 34 is that it is able to do quite a lot of dataset training even though the hardware used is not the most qualified.

The design of this object detection system uses semantic segmentation, Neural Network, and Image Processing methods as input signals in the form of images, and a FLIR thermal camera which is like an eye for a vehicle which receives an input signal, the process from start to finish is processed using the Jetson AGX Xavier. The cability of semantic segmentation was tested offline with 40216 image frames, there are three categories, namely, good, sufficient, and not good. Which includes the good category as many as 23389 frames, 10706 frames is a sufficient category, and for not good category there are 6120 frames. The mean Intersection over Union (IoU) obtained at the time of this study was 78.56%.