Image Shadow Detection and Removal in Autonomous Vehicle Based on Support Vector Machine

,


Introduction
Sensing systems play an important role in autonomous driving. Most sensing systems are camera-based. Cameras can see the world in the same way as humans and are much less expensive than other sensors. However, image recognition based on visible light cameras is greatly affected by the light. In particular, the existence of shadow brings great challenges to the recognition and classification of the environment. The shadow divides a projection object into two parts: a shadow area and a non-shadow area. For the same object, the brightness of the two areas is different, which leads to a decrease in the recognition accuracy of the object, resulting in misjudgment and missing judgment. To improve the accuracy of image recognition of driverless vehicles, we use an appropriate algorithm based on a camera to segment shadow and non-shadow effectively, and we remove the shadow and reproduce the image to improve the accuracy of image recognition of a driverless vehicle.
At present, there are two steps to eliminate shadow: shadow detection and shadow removal. In the research of shadow detection, by considering the reflection light of an image, Dong et al. realized the accurate detection of image shadow in a new means of human-computer interaction, but this method needs manual operation and it is difficult to achieve automation. (1) Shen et al. divided an image into two parts, an incident component and reflection component, by derivative classification, but this method needs a large amount of data. (2) Salvador et al. calculated the average brightness based on the color constancy theory to detect the shadow area. (3) Levine and Bhattacharyya used a support vector machine (SVM) to detect shadow areas according to the brightness difference. (4) Zhou and Liu obtained the seed region for region merging using a visual significance detection algorithm, and then used the region-merging algorithm to realize region merging and shadow detection. The shadow detection effect is better in a simple environment, but the experimental results of this algorithm are poor in a complex environment. (5) Sigut et al. used a watershed algorithm to select seed points in LAB color space to segment an image. (6) In the research related to shadow elimination, Song et al. divided the shadow area into two parts, hard and soft, and matched it with the non-shadow area, using the area compensation method to eliminate the shadow with the color characteristics of the non-shadow area. (7) Guo et al. used classifier training to generate a segmentation curve, and matched the shadow region and non-shadow region. (8) Sun et al. and Graham et al. proposed Retinex image enhancement and entropy minimization based on the mechanism of the gradient domain, realized shadow elimination of a single image, and enhanced image feature information, but the effect of the image color was poor. (9,10) Huang et al. established a dynamic threshold in HSV color space to achieve shadow elimination of a single image. (11) Using a convolutional neural network, Khan et al. divided an image into blocks, and the shadow area of a single image was automatically detected and eliminated. (12) Although the above methods have achieved good results to a certain extent, they are generally only suitable for the elimination of shadows in specific scenes and find it difficult to cope with the complex and changeable environment of vehicles driving on a road. In this paper, a shadow detection method based on an SVM is proposed. By transforming an RGB image into LAB color space and gray space, the shadow region is detected comprehensively by using the changes in image information in the two spaces to improve the accuracy of shadow detection, so as to achieve the effect of optimized shadow elimination.

LAB color space conversion
LAB color space is a uniform color space, which contains all the colors that humans can perceive. LAB color space is composed of a lightness channel L and two color channels A and B. Channel L determines the lightness of an image. Its range of values is [0, 100]. Channel A has a range from magenta to dark green of [127, −128], and channel B has a range from burnt yellow to blue of [127, −128]. LAB color space can be transformed from RGB color space by first converting RGB color space to XYZ color space and then converting XYZ color space to LAB color space. (13) RGB and XYZ color spaces have the following relationship.
In the formulas, R, G, and B are the color channels of pixels and the value range is [0, 100]. L * , A * , and B * are the channel values of LAB color space.

Gray space conversion
In gray space, the gradient feature can reflect the texture information very well, involves little calculation, and it is not sensitive to changes in light. In this paper, the classic weighted average method is used to transform the image from RGB color space to gray space.
In gray space, the horizontal, vertical, and diagonal gradients of a spatial site are obtained, and then the average is obtained using the gradients of each direction, so that regardless of the number of calculated directions, the average measurement is equal, so as to represent the gradient information of the spatial site. Then the gradient value at coordinates (x, y) is expressed as where g(x, y) represents the gray value at coordinates (x, y).

Shadow detection model
In LAB color space, the brightness of the shadow area is much less than that of the nonshadow area, and the pixel values of the non-shadow area and shadow area are markedly different. In an outdoor road scene, owing to the effect of blue sky, the blue component in the shadow area increases more than that in the non-shadow area, and the change in the yellowblue channel on both sides of the shadow boundary is greater than that in the red-green channel. (14) Therefore, the value of the L channel and the gradient values of the L, A, and B channels are observed to detect the shadow area. In addition, the gray value of a pixel covered by the shadow is generally smaller than that of the non-shadow region, and the texture characteristics of the shadow region are also changed. In gray space, the gray gradient can represent the image gradient. Therefore, we establish a shadow detection model based on an SVM by observing the L-channel value: the L, A, and B-channel gradient values, and the gray-scale gradient value in gray space (Fig. 1).

SVM algorithm
The SVM was first proposed by Cortes and Vapnik in 1995. (15) The advantages of SVM are less samples are needed for observation and training and it can deal with nonlinear problems and achieve high-dimensional pattern recognition. The SVM algorithm has strong nonlinear classification ability and can be used for classification and regression, especially as a superior classification tool in pattern recognition and other fields.
A two-class SVM algorithm is used in this research. The two types of training samples are shadow and non-shadow zones. The optimal classification surface of the SVM requires that the classification line not only separates the two types of training samples correctly, that is, the training error rate is zero, but also maximizes the classification interval (margin). As shown in Fig. 2, H is the classification line that separates the two types of samples, H 1 and H 2 are  the lines through the closest points to H and parallel to H, and the margin is the perpendicular distance between H 1 and H 2 . The red and blue points in the graph are support vectors, which are also the points closest to the optimal classification surface.
The two-class SVM algorithm is described as follows. The training data set is (x 1 , y 1 ), (x 2 , y 2 ), ..., The discriminant function is normalized to ensure that all the samples of the two classes satisfy the condition |g(x)| ≥ 1, and when y = −1, |g(x)| ≤ −1; when y = 1, |g(x)| ≥ 1. In particular, when |g(x)| = 1, the corresponding sample point is closest to the classification plane.
The objective of an SVM is to find the decision-making surface with the largest classification interval. Firstly, the margin classification interval needs to be found. Because the nearest sample to the classification surface satisfies |g(x)| = 1, the margin classification interval is defined as The maximum value of the margin is obtained when ||w|| has the minimum value, so the optimal classification surface problem can be expressed as the following optimization problem.
To obtain a high-precision shadow detection model, we use the method of manual marking to add labels to the shadow parts, and the pixel coordinates of manual marking are trained to obtain the shadow area detection model. According to the SVM theory, the closer the sample point is to the optimal classification surface, the greater the probability of misclassification of the sample. Therefore, according to the result obtained from the SVM empirical function, the image is divided to establish a shadow detection model.

Shadow Elimination
When the light intensity in the shadow region is uniform, a uniform shadow will be produced; in the shadow boundary, an uneven shadow will be produced owing to the change in light intensity. Because the shadow will change in the shadow area and at the shadow boundary, we use different methods to eliminate the shadow, as shown in Fig. 3.

Removal of shadow area
In an RGB image with a shadow region, the light intensity of the shadow region is usually lower than that of the non-shadow region, and the hue information of the shadow region also changes. Therefore, we use the method of shadow factor estimation for reference. In LAB color space, the L, A, and B channels are separately processed to complete shadow elimination in the shadow area.
For the L channel, the method of shadow factor estimation is used to modify the brightness information of the image. (17) If there is partial shadow in the image L 0 in the L channel, the image L 0 can be expressed as where L T (x, y) is a shadow-free image in the L channel and C S (x, y) is a shadow factor. The shadow-free image can be obtained by solving the shadow factor C S (x, y).
The shadow factor C S (x, y) in the shadow area is expressed as where C S is the ratio of the average brightness of the shadow area to the average brightness of the non-shadow area in the image, m S is the average brightness of the full shadow area, and m T is the average brightness of the non-shadow area.
In LAB color space, A * and B * are defined as chromaticity attributes. Through the analysis of color values of different images, at a certain color space point in the shadow area, the closest chromaticity value to the non-shadow area is selected, and then the point in the shadow area is processed to make the chromaticity value of the shadow area match that of the non-shadow area. The specific processing method is defined as Here, *

Removal of shadow boundary
In the shadow boundary region, the light intensity will vary, so the shadow factor cannot be regarded as a fixed value. Therefore, in this paper, the shadow boundary is transformed into the gray space, and the shadow boundary is smoothed by the median filter principle. (18) Suppose that, in gray space, the gray value of a space point (x, y) is g(x, y), and the gray value (x, y) output by a median filter is where N is the set of points comprising this point and its adjacent points in the horizontal, vertical, and diagonal directions.
In the shadow area, starting from the point adjacent to the non-shadow area, the gray value of the point and its adjacent points are replaced by the median gray value of the point, and the shadow area is gradually searched and updated to eliminate the shadow in the shadow boundary.

Shadow Removal Results
Shadow is an important factor in the image recognition of autonomous vehicles. To verify the effectiveness of our method, we simulated the method proposed by Guo et al., who realized shadow recognition based on paired regions and recovered shadow-free images by relighting each pixel. (8) Meanwhile, we simulated the method proposed by Yu et al., who derived a linear shadow image model to perform color-line identification and established an offset correction relationship to remove the effect of ambient light. (19) Then, several pictures with shadows were processed by our method, the method proposed by Guo et al.,

Conclusions
In this paper, we presented a method of shadow detection and removal based on the SVM algorithm. Specifically, we established the SVM shadow detection model in LAB color space and gray space to effectively identify the shadow region, shadow boundary, and light region. Moreover, the shadow of the shadow region is eliminated on the basis of the brightness difference between the shadow region and the light region. At the same time, we constructed an image gradient within the shadow boundary, and the boundary shadow was replaced by smooth interpolation to achieve a smooth transition from the light region to the shadow region. The experiments on shadow images containing a road demonstrated the effectiveness of the proposed method. Our method provides users with an option to manually specify the shadow boundary. In the future, an automatic shadow detection algorithm can make this method easier to use.