Detection of Defect Inside Duct Using Recurrent Neural Networks

Prestressed concrete (PSC) box-girder bridges are grouted after inserting a tendon in the duct in order to protect the tendon from the risk of corrosion. However, because of the small inside diameter of the duct, it is difficult to completely fill it with concrete (grout) and even small mistakes can cause defects. Today, the complex and professional analysis of the signals measured by nondestructive testing (NDT) is conducted by a geophysicist to detect defects. However, owing to the limitations of NDT, accurate detection is very difficult. We introduce a deep learning model for defect detection to help working-level officials in the field immediately perform defect detection without the need for such complex and professional analysis. In this study, we apply the long short-term memory (LSTM), which is one of the deep learning models with good performance for time series data. Moreover, we use raw impact echo (IE) signals measured using IE equipment and some characteristics of the bridge (concrete thickness, depth of duct, and distance between the measured point and hit point). At this time, because the value of the signal was very small, we standardized the raw data to perform normalization. As a result of the experiment, we obtained an average accuracy of 82.58%.


Introduction
Prestressed concrete (PSC) box-girder bridges are resistant to bending and twisting, and are widely used for highway and rail bridges owing to their relatively low cost and ease of maintenance. (1) However, long-term cracks have been observed over the past decades, which can lead to the structural collapse of bridges. (2) In particular, a tendon, which is a core element of PSC box-girder bridges, is likely to collapse rapidly owing to corrosion. Thus, by inserting the tendon into the duct and grouting the duct, corrosion of the tendon is prevented. However, since the inside diameter of the duct is very small, it is difficult to grout the duct completely, so small cavities can remain. Owing to this unpredictable damage, the status of the ducts inside PSC box-girder bridges must be determined. Therefore, in many cases, nondestructive testing (NDT) is used to inspect the ducts inside PSC box-girder bridges.
In the past, the task of exploring the inside of concrete structures has required very accurate techniques because it is very important to check the stability of concrete structures. NDT has been used to determine the state of concrete. In NDT, ultrasonic waves, impact echo (IE), and ground-penetrating radar technologies are used. (3)(4)(5)(6)(7)(8) These methods can detect defects such as cracks or cavities occurring inside a concrete structure. Among them, the IE method (8) is one of the most well-known NDT methods, and it detects the state inside concrete using elastic waves.
NDT such as by IE methods is also used for PSC box-girder bridges because the bridges can be inspected from the surface of their concrete structures. (9) However, these NDT methods require a complex and academic interpretation of the data by a geophysicist. Therefore, the ducts inside PSC box-girder bridges cannot be directly checked after completion, and it takes much time and money to perform NDT by, for example, the IE method. We intend to introduce a deep learning model for defect detection so that practitioners in the field can immediately determine the presence or absence of defects without the need for such complex and professional analysis. In this study, the type of defect we want to detect is cavities that occur inside a duct.
In this study, we fabricated concrete specimens using the same materials as those of actual PSC box-girder bridges. With these concrete specimens, the raw IE signals measured by IE equipment using a mounted sensor and the information of the concrete specimens are used as the features of the deep learning model. Signals such as raw IE signals have time-series characteristics. Therefore, in this study, we use the long short-term memory (LSTM) model, which shows good performance for time-series data. (10) The use of NDT signals requires a complex preprocess used in signal processing. In this study, we reduced the training time by performing data normalization by a standardization method using the mean and standard deviation of the data. In addition to the raw IE signals, three parameters of the concrete specimen (concrete thickness, depth of duct, and distance between the measured point and hit point) are used as additional features.
In this study, seven concrete specimens were fabricated and the data of PSC box-girder bridges were collected from these concrete specimens. The proposed model computes raw IE signals (0-5000 μs) and the three features related to the concrete specimen in each LSTM cell. That is, it learns meaningful information about each feature and passes it on to the next feature. Then, each LSTM cell computes 200-dimensional outputs from the input values. These outputs are used as inputs to the softmax layer, and finally, the model learns the presence or absence of a defect.
We outline the related works in Sect. 2. In Sect. 3, we explain PSC box-girder bridges and the fabricated concrete specimens. In Sect. 4, we show our methodology, and in Sect. 5, we discuss the experimental results. Finally, concluding remarks end this paper in Sect. 6.

Related Work
In recent years, several researchers have used machine learning methods to detect corrosion or cracks in concrete structures. Their studies were based on analyzing signals or images obtained from NDT. Zhao et al. detected defects in concrete structures by applying a piezoceramic-inducted ultrasonic wave and the time reversal method. (11) Ultrasonic waves obtained by NDT are used as signals containing defect information. To improve accuracy, the time reversal method is used to identify the location and characteristics of defects. At this time, the defect is analyzed by considering the interaction between the steel and concrete and by imaging the defect through cross-sectional scanning.
The study of Zhang et al. (12) is similar to our study. They used the wavelet transform, (13) one of the signal processing techniques, to extract features from raw IE signals. The wavelet transform is suitable for analyzing periodic and transient noise signals because they can provide both time and frequency information simultaneously. These features are applied to the extreme learning machine (ELM), which constitutes a single hidden layer with multiple neurons and performs better than a multilayer perceptron or support vector machine (SVM) in terms of speed and complexity. (14) However, Zhang et al. fabricated concrete specimens that are much simpler than actual PSC box-girder structures (Fig. 1).
The characteristics of the concrete specimen fabricated by Zhang et al. (Fig. 1) are as follows. First, it seems that the bar was laid in a very simple manner. Second, it was difficult to check whether a tendon was inserted. Third, a PE pipe was used instead of a metallic pipe, which is used for the duct of an actual PSC box-girder bridge. The void existing inside the pipe was also different from that in the field. Fourth, the concrete strength was 30 MPa, not the 40 MPa of the PSC box-girder bridge. Fifth, the concrete was carefully surface-treated. Finally, the exploration method was designed to obtain the best data, unlike in the actual field. These conditions can make the signal from the concrete specimen more clearly identified, but in the practical application in the field, there are many difficulties.
In our study, we aim to develop a method that can be applied to actual exploration work. In other words, we fabricated concrete specimens as a general form of the PSC box-girder structure used in the field. Moreover, the exploration conditions were applied in the same way as in the field to collect the data.

Materials for Experiment
In this section, we describe the structure of a PSC box-girder bridge and the concrete specimens fabricated similarly to the actual PSC box-girder structure. We also describe the process of obtaining data by exploring the interior of these concrete specimens.

PSC box-girder bridge
Concrete is strong against compressive stress but weak against tensile stress. To overcome this disadvantage, reinforcing bars with high tensile strength are inserted into the concrete. In PSC box-girder bridges, a tendon is inserted into concrete then pulled firmly to ensure that compressive stress is applied to the concrete. When a tendon is inserted in the manner shown in Fig. 2, an uplifting force is generated and concrete having a high bending strength is produced.
However, tendons are at risk of corrosion when exposed to air, which can lead to the collapse of bridges. Therefore, after inserting a tendon into a duct, grouting is performed inside the duct. Grouting prevents the tendon from being exposed to air. Figure 3 shows the grouting process in the duct.

Concrete specimens used for data collection
In this study, concrete specimens were fabricated with the general structure and materials of actual PSC box-girder bridges. The rebar diameters were 16 and 25 mm, and the lattice arrangement was at intervals of 125 and 150 mm. This is a commonly used arrangement for real structures. In addition, more than 15 tendons were arranged inside each duct to fabricate  the actual structure. A metallic pipe used in the actual field was used, and voids were reproduced similarly to those in the actual field. The concrete surface of each specimen was also fabricated to be rough and irregular. Concrete with a strength of 40 MPa, which is often used in PSC box-girder bridges, was used. The shape and properties of the concrete specimens thus produced are shown in Fig. 4 and Table 1, respectively.
In this study, seven different PSC box-girder blocks with different sizes and locations of defects were fabricated by the standard casting process. We adjusted the thickness of all PSC concrete blocks to 60 cm. The ducts were located at depths of 10, 20, 30, and 40 cm from the surface. We made one or two defects of square or trapezoidal shape in the duct. Figure 5 shows blocks with various built-in ducts.     We collected the raw IE signals using IE equipment under the same conditions as in the actual field. The IE equipment uses sensors, which are mounted to measure raw IE signals from the concrete structure. The sensors measure the raw IE signals obtained using a striker mounted on the IE equipment. Generally, the IE equipment simply processes the measured signal so that the user can check the peak frequency. Therefore, this equipment has limitations in the detection of defects in concrete in complex environments. However, since raw IE signals contain more information than the peak frequency, we utilize them. Figure 6 shows the IE equipment and how it works. The raw IE signals generated by this equipment are converted to data and stored in a database to be used for deep learning through the process shown in Fig. 7. Figure 8 shows the structure of the proposed model for defect detection inside the duct in a PSC box-girder bridge using LSTM. This model takes raw IE signals and three additional features related to the specimen in sequence and performs its learning process. Additional features related to the specimen are also used to learn the effect of the characteristics of the specimen on the signals. The following is a description of each element of learning based on LSTM.

Standardization
In this study, normalization is performed by applying the standardization method to raw IE signal data. This is because the raw IE signals have very small values, which can degrade the efficiency of learning. The standardization method uses the mean and standard deviation of the data. The position of a given value relative to the mean of the entire distribution is converted to the number of standard deviations. The standardization in this study is carried out using Here, x is the value of the current data, μ is the mean value of the entire data, and σ is the standard deviation of the population. The normalized IE signal is used as input to the LSTM layer with three additional features associated with the specimen.

LSTM
A recurrent neural network (RNN) is one of the artificial neural network models in which hidden nodes are connected to a directional edge to form a cyclic structure. This model is known to be suitable for processing sequential data such as text, voices, and signals. (15)(16)(17) The most significant characteristic of the RNN, as shown in Fig. 9, is that it has a structure to receive and update the hidden state (h i ) from the previous stage cell. Thus, it has a structure to receive and learn the previous information. As shown in Fig. 9, the RNN has a structure that accepts inputs and outputs regardless of the length of the sequence, which is advantageous in that it is flexible and can be varied depending on the task.
However, there are also disadvantages to RNNs. For example, if the distance between the two inputs is too large, the gradient is less likely to be properly learned because of the decrease in the learning efficiency of the backpropagation process. This is called the vanishing gradient problem. To solve this problem, we propose the use of LSTM.
LSTM uses two additional gates in the hidden state of the existing RNN. The two gates are the forget gate and the input gate, which operate as described by Eqs. (2) to (7). In this case, the t t i g  operator is typically used to represent element-wise multiplication in LSTM.
First, the forget gate ( f t ) is used to forget the previous information. The sigmoid function (σ) is applied by receiving h t−1 transmitted from the previous cell and the current input x t . The sigmoid function outputs a value from 0 to 1; if the result of the calculation is close to 0, the previous information is forgotten, and if it is close to 1, the previous information is stored. Second, the input gate ( t t i g  ) is used to store the current information. The input gate calculates the following: i t is calculated by applying the sigmoid function to h t−1 received from the previous cell and the current input x t , and g t is calculated by applying the hyperbolic tangent to the same value. By applying the t t i g  operator to these two values, the result for the current input x t is output. Fig. 9. Basic structure of RNN. (a) Basic, (b) one to many, (c) many to one, and (d) many to many.
In this study, each feature was input to the LSTM cell and the results were output as a vector of 200 dimensions for each feature. As shown in Fig. 9(a), one output is generated for one input, and all the generated vectors are concatenated into a single vector.

Softmax function
In artificial neural networks, the softmax function is used to classify the class of input data. The softmax function classifies a given example by proceeding with normalization in the output layer. For example, when determining whether a cancer is benign (1) or malignant (0), this function computes the probability of being positive by inputting data received from the hidden layer. This function finally assigns values of X% and (100 -X)% to positive and negative, respectively, and selects the group with the highest probability as the classification result. The softmax function is expressed by In Eq. (8), K denotes the number of classes to be classified, and the softmax function is calculated by an exponential function for the input x. In this way, a model that detects the existence of defects is learned.

Experimental Results
In this study, the LSTM model is trained by using the raw IE signals and some features of our specimens. Hyperparameters used in this study are shown in Table 2. We used the number of training epochs, batch size, the number of LSTM output dimensions and activation functions as our hyperparameters. Hyperparameters are selected to show the highest performance. When selecting the best number of epochs, we consider only the level of the cost. However, the other parameters were selected by considering the overall performance.
The numbers of features used as input was 232, comparing 229 raw IE signals plus the concrete thickness, the depth of the duct, and the distance between the measured point and hit point. The raw IE signals were normalized by the standardization method and then used for learning. The total number of data collected through the experiment was 3031, and the ratio of the numbers of data in the training set to the test set was 80% to 20%. Table 3 shows a summary of the features used in this experiment. Experiments with various hyperparameters showed that the accuracy of the best hyperparameters is 82.58%. This performance is the average result of 10 iterations after randomly mixing data. The overall performance evaluation and confusion matrix of this experiment are respectively shown in Tables 4 and 5.
In Table 4, the accuracy is 82.58%, and the F1 scores are 77% for Normal and 86% for Defect. These are very high compared with the originally expected performance and suggest the high potential of this method. In particular, it is fortunate that the Defect F1 score is higher than that for Normal in this study because missing the defects causes disasters. However, the lower recall value than the precision value for Defect is undesirable and we need to find a way to significantly improve the recall in the future to ensure that detects are found and to put the method to practical use.
To confirm the reliability of the proposed model, the performance was verified by comparison with the model of Zhang et al. In addition, the performance of learning the three additional features used in this study together with the model of Zhang et al. was also confirmed. Here, the experiment confirmed the performance with the average result of 10 iterations. The results are shown in Table 6.
As shown in Table 6, the proposed model showed better performance for data of concrete specimens that are similar to concrete in actual PSC box-girder bridges.

Conclusions
The existing methods of detecting defects in ducts require a complex and academic interpretation of the results of NDT, which is expensive and time-consuming. We have introduced the LSTM model so that a field practitioner can immediately determine whether ducts have defects. In this study, we proposed a duct fault detection model for PSC box-girder bridges by applying raw IE signals and the characteristics of concrete specimens (concrete thickness, depth of duct, and distance between the measured point and hit point) to the LSTM model. The characteristics of the specimen were added because inspection is carried out at various positions to detect defects in the duct. The proposed model showed 82.58% accuracy.
In the future, first, we will look for a more seamless way to collect data. We have found that there is much difficulty in collecting data on PSC box-girder bridges. If we use the same materials as in actual PSC box-girder bridges and collect data from a larger variety of specimens, it would be possible to create a better defect detection model. Second, we will build a more on-the-spot model. By default, raw IE signals will have different appearances depending on the situation in the field. To prepare for this diversity, various situations of the field should be modeled and integrated. Finally, if possible, we will utilize aged real bridges for experiments. This should be possible if several companies and researchers work together.