Nondestructive Evaluation of Ducts in Prestressed Concrete Bridges Using Heterogeneous Neural Networks and Impact-echo

1Department of Convergence Software, Hallym University, 1, Hallymdaehak-gil, Chuncheon, Gangwon-do 24252, Republic of Korea 2BIT Research Lab., Hallym University, 1, Hallymdaehak-gil, Chuncheon, Gangwon-do 24252, Republic of Korea 3AIbridge Co., Ltd., Room 502, 20, Gwangnaru-ro 54-gil, Gwangjin-gu, Seoul 05117, Republic of Korea 4Korea Institute of Civil Engineering and Building Technology, 283, Goyang-daero, Ilsanseo-gu, Goyang-si, Gyeonggi-do 10223, Republic of Korea


Introduction
Over the last several decades, many prestressed concrete (PSC) girder bridges have been built for reasons such as economy and stability. Their advantages are attributable to the tension of the tendons inserted inside ducts. PSC girder bridges are widely used for highways and railroads that must be designed in long sections. (1) However, various defects (deterioration, cracks, or internal defects) can form in a PSC structure used for a long time, which, if not detected early, may lead to a major accident. (2) Various nondestructive evaluation (NDE) methods have been devised to identify defects in concrete structures. (3)(4)(5)(6)(7) NDE can evaluate the internal condition of a concrete structure without damaging the surface. This characteristic has the advantage that it can be applied to an entire large-scale concrete structure for more efficient evaluation. A typical NDE method used for concrete structures is the impact-echo (IE) method. (8) However, owing to the complex internal structure of a PSC girder bridge, an appropriate NDE method has not yet been established. According to a US government report, the best performance of NDE-based geophysical exploration detected only 33% of voids in a duct, (9) limiting its application in the field.
Some recent studies have introduced machine learning (or deep learning) to detect defects in bridges. (10)(11)(12)(13) These studies focused on identifying cracks on the surface or deck of a bridge. However, corrosion of the tendons of PSC girder bridges caused by exposure to air may cause the sudden structural collapse of bridges. To prevent this problem, tendons are inserted in ducts and then protected by pouring concrete into the ducts. However, a duct has a small inner diameter and is designed in a complex structure, so a void may occur inside the duct. Therefore, checking ducts for internal voids is very important to confirm the safety of an existing or new bridge.
In this study, by introducing a neural network into the IE method, we propose a model for detecting the internal voids of ducts with higher accuracy than before. We collected IE signals by fabricating a PSC specimen as similar as possible to that of a PSC girder bridge. Also, we gave the sample various thicknesses. A digital filter (low-pass filter, high-pass filter, or bandpass filter) was applied to the IE signal to remove noise and improve its quality. Then the frequency spectrum of the IE signal was analyzed and trained with a neural network. The filtered IE signal was trained with a long short-term memory (LSTM), (14) which has demonstrated excellent performance for time-series data. In contrast, since the frequency spectrum is not a time series, we trained it with a feed-forward neural network (FFNN). The outputs of the two networks perform a multiplication operation to compute the integrated features and train them. Through this, the presence or absence of a void is predicted with a classifier composed of a softmax function.
We trained the IE signal efficiently by setting specific ranges of 0-5000 and 0-10000 μs, and we evaluated the performance for each range. In addition, K-fold cross-validation was used to prevent bias in the data used for evaluation. In our experiment, average accuracies of 95.084% and 97.474% were obtained for the ranges of 0-5000 and 0-10000 μs, respectively We describe related works in Sect. 2. In Sect. 3, we explain the PSC specimen and IE equipment used in the experiment. In Sect. 4, we explain our proposed methodology. In Sect. 5, we discuss the experimental results. Finally, concluding remarks are given in Sect. 6.

Related Works
The IE method is the impact-generated stress wave-based NDE method designed to evaluate concrete structures, where the waves are reflected by internal defects and external surfaces. (8) It has been widely used in the detection of defects in concrete structures. (15)(16)(17) In general, the IE method analyzes the frequency spectrum generated through the Fourier transform and is effective. However, the frequency spectrum is difficult to interpret correctly owing to its multiple peaks and the vibration and thickness of the structure to be analyzed. (18) Attempts have been made to apply signal processing to the IE method to improve its performance. Signalprocessing-based approaches have been applied to time-frequency analysis such as the shorttime Fourier transform, Hilbert-Huang transform, empirical mode decomposition (EMD), and wavelet transform (WT). (18)(19)(20)(21)(22) Shokouhi et al. (23) confirmed the WT to be the most efficient time-frequency analysis, and the WT is still widely applied to IE signals in signal-processingbased approaches. Recently, an approach that applied variational mode decomposition (VMD) instead of EMD has also been tried. (24) In contrast, machine-learning-based approaches focus on evaluating specific elements or the overall state of structures. The main topics for which these approaches are applied are structural condition monitoring (SHM) (10) and NDE. (11)(12)(13) Zhang et al. (11) performed a full-condition assessment of concrete by applying an extreme learning machine (ELM), (25) which was the latest technology at the time, to an IE signal processed by the WT, and confirmed the possibility of using machine learning with excellent performance. Dorafshan and Azari confirmed the potential of the convolutional neural network (CNN) (26) and LSTM (14) for the detection of internal defects in a bridge deck. (12,13) Owing to this trend, attempts have recently been made to create a standard dataset of IE signals for concrete structures. (27) As such, machine learning has been applied in many studies. However, although internal void detection of ducts is a very important problem, many studies have mainly used signalprocessing-based approaches and machine learning has been rarely applied. (28) Oh et al. (28) trained a standardized raw IE signal and structure information (concrete thickness, depth of duct, distance between the measured point and impact point) with LSTM to detect the internal voids of ducts. The raw IE signal contains various noises caused by the environment. However, it is difficult to remove noise effectively by standardization, which may disturb the training of neural network models. To effectively apply machine learning to this problem, it is important to improve the quality of data by appropriate signal processing and to develop an appropriate model without overfitting or underfitting.

Materials
In this section, we describe the PSC specimens and IE measurement equipment manufactured to proceed with the experiment. We manufactured PSC specimens in three formats, and the IE equipment was manufactured by Olson Instruments Inc. (https://olsoninstruments.com/).

PSC specimens
To train and evaluate the proposed method, we construct PSC specimens and collect IE signals. In a PSC girder bridge, tendons are inserted inside ducts after the ducts are installed. The tendons are pulled in both directions and then fixed to generate compressive stress. The duct structure shown in Fig. 1 is used to generate an uplift force, making it resilient against bending stress. After that, grouting (casting concrete) is performed inside the duct to prevent the occurrence of defects due to exposure of the tendons to air.
The three types of PSC specimens fabricated in this study are as follows: (1) seven block specimens with a thickness of 600 mm and different void locations and shapes (Fig. 2), (2) a single specimen with different thicknesses of 450, 700, 900, and 1150 mm (Fig. 3), and (3) a   bridge-type specimen identical to an actual PSC girder bridge based on the standard PSC casting process for data collection, with thicknesses of 250, 280, 390, 400, 490, 526, and 1000 mm (Fig.  4). Their material properties are shown in Table 1.

IE
To collect IE signal data from these specimens, we collect the raw IE signal using IE equipment. In the IE equipment, the impactor impacts on the surface of the concrete structure, and the sensor (receiver) measures the impact-generated stress (sound) waves reflected by an object (such as a defect or an opposing surface). This measured wave is the IE signal. The process of this operation is shown in Fig. 5. The raw IE signals measured in this way are transformed through filtering and Fourier transforms into input features used in neural network models.

Methodology
In this section, we explain the neural-network-based approach for the internal void detection of a duct. The whole architecture used for internal void detection is shown in Fig. 6. The proposed model consists of features created by feature extraction and a neural network model that trains them. The proposed model is made up of the following layers. The filtered IE signal is trained by the LSTM layer. The frequency spectrum is trained in the first FFNN layer. The vectors that are element-wise multiplied values on the LSTM layer's output (vector) and the first FFNN layer's (scalar) output are trained in the second FFNN layer. The output layer is used to train the output of the second FFNN layer to classify normal ducts and internal voids. Each component is explained in detail in the rest of this section.

Feature extraction
In machine learning, feature extraction is one of the important processes used to facilitate a model's training and generalization by extracting useful information from data. It also helps humans interpret data. Various feature extraction methods have also been proposed in NDE. The IE signal occurs instantaneously and contains noise due to various environmental factors. Therefore, many studies have devised effective feature extraction methods. For example, at Olson Instruments Inc., which develops NDE instruments, the IE signal is processed with a digital filter and then the frequency spectrum is analyzed with the Fourier transform. This method is a simple but effective feature extraction method, and is widely used in the field to check for the internal voids of ducts. Thus, we used the same method to extract the features, and the result of processing the IE signal with the digital signal is shown in Fig. 7.
There are three types of digital filters: low-pass filters (LPFs), high-pass filters (HPFs), and band-pass filters (BPFs). (29) LPFs pass a signal with a frequency lower than a cutoff frequency and remove a signal with higher frequencies. Unlike LPFs, HPFs pass a signal with a frequency higher than a cutoff frequency and remove a signal with lower frequencies. BPFs only pass signals between the lower and upper cutoff frequencies. In the NDE equipment used in this study, each filter is applied in the following situations.
• If the concrete thickness is less than 76.2 mm, apply a BPF with a frequency range of 10000-50000 Hz. • If the concrete thickness is 76.2 to 152.4 mm, apply an HPF with a cutoff of 6000 Hz. • If the concrete thickness is 152.4 to 304.8 mm, apply an HPF with a cutoff of 3000 Hz. • If the concrete thickness is more than 304.8 mm, apply an HPF with a cutoff of 1000 Hz.
The frequency spectrum obtained by the Fourier transform (30) is used for various purposes such as concrete thickness analysis and defect location analysis. The Fourier transform decomposes the original signal into sine and cosine functions in various frequency bands from low frequency to high frequency and converts the signal in the time domain into the frequency domain. In this study, we use a fast Fourier transform (FFT), which performs a discrete Fourier transform (DFT) with a small amount of computation.

Neural network approach
In this study, we use two features with different characteristics. Accordingly, we use different neural network models (LSTM and FFNN) suitable for each feature.

LSTM
LSTM (14) is an algorithm based on recurrent neural networks (RNNs) and is widely used in various time-series data such as text and signals. LSTM is composed of three gates (forget, input, output) and a cell state (C t ) as shown in Fig. 8(a). C t decides whether to use the information created at this stage by using the calculated results at the three gates. , , 1 ( ) Fig. 7. Example of applying digital filter to raw IE signal.
Here, h t−1 is the information received from the previous time step and x t is the current information. A forget gate ( f t ) decides whether to remember h t−1 . It considers h t−1 and x t , and forgets the information of the previous time step if the result is 0 and remembers it completely if it is 1. The input gate ( t t i g ) decides the amount of x t to be stored in h t−1 . The output gate (h t ) considers C t and o t to decide how much of the information calculated in the current cell is sent to the next time step. LSTM can be used in various structures, as shown in Fig. 9, depending on the purpose.
In this study, LSTM trains the filtered IE signal. The filtered IE signal represents the result of reflection at some location away from the impact. That is, each time step may have meaningful information. Therefore, we train the signal for each time step, as shown in Fig. 9(a), as follows:

FFNN
An FFNN, which is shown in Fig. 8(b), is the artificial neural network with the simplest structure that was first devised among the various neural networks. An FFNN consists of weights ( 1 2 , , ..., n w w w w = ) and bias (b) that train the relationship between inputs and outputs, and constructs a hidden layer with an activation function to perform nonlinear operations. In this study, an FFNN trains the following features: (1) the frequency spectrum and (2) the product of the outputs of two networks. Unlike the raw IE signal, the frequency spectrum consists of values that do not affect each other, so LSTM is not suitable. Therefore, we train the frequency spectrum using an FFNN. For the frequency spectrum ( f ), Leaky ReLU (rectified linear unit) was used as an activation function, as shown in Eq. (9), where we set α = 0.01.
The output of the LSTM is trained for each time step and represented as a 100-dimensional vector. The output of the FFNN trained on the frequency spectrum is represented as a single real number. The 100-dimensional vector of the IE signal output by the LSTM and the single scalar value of the frequency output by the FFNN become a 100-dimensional vector through multiplication. This feature is trained with a higher FFNN using ReLU, as shown in Eq. (10), and this result predicts whether there are internal voids in the duct by using a softmax classifier.

Normalization and regularization
We used batch normalization (31) for the LSTM layer (filtered IE signal) and FFNN layer (frequency spectrum). Batch normalization is the process of adjusting the mean and variance of features included in the batch together during the training of the neural network model. This solves the problem of an internal covariance shift, in which the variance of input values varies for each layer or activation function. That is, for each layer, a batch normalization is performed to ensure that a deformed variance does not appear, as shown in Eqs. (12)-(15). adjusts the parameters (γ, β) used in backpropagation. For regularization, we used Dropout (32) for the feature that is the output of the multiplication operation. Dropout is a way to exclude a proportion of the input features (r) from training, and we set r = 0.1.

Experimental Results and Discussion
In this study, the features used as inputs are the filtered IE signal and the frequency spectrum. In the IE signal, we set and use some ranges, not the entire signal, and the types of features are shown in Table 2. The total number of data collected for the PSC specimen is 22186.
As shown in Table 2, we select a specific range (0-5000 or 0-10000 μs) of the IE signal and use it for training. This is because the entire range of the IE signal may contain information that causes noise, which may cause an error when used in an application. When training these conditions and features, we search for optimal hyperparameters by experiments, and the optimal hyperparameters are shown in Table 3.  To validate the proposed method, we used K-fold cross-validation, which is a method of evaluating a dataset by dividing it into K folds. In the experiment, we set K = 5. The average performance and confusion matrix of the model trained with a specific range (0-5000 or 0-10000 μs) of features selected by us are shown in Tables 4 and 5.
As shown in Tables 4 and 5, compared with the baseline, the range of 0-5000 μs showed an average accuracy of 95.084% (+1.716%) and the range of 0-10000 μs showed an average accuracy of 97.474% (+4.309%). In both cases, improved performance compared with the baseline was found. However, the important point in detecting the internal voids of a duct is to increase the recall degree. For example, if the model predicts voids as normal and humans do not see them, this can lead to a dangerous situation.

Conclusions
Voids that occur inside the ducts of PSC girder bridges are a very dangerous problem because they can cause sudden structural collapse. General NDE to detect voids is complex and requires academic interpretation by experts. In this study, we introduced various neural networks that efficiently detect internal voids of a duct by considering the characteristics of data. When heterogeneous neural networks were used for void detection in this study, the accuracy of void detection ranged from 95.084 to 97.474%.
In the future, we will develop a model for detecting voids inside the ducts of older PSC girder bridges that were constructed with a different casting method from the present method. If the casting method is different, the shape of the IE signal will also be different, which weakens the relationship between learning and inference. In addition, considering that the IE signal shows different shapes according to the concrete thickness, we intend to collect more data for each thickness level. The model trained using this data will be applied to bridges of various thicknesses. Table 4 Average performance evaluation of fivefold cross-validation (0-5000 μs).