Automated Reconstruction of Railroad Rail Using Helicopter-borne Light Detection and Ranging in a Train Station

A coordinate-based 3D model of a railroad rail is essential for the maintenance of railway services. However, automated processing to reconstruct objects from light detection and ranging (LiDAR) data in areas where such facilities are installed in a complex manner is still a challenge. In this study, our objective is to develop a method for the automated reconstruction of rails from LiDAR data in a complex area. Unlike the running sections of a train where one or two rails are present, many rails are installed by joining or branching in a railway station. Three factors, namely, height difference, spatial relationship with other objects, and point density difference, were considered in detecting the rails in this complicated area. For tracking and modeling rails, an iterative random sample consensus (RANSAC) and singular value decomposition (SVD) algorithms were used. The results showed that about 90% of the rail tracks were extracted, and the precision rate was about 99%. All processes were fully automated, and the method proposed in this study was developed to be applicable to various line-type facilities as well as rails.


Introduction
Trains are now recognized as one of the most economical and universal transportation systems because of their advantage of being capable of transporting many people and cargo at a low cost and with an accurate arrival time. However, unlike other transportation systems, trains can only be operated on rails, which restrict free driving. Because a derailed train can cause large-scale casualties and economic losses, a continuous and fast inspection of rails is essential.
Three-dimensional models of rails provide very important information to the manager or operator of a railway facility that provides this service. However, the extraction of slender rails is technically and economically very difficult and inefficient with stereo plotting using aerial photographs or traditional ground survey. Therefore, in recent years, there have been research studies on reconstructing such long and narrow objects by a laser scanning technique. However, most studies have been conducted on the automatic extraction and modeling of one or two rails in the driving section between stations, and there were very few studies on railway stations where many railway facilities were installed in a complicated manner. In this study, therefore, we focused on the automatic extraction of long and narrow rails in a complex area. As shown in Fig. 1, a railroad wire is composed of two wires called contact and mounting wires, and rails are installed just below the wires.
In many studies, extracting linear facilities such as power lines from laser data has been a major topic for several years. Neubert et al. proposed a method for the automatic extraction of railway infrastructures using an adapted random sample consensus (RANSAC) algorithm and a knowledge-based method using a 2D cut from combined very dense light detection and ranging (LiDAR) data and high-resolution digital aerial images. (1) Beger et al. presented an approach to reconstructing rail track center lines through the fusion of high-resolution aerial imagery and LiDAR data by employing object-based image analysis, the height information of laser points, and the adapted RANSAC algorithm. (2) Lou et al. presented an automated method for detecting rails using the physical shape, geometrical properties, and reflection intensity of rails. (3) In their study, they investigated the feasibility of a fast reconstruction of railroads from mobile laser scanning (MLS) data.
Gabara and Sawicki presented a method for measuring image-based point clouds for the determination of the geometric parameters of a railway track. (4) Their study was performed using digital images and a reference control network. Jwa and Sohn suggested a solution for the automatic reconstruction of a 3D power line model from airborne laser scanning (ALS) data. (5) The method starts by extracting power line candidate points, which are converted into a catenary curve model, and the model is extracted by producing hypothetical growing models and selecting an optimal one. Zhu and Hyyppa proposed approaches to reconstruct railway objects from ALS and MLS data using an image processing technology, a 3D randomized hough transform (HT), and a quadtree approach. (6) Guo et al. presented an algorithm for reconstructing power lines from ALS data using local shape knowledge, distribution properties, and a RANSAC rule-based algorithm. (7) Elberink et al. presented a method for reconstructing rails using MLS data. The detection is carried out using shape knowledge and the spatial relationship with other objects, and the modeling is performed by first fitting a parametric model of a rail piece. (8) Liang suggested an approach to extract roadsides from LiDAR data using the height jump property between a road surface and a sidewalk. (9) Kumar et al. presented an algorithm based on a parametric active contour or a snake model to extract road edges from terrestrial mobile LiDAR data. (10) Nassu and Ukai presented an approach to extract rails by matching edge features to candidate rail patterns modeled as sequences of parabola segments. (11) Ussyshkin and Smith proposed a simplified approach, that is, evaluation of the vertical and horizontal accuracies separately for a corridor mapping application. (12) Most of the research studies handled only a few wires or rails, and researchers also said that it is difficult to reconstruct whole line objects automatically in a complex area where many line-type facilities are installed. Therefore, the goal of this study is to develop a method for the automatic extraction of railroad rails from high-density helicopter-borne LiDAR data in a complex area where many line-type facilities are installed in a complex manner.

Detection of candidate points of rails
Before detecting the laser points of rails, the models of contact wires are reconstructed in advance since the position of the contact wire becomes a reference of the spatial relationship with the rail tracks. The automatic extraction of contact wires from LiDAR data is relatively easy because the objects are clearly separated from other objects such as the ground and vegetation. In this research, the method of Jeon and Choi is used to extract contact wires from LiDAR data. (13) The method takes advantage of shape knowledge such as the installation height, the length of contact wires, and the multiecho property of LiDAR data to detect the laser points of the contact wires.
As shown in Fig. 2, the rails are installed about 15 cm higher than the surrounding ground surface and located right below the contact wires. In addition, the density of rail laser points is different from those of other objects such as vegetation because of their long narrow features. Therefore, rail points can be detected more precisely using the height difference between the rail tracks and the ground surface, the spatial relationship with the contact wires, and the point density difference with other objects.
The processes performed to detect rails are (i) classifying the 1st candidate points using the height difference between the rails and the ground surface, (ii) selecting the 2nd candidate points using their spatial relationship with the contact wires, (iii) selecting the 3rd candidate points considering the point density difference between the rails and other objects, and (iv) finally, tracking the rail points. Figure 3 shows the flowchart of the proposed method.
In contrast to railroad wires, it is not easy to classify the objects of rails from laser point clouds since the rails are interfered by other objects such as the undulation of the ground surface and vegetation. To classify the rail points using the height difference, first, the whole target area is divided into cells of regular size. The lowest point is then found from a cell and by selecting the points included in the height range corresponding to the rails. These points become the 1st candidate points.
The 2nd candidate points are selected using the spatial relationship with the contact wires. Since the rails are located right below the contact wires, as shown in Fig. 4, when the points, which are located within a corresponding distance from the contact wires, are selected among the 1st candidate points, many nonrail points can be excluded. The 3rd candidate points are classified using the point density from the 2nd candidate points. Since the rails are very narrow, their point density is much lower than that of vegetation. However, the point density of the rails is sometimes higher than that of vegetation in the cross section of the rails, as shown in Fig. 5. The 3rd candidate points are classified by considering the maximum point density of the rails after analyzing the laser point clouds for rails in the target area.

Tracking rail points
Almost all the rails in a train station are straight, but there are also a few curved sections. Therefore, it is necessary to process rail points by separating the straight and curved sections. However, in contrast to travel routes, a curved rail track is very short, and the shapes of curves are so varied in the limited space of a train station. Moreover, a gross irregularity of laser point clouds in a cross section shown in Fig. 5 makes it almost impossible to distinguish between curved and straight lines. Furthermore, a straight line is very often mistaken as a curved line. All rails are, therefore, assumed to be straight lines. The final rail laser points are classified by tracking.
In the tracking step, the distances between a line segment and all points have to be calculated to determine whether the point is for a rail or not. However, it is computationally very expensive owing to the capacity of point clouds. To increase tracking performance, the neighboring points of each point are defined using a k-nearest neighbor (KNN) approach. The calculation of the point density is based on the volume density. The number of neighboring points in the KNN approach is limited to a maximum of 30 within a radius of 10 m in consideration of computer performance, processing time, and point density of rails.
Before tracking the rail points, a search window is designed for detecting a seed line segment, which becomes a starting point for rail tracks. To be recognized as a seed line segment, the points have to satisfy three conditions. Firstly, the height difference between the lowest point and the highest point in the window should be small. Secondly, there should be an appropriate number of points within the window. The minimum number of points in the window takes into account the rails, and the maximum number of points takes into account the cross section of the rails, vegetation, and ground undulation. If these two conditions are met, a RANSAC algorithm for a straight line is applied. Finally, after applying the RANSAC algorithm, the number of inlier points has to be appropriate for a line. When these three conditions are met, the line segment becomes a seed line segment for tracking.
By using the points of a seed line segment, we start tracking the rail. The processes of tracking are as follows: 1. Calculate the parameters of the 3D line segment by singular value decomposition (SVD). If the three points P 1 = (x 1 , y 1 , z 1 ), P 2 = (x 2 , y 2 , z 2 ), and P = (x, y, z) are lying on a straight line in 3D space, then where t = |PP 1 | / |P 1 P 2 |. Equation (1) can be modified as Eq. (2).
Here, a, b, and c represent the direction cosines of the 3D straight line.
To determine the direction vectors of the 3D straight line, x 1 , y 1 , and z 1 are substituted to the mean values ( , , x y z ) of the whole points, and a matrix is constructed as Fig. 5. Comparison of point density between rails and vegetation.
By using the matrix A, we apply the SVD approach. The SVD is one of the methods for solving linear least-squares problems. The M × N matrix A, whose number of rows, M, is greater than or equal to its number of columns N, can be written as the product of an M × N column-orthogonal matrix U, an N × N diagonal matrix S, which has nonzero values on the diagonal, and the transpose of an N × N orthogonal matrix V. The matrices U and V are unitary matrices, and the relationship among the matrices is A = U × S × V T . (14) U is called a left singular vector of the matrix A and V is called a right singular vector of A. (15) The direction vector can be retrieved by finding the largest singular value in S and extracting the elements of the corresponding column in V.
The tracking of rail points is conducted in both directions. After finishing the tracking process for a line, the detected points are removed, and the search window does not move and we try again to detect other lines in the window. This tracking method can be applied to not only straight lines but also small curve parts of rails. The tracking method is presented in Fig. 6.

Modeling of rails
After finishing the detection of the rail laser points, the detected points are used for rail modeling. As mentioned in Sect. 2.1, because of the irregularity of laser point clouds in the cross section of a railway and the so many different shapes and small length of the curved rails, all the rails are assumed to be straight lines. The modeling of a straight line is relatively easier than those of other features because of its simple geometry. Since the tracked points for a rail could consist of one or more straight lines, an iterative RANSAC (p = 99%) algorithm is employed.
There are still a few blunder points even if the outliers are removed after running the RANSAC. Therefore, the least-squares method is performed twice. One is to remove the blunder points and the other is to estimate the model parameters. To connect these line models, an intersection point between two adjacent models has to be defined. However, it is nearly impossible that two straight lines in a 3D space will meet at a certain point even though the data has a very high accuracy. The intersection point is therefore calculated in the x-y plane and adjusted in the z-axis direction.
Sometimes, two adjacent lines, whose direction vectors are very similar, generate an unexpected intersection point. In extreme cases, the intersection points are generated outside of the study area, as shown in Fig. 7. In such cases, the left endpoint of the right model is moved to the right endpoint of the left model. Figure 8 shows the study area. The area was chosen considering the high geometric complexity of the railway. The area, which is around the train station of Enschede located in the eastern part of the Netherlands, is about 650 m long and 50 m wide. There are a maximum of 13 rails across the railway direction.  The original point clouds were acquired with the FLI-MAP 400 system of the Fugro-Inpark company. The system is a helicopter-borne LiDAR system, which consists of a laser scanner, a digital camera, and two video cameras.

Study area
The original data block is divided into 61 tiles. Each tile is 25 m long and 25 m wide. The total number of points is about 1.6 × 10 6 , and the density of points is approximately 20 pt/m 2 . The systematic errors of the data are approximately 4-8 cm in the horizontal direction and 2-3 cm in the vertical direction. The data is shown in Fig. 9. Figure 10 shows the results of the detected rail points. The rail points are classified using the height difference from the ground (1st candidate points), the spatial relationship with the contact wires (2nd candidate points), and the point density difference (3rd candidate points), and finally detected by linear tracking. Even if the order is different, the results are almost similar. They are shown separately to show the results for each step.

Results of rail reconstruction
Since the rail is about 10-15 cm higher than the ground surface and the accuracy of the laser point is about 3 cm in the vertical direction, the 1st candidate points can be classified using those properties. The thickness of the laser point clouds on the ground is more than 10 cm because of ground undulation, gravel, and the accuracy of the laser point itself, and the point clouds on other objects such as vegetation, which are higher than the rails, exist. Therefore, the laser points, which are higher than the ground and lower than the vegetation height, have to be classified. Figure 10(a) shows the result of the 1st candidate points. The shape of the rail is indistinct, but there are still many other objects such as vegetation and ground undulation.
The 2nd candidate points are classified using the spatial relationship with the contact wires from the 1st candidate points. The contact wire is expected to be situated in the middle of a pair of rails; however, some wires are not located in the middle. Therefore, the planar distance of 1.2 m between a contact wire and a rail is applied. Figure 10(b) shows the processing result of the 2nd candidate points. The result shows that many nonrail objects are removed.
The 3rd candidate points are detected using the point density property from the 2nd candidate points. The point density of a rail is usually lower than those of other objects such as vegetation and artificial structures because of its long narrow feature. As a result of the analysis, the point density of a rail is verified to be about 2-8 pt/m 2 . Therefore, the points, whose density is more than the maximum point density on rails, are removed. Figure 10(c) shows the processing result of the 3rd candidate points. Compared with the 2nd candidate points, many nonrail points are removed, but a few rail points are also removed. However, they do not affect the modeling considerably because the removed rail points are very few. The final rail laser points are classified by linear tracking from the 3rd candidate points, as shown in Fig.  10(d). Figure 11 shows a result of the comparison of the detected point cloud and the orthoimage by overlapping to check the completeness of detection.
The rail models reflect the problems of the detected point cloud, but it is generally well modeled except for the parts where the rail intersects and the vegetation is mistaken as rails. Most of these errors are due to problems that occurred during the detection process rather than the modeling itself. In some cases, some incorrectly detected points are removed during the modeling process. Figure 12 and Table 1 show the modeling results for rails. On average, the standard deviation of the horizontal coordinate is about 6 cm, and the standard deviation of the vertical coordinate is about 2 cm.

Analysis of rail reconstruction
To evaluate the quality of detection results, all rail points were manually extracted and compared with the detected points. We then calculated the recall and precision rates using Eqs. (4) and (5), respectively. The recall rate indicates the proportion of relevant objects that are correctly detected, and the precision rate is the proportion of detected objects that are relevant.
The recall and precision rates are 89.8% (8473.9 m out of 9432.2 m) and 98.9% (8473.9 m out of 8570.1 m), respectively. Although the result shows a high extraction rate on the rails, there are also some defects in this method. The rail points are not well detected and modeled in the cross sections as shown in Fig. 13 because of the irregularity of the point clouds, and the vegetation points lying parallel to the rail tracks are sometimes incorrectly classified as rail points. The large gaps in the rail models are not errors in the extraction process, but errors from the gaps of the original data itself due to the occlusion.   In this study, all of the railway rails were assumed to be straight lines. However, there are curves in the connection and cross sections of the railways. The modeling of gentle curves in the running section is expected to be possible; however, in the railway station, the radius and length of the curve section are very small, and the shapes of the curves are also highly varied. Furthermore, owing to the irregular distribution characteristics of the point cloud, as shown in Fig. 5, it is almost impossible to automatically distinguish between the curves and the straight lines using algorithms. Therefore, although the whole process of this study was automated by programming, it seems that some manual work will be needed after the modeling is completed.

Conclusions
The purpose of this study is to develop an algorithm to automatically extract rails from a railway station where many facilities are installed in a complex manner using high-density laser scanning data. The following conclusions can be drawn on the basis of the results obtained.
By using the high-density laser scanning data and RANSAC algorithm, we can automatically extract the rail accurately even in an area where many objects are installed in a complex manner. When extracting a linear object from laser scanning data, the shape knowledge of the object can be used to improve the accuracy and completeness of object extraction by efficiently removing nonlinear objects such as the ground surface.
It is more difficult to extract rails that are highly interfered with by other objects, such as vegetation, than railway wires that are completely separated from the ground surface. However, extraction is possible by using the height difference from the ground surface and the spatial relationship between other facilities and the point density. When the extracted rails are compared with the original data, the result shows that 89.8% of the rail tracks are extracted and the precision rate is about 98.9%, and the standard deviation of each model is less than 6 cm.
It is expected that the developed algorithm in this study will be useful for the management of railway facilities such as accident prevention by periodically monitoring the linearity abnormality. The extraction algorithm proposed can be applied to not only rails but also various types of linear objects, and all processes can be programmed to be performed automatically with one initial click.