Repositioning Technique Based on 3D Model Using a Building Shape Registration Algorithm

The demand for 3D spatial data and effective dataset construction has been increasing. However, the construction of 3D spatial datasets is more time-consuming and costlier than that of 2D spatial data. In addition, maintaining 3D city models up to date long after their initial construction is difficult. In this study, we developed a method of updating 3D building information within a relatively short time frame using highly accurate 2D building information. This method can be used to correct 3D building information automatically. Unmanned aerial vehicle (UAV) imagery of the study area was obtained, and a 3D model was developed using commercial software. Subsequently, the 3D model information was mapped onto the 2D information. After transforming the paired objects into point data at a set interval, registration parameters were calculated by applying the iterative closest-point technique. The calculated parameters were used to reposition the 3D model, enabling the creation of a model that overlaps with more than 98% of the existing spatial information data. Thus, it was confirmed that 3D building models can be produced without ground control points and can be readily updated at low cost.


Introduction
Owing to rapid urbanization worldwide, the rate of development of smart cities, which can overcome various problems associated with urbanization through the incorporation of new information and communication technologies, has been increasing. Such cities have been attracting global attention. Therefore, the demand for 3D city models for the integration of various types of city information and as a basis for visualization has also been increasing. Unlike a conventional 3D map, a 3D city model elucidates the spatial characteristics of objects in the city, thereby facilitating the integration and visualization of various types of city information. A 3D city model represents essential spatial data infrastructure that can be used for urban management such as urban planning, traffic control, disaster management, and change detection. (1,2) A 3D building model, a component of the 3D city model, has conventionally been modeled and textured on the basis of aerial images. However, this process is costly and time-consuming; therefore, the model is typically not updated after the initial construction. To address this limitation, various studies have been conducted to construct 3D models at low cost using unmanned aerial vehicle (UAV) images. To evaluate the quality of UAV images and determine their potential as alternatives to conventional aerial images, the accuracies of the images or 3D models were analyzed using image acquisition parameters such as the altitude, overlap degree of images, and number of ground control points (GCPs). (3)(4)(5)(6) In addition, the results of 3D model construction based on the use of multi-directional images whose resolution exceeds that of aerial images were examined. (7,8) A method for modeling multiple buildings in a large area using UAV imagery with a relatively narrow shooting range was established. (9) Furthermore, following the increasing demand for high-quality 3D building models, a technique to determine factors that lead to the loss of details such as occlusion and distortion has been developed. (7) Despite extensive research, UAV imagery has limited usefulness for the construction of 3D building models. UAVs use small sensors whose accuracy is lower than that of aircraft, resulting in the acquisition of images with low positional and attitudinal accuracy. This necessitates GCP measurements, increasing the time and cost required to build 3D models. To overcome these limitations, studies have been conducted to improve the accuracy of UAV imagery without using GCPs. A micro-electromechanical system (MEMS)-type sensor with an integrated global navigation satellite system (GNSS) and inertial sensors, that is, a real-time kinematic (RTK) GNSS receiver, was installed on a UAV, and images were directly georeferenced using location and altitude data obtained from the installed sensor or receiver. (10,11) In addition, a method that can be used to improve the positional accuracy by post-processing the GPS data was established. (12) However, these methods require the installation of additional expensive or heavy equipment on the UAV, necessitating post-processing and consequently increasing the processing duration.
To overcome the above-mentioned limitations, in this study, we developed a method of correcting positions based on existing high-quality 2D building data. This method requires neither GCP measurements nor the installation of additional sensors. UAV images of the study area were acquired, and a 3D model was created without GCPs. The 3D model was flattened, and the same object was searched by comparing the positions using the 2D building as a reference. A dotted line was drawn along the outline of the paired datasets, and the building shape registration (BSR) coefficient was calculated by applying the iterative closest point (ICP) technique to point cloud data registration. Subsequently, the calculated BSR coefficient was used to reposition the 3D model, and the accuracies of the model and the 2D building were compared.

Definition of reference data for position correction
The building data of the road name address map provided by the Ministry of Interior and Safety, South Korea, were used as the target data, which served as reference data for position correction. Digital maps have been implemented since 2014 to indicate road names, building numbers, and detailed addresses assigned under the Road Name Address Act. The digital map contains data regarding the entire country, which is managed by the government, and is drawn using 25-cm-level digital aerial orthoimages. It consists of 11 types of data including those regarding buildings, roads, and administrative districts. The constructed data are provided as a vector file in .shp format in the open data system (https://www.juso.go.kr) used for downloading digital maps; the system can be accessed through the internet and easily integrated with other data. Figure 1 shows the results obtained upon searching 'building' in the digital map of the Yeouido area of Seoul, on which a transparent UAV image is superimposed to indicate the experimental area.

Acquisition of UAV images
Positional errors may occur while operating UAVs owing to fuselage calibration in the case of multiple flights. We obtained images during a single flight. The target area was Yeouido, Seoul. The UAV image area comprised an apartment complex consisting of five buildings, schools, and school annex buildings. Roads and a playground were located between the apartment complex and the schools. UAV images were obtained on February 28, 2019, to create a model for the Yeouido Elementary School and Middle School buildings as well as the apartments in the area. Images were obtained at an altitude of ~150 m. The area (~288000 m 2 ) was divided into nine strip sections, and 147 images were acquired (Fig. 2). The K-mapper X1 developed by SisTech, South Korea, and a mounted sensor (Sony A6000) were used for image acquisition (Fig. 3). The    specifications of the camera and UAV are listed in Table 1, and the conditions used for acquiring the actual images are listed in Table 2.
Based on the specifications of the UAV, the available flight time was ~40 min. However, the images were obtained in only ~13 min because they were acquired during the winter, which led to the rapid exhaustion of the equipment battery. In addition, in the case of oblique imagery, creating a model with textures on the side surface is favorable. (13) To minimize the number of external variables, images were obtained in the vertical direction only, thereby reducing the acquisition time. To examine the image acquisition results, orthomosaic images were generated from 147 images using the commercial software ContextCapture (Bentley Systems). In Fig. 2, an orthomosaic is used as the background, and the image acquisition positions are expressed as points. In addition, information that can estimate the accuracy of the image and location during image processing using ContextCapture is summarized in Table 2. An average of 1970 tie points were identified in each image, and a total of 46381 tie points were identified; the position uncertainties in the x and y directions were 0.0126 and 0.011 m, respectively.

2D orthophotos based on UAV images
Because the 3D model was created using UAV imagery without GCPs, the positional accuracy of the images corresponds to the accuracy of the 3D model. Therefore, the initial quality of UAV images is an important factor in model accuracy. To examine the accuracy, 2D orthophotos were created from the acquired images without performing position correction and then superimposed on the digital map (reference data) for comparison (Fig. 4). Figure 5 shows enlarged views of one of the schools (Building 1; upper left) and an apartment (Building 7; lower right). Most buildings partially overlapped the reference polygon, thereby necessitating repositioning.

3D model creation using UAV images
Seven buildings in the test area were modeled using software developed by 3DLabs Inc., South Korea (Fig. 6). A commercial program that is commonly used to process UAV images was   used to create a model for the entire photographed area. Meanwhile, the 3DLabs software was used to extract planes parallel to the x-y, y-z, and x-z planes constituting the point cloud; separate models were created for each building object based on the extracted planes. (14) In this study, an efficient technique that can be used to update the 3D building model was identified, and the building models were repositioned using an existing high-quality 2D building dataset. Therefore, a separate model was used for each building.

BSR algorithm
Even when a detailed 3D model is constructed, its applicability decreases when it does not accurately overlap with previously established spatial information including 2D buildings, roads, and land parcels. In other words, it is important to improve the absolute positional accuracy of the 3D model. However, in terms of applicability, it is sufficient to improve the accuracy to the level where model data overlap the existing spatial data precisely. Accordingly, we proposed a BSR algorithm that can be used to reposition 3D buildings modeled without GCP with high accuracy (Fig. 7). The BSR algorithm comprises a data preprocessing process for point-set extraction and the calculation of BSR coefficients based on the ICP algorithm. The horizontal and rotational variations of the calculated BSR coefficient were used to reposition the 3D model. Because the vertical position of the 3D building can be displaced using the elevation of the corresponding position upon the determination of the horizontal position, only the horizontal positions of the 3D buildings were considered in this study.

Data preprocessing
To perform a registration operation based on the ICP algorithm, preprocessing was performed, during which two types of data were mapped onto the same type of point-set data. For the 3D model and 2D building object, that is, the target of position correction and the reference, respectively, point sets were created along the profiles and outlines of the polygons. In general, 3D building data in the spatial information field were modeled using a polygonal method comprising a mesh with a set of points, lines, and surfaces. These data contained normal information, that is, the vertical information regarding each surface that constituted the mesh. Moreover, profile information, that is, flattened model data, were generated using normal information and by extracting all surfaces except for the surfaces of the buildings that were perpendicular to the ground surface. The algorithm used for this preprocessing step is shown in Fig. 8. Point sets were generated by dividing the outline of the created profile into points at 0.5 m intervals. Because the original outlines of the polygon can be used in a 2D building model, additional processing is unnecessary and point sets can be generated by categorizing the original data into points with the same intervals as those used for the profile. The procedure for converting the 3D model data to point sets is shown in Fig. 9.

Calculation of BSR coefficient
The ICP algorithm is the most widely used algorithm for automatic registration of a point cloud acquired using a 3D scanner. (15) In the basic ICP algorithm, point pair matching, outlier rejection, error minimization, and transformation are iteratively performed after point filtering and neighborhood selection until the distance between corresponding points falls within a threshold. The performances of various algorithms have been improved by initializing the image alignment, (16,17) reducing the point weight, and enhancing the computational complexity; (18,19) however, the basic procedure remains unchanged.
This algorithm exhibits superior performance when registering two point clouds with good a priori alignment and rigid transformation. (20) When the a priori alignment is poor, an incorrect registration coefficient is obtained owing to the local minimum error. (21) The position error between the 3D model and 2D building reached 30 m, and the rotation error was within 20°, indicating a priori alignment suitable for the application of the ICP algorithm. In addition, although the absolute positional error of the initial 3D model was large, the distance between the vertices constituting the model was relatively accurate. Therefore, whereas the 3D model and 2D building data were acquired using different methods and equipment, they were based on the same scale. Owing to the use of the same scale, a rigid-body transformation that only considers rotation and displacement is applicable. Therefore, we used the ICP algorithm to calculate the BSR coefficient for the repositioning of the 3D building model relative to the 2D reference object. Table 3 presents the main elements required for ICP implementation for matching two unorganized point sets. We used all sampling points and point-to-point matching because we used point sets (not faces), and it is important to match their shape. As one of the aims of this study is to correct the position of the 3D model, we used the distance weight. Furthermore, because both datasets comprise points extracted from polygons, outliers were not considered. The algorithm for calculating the BSR coefficient, which is used to correct the position of the 3D model (source data) based on the elements defined in this manner, is shown in Fig. 10. Table 4 details the specific parameters of the ICP-based algorithm.

Data pairing
The superimposition of the 3D model created without GCPs onto the national standard 2D data is shown in Fig. 11. Before the calculation of the BSR coefficient using the ICP algorithm, a pairing process was essential. Each red line in Fig. 11 extends the center point between each paired object by 1:1. When each object is rectangular, the center point is the intersection of the inner diagonal lines. When the object is not rectangular, the center point is selected by   considering the shape to be rectangular by using the outermost lines in four directions or virtual lines connecting the outermost points, as shown in Fig. 12 for Building 7. Figures 4, 5, and 11 show that when comparing 3D building objects with reference 2D polygon data, most building objects are oriented toward Buildings 2 and 3 in the image. Repositioning was performed by applying the BSR algorithm to each of the seven paired building models.

Calculation and application of BSR coefficient
The paired 3D model and 2D building information were set as the source and target data, respectively, and the BSR coefficient of each building was calculated using the algorithm discussed in Sect. 3.3 (Table 5). To simplify the comparison of the BSR coefficients of the seven buildings, the coefficients presented as a matrix were mapped onto horizontal and rotational variations ( Table 6). All seven buildings exhibited marked overlapping, with the rotational variation between the registration source data and target data being within 1° clockwise. However, considerable horizontal variation was observed with maximum, minimum, and average differences of 6.98, 2.53, and 4.76 m, respectively, between the target and reference data. In particular, when the direction was determined on the basis of the elements of the BSR coefficients in Table 5, the values ranged from −7.80 to 2.70 m in the x direction and −1.21 to  −6.26 m in the y direction, indicating large variations between the buildings. Because 3D models created using images acquired from the same mission on the same equipment yield different positional errors, each optimized matching coefficient must be calculated. Repositioning was performed by applying the seven calculated BSR coefficients to the corresponding 3D buildings. Figure 13 shows the superimposition of the 3D building model onto the 2D building object, that is, the reference data, before and after repositioning.

Evaluation of the result of position correction
To evaluate the result of the position correction of the 3D model, the improvement in relative positional accuracy was examined with reference to the highly accurate 2D reference polygons.
In general, because the profile of a 3D model and the geometry of the 2D polygon differ, it is difficult to specify an identical point in the two and make comparisons. In addition, because the difference in the distance between the two points reflects only the difference in a plane, it is difficult to comprehensively determine the magnitude of the correction in terms of the rotation. Therefore, the result of position correction was evaluated with reference to the 2D polygons, and not based on the comparison between the two equivalent points, using the registration improvement rate of the 3D model profile. The registration improvement rate was calculated using the difference between the overlap ratios of the 3D model and 2D polygons before and after repositioning: where P b is the profile polygon of the 3D building before repositioning, P a is the profile polygon of the 3D building after repositioning, T is the 2D polygon used as the repositioning target, and R imp is the rate of 3D building registration improvement. The registration improvements for the seven building models are shown in Table 7. The maximum, minimum, and average overlap ratios of the 3D model superimposed on the 2D reference polygon before repositioning were 93.3, 49.7, and 72.7%, respectively. However, after repositioning, the maximum, minimum, and average overlap ratios were 99.7, 97.9, and 98.86%, respectively, with an average improvement of 26.14%. These results indicated that the average overlap rate was improved by 26.14% through position correction. This also confirmed that the 3D model was effectively positioned through the building shape matching algorithm proposed in this study.

Conclusions
The procedure of geometry-based position correction proposed in this study is as follows. First, the outlines of the profiles created on the basis of the 3D model (target for position correction) and 2D object (reference data) should be mapped onto point sets with equal intervals. Subsequently, the registration BSR coefficient for each building for which the distance between two point sets is minimized should be calculated using the ICP algorithm and then applied to the respective 3D model to complete the repositioning.
To test the algorithms proposed, images were acquired through the sole use of sensor information from UAVs without GCPs, and seven 3D buildings were modeled. Then, the BSR coefficients for each building were calculated using ICP algorithms, and the position was corrected by applying the coefficient to each 3D building. The effects of repositioning were identified by comparing the overlap ratio of the 2D reference object with the x-y plane of the 3D model before and after repositioning. The results show that the overlap ratio of the seven tested models improved by ~26.2%, from ~72.7% before repositioning to ~98.9% after repositioning. In particular, the minimum pre-repositioning overlap ratio was 49.7% but it was 97.9% or higher for all buildings after repositioning.
The findings from this study are as follows. 1) The proposed positional correction algorithm allows the quick and low-cost fabrication of 3D models without GCPs, as well as repositioning.
2) Using periodically acquired UAV images, updates to the 3D model can be automated, thereby ensuring up-to-date urban models. 3) Even buildings modeled on the same imaging strip image have different positional errors and, therefore, individual BSR coefficients for each building model are required. Table 7 Overlap ratio of the 3D model onto the 2D target data before and after repositioning.

Building No.
Overlap ratio before repositioning (%) Overlap ratio after repositioning (%) Improvement rate (%) In this study, 2D data were selected as reference data for repositioning because they are updated on a monthly basis. However, in some cases, 2D reference data corresponding to the 3D model may be missing or the geometry may vary significantly. In such cases, the positions of all buildings can be corrected by applying the registration coefficient of a building adjacent to the applicable building or using the average registration coefficient of the applicable area.