Statistical Analysis for Usability Evaluation of Unmanned Aerial Vehicle in Geomatics

Unmanned aerial vehicles (UAVs) have advantages such as a relatively low operating cost and excellent ground–sample distance (GSD) compared with conventional manned aerial photogrammetry. Recently, the development of sensor technology has enabled digital mapping, terrain model generation, economic evaluation, monitoring, cadastral surveying, coastal surveying, and ground surveying. However, existing studies have shown the limited validity of UAVs for spatial information construction owing to limited experiments and analyses. In this study, the validity of UAVs for spatial information construction was evaluated, and recent case studies related to the accuracy of the results were analyzed. The results of 34 studies including quantitative results of UAV accuracy within the last 5 years were investigated, and the correlations of flight altitude, accuracy, and ground control point (GCP) number are presented through the analyses of existing studies. The horizontal and vertical accuracies of terrain information using a UAV were 0.51 m and 0.56 cm, respectively. The horizontal and vertical accuracies were correlated with the flight altitude and GCP number, with the flight altitude having a greater effect than the GCP number. In the future, the results of this study will be used as a basis to examine the validity of geospatial information construction using a UAV.


Introduction
Recently, many studies related to unmanned aerial vehicles (UAVs) have been conducted owing to the development of information communication technology and sensor technology. (1,2) In addition, as map information and location services provided through various media such as smartphones, Internet portals, and navigation software are popularized, the application of unmanned airplanes is increasing as an efficient method of constructing geospatial information. (3,4) UAVs can shoot at a low altitude as compared with conventional manned aerial photogrammetry, so they can be applied even in slightly cloudy weather, have a relatively low operating cost, and have an excellent ground-sample distance (GSD). (5,6) Recent studies related to UAVs have been applied to a variety of fields, such as digital mapping, terrain model generation, economic evaluation, monitoring, and current surveying. (2,7) However, existing studies are mainly limited to experiments and analyses of specific applications. Therefore, the statistical validity of presenting the characteristics of geospatial information using a UAV is not high, so an integrated analysis of the results of previous research is needed. In this study, recent papers related to UAVs were examined, and a meta-analysis was conducted on the results of these studies. Figure 1 shows the flow of this study.

Selection of Articles for Analysis
The amount of UAV-related research studies has been increasing recently owing to the development and distribution of commercial UAVs. Starting from the development of systems in the early 2000s, research has been conducted on calculating the output and the accuracy of verification for application to various fields. (8,9) In this study, a literature survey was conducted for research within the last 5 years using the National Research Foundation of Korea website to select the studies to be analyzed. A total of 11297 articles using "UAV" as a keyword were found through a website survey, and 34 of the articles from the last 5 years related to "Accuracy" were selected. Table 1 shows the selected articles.
The recently published UAV-accuracy-related research studies were carried out on a variety of topics, such as cartography, terrain model generation, and spatial analysis. In most of these studies, the accuracy of the UAV results was verified to determine the validity and usability. In this study, a comprehensive analysis was carried out on the studies that included quantitative results on the degree of purification in the existing studies.  Building of 3D Terrain Modeling and Evaluation of Location Accuracy Using UAV in Beach Area (26) 2016 Journal of the Korean Cadastre Information Association 24 Damage Analysis and Accuracy Assessment for River-side Facilities using UAV Images (27) 2016 Journal of the Korean Society for Geospatial Information Science 25 Accuracy Analysis of Coastal Area Modeling through UAV Photogrammetry (28)  Availability Evaluation for Generation of Geospatial Information Using Fixed Wing UAV (35) 2014 Journal of the Korean Society for Geospatial Information

Meta-analysis
Meta-analysis is a statistical method of synthesizing a pooled estimate by combining estimates from two or more individual studies. In other words, it is a statistical technique used to quantitatively estimate an integrated summary of estimates of the results presented in the studies and to evaluate effectiveness and efficiency. (36) In this study, the accuracy of UAVs is classified into horizontal and vertical accuracies, and their relation with flight altitude is analyzed.
The effect size is a quantitative index used to summarize the results of the study in a metaanalysis. (37) The effect size can be used to give a specific figure for relevance. A method of estimating the effect size using a correlation coefficient was used. The formula used is as follows. ( z: Z-transformed sample correlation coefficient, r: sample correlation coefficient The Z-t ransfor mation of the number of model relations cor responding to the Z-transformation of the sample correlation coefficient is as follows.
ρ: correlation coefficient The Z-transformed values of the correlation coefficients generally follow the normal distribution, and the uncertainty of the sampling of the Z-transformed correlation coefficients can also be expressed using the following dispersion formula.
n: number of samples in the study In this study, flight altitude, horizontal accuracy, vertical accuracy, minimum value, maximum value, and variance were used to analyze the correlation between flight altitude and accuracy. Meta-analysis was performed using the R statistical website. Table 2 shows the data used for effect size analysis and Figs. 2 and 3 show forest plots for horizontal and vertical accuracies, respectively.
As shown in Figs. 2 and 3, the effect sizes of horizontal and vertical accuracies and flight altitude are 1.2 and 1.07, respectively. Therefore, in this study, statistical analysis was performed to show the general horizontal and vertical accuracies of geospatial information obtained by a UAV. Figures 4 and 5 show the fixed effect models for horizontal and vertical accuracies, respectively.

Statistical analysis
In the previous studies, only the accuracy evaluation was performed under the experimental conditions of the individual studies, and the analysis of the relation between flight altitude, GSD, and accuracy was insufficient. In this study, the statistical analysis of flight altitude, GSD, and accuracy was performed for the integrated analysis of geospatial information constructed by unmanned aerial photogrammetry. Table 3 summarizes the results of previous studies analyzing the horizontal and vertical accuracies of geospatial information constructed by a UAV. The main summary items are horizontal and vertical accuracies, flight height, and ground control point (GCP) number.
The method of statistical analysis used is regression analysis, and the dependent variables are horizontal and vertical accuracies. The independent variables used are the flight altitude and GCP number. Table 4 shows the statistics for accuracy and Table 5 shows the correlation coefficients.
The horizontal accuracy of the geospatial information constructed using a UAV was 0.51 m on average and the vertical accuracy was 0.56 cm on average. Although the accuracy of a general UAV cannot be presented because of the different experimental conditions in the studies, the results of the research showed that the accuracy of geospatial information constructed by a UAV can be predicted. Also, as shown in Table 5, the horizontal and vertical accuracies are inversely proportional to the flight altitude and GCP number. It can be seen that the GCP number has a greater impact on accuracy than the flight altitude. Figures 6 and 7 show histograms of the dependent variables. Figure 8 shows the P-P plot for the horizontal accuracy and Fig. 9 shows the P-P plot for vertical accuracy.     The accuracy of each dependent variable is affected by the number of flights and the GCP number. As shown in Figs. 4-7, the accuracy in the vertical direction appears to be less influenced by these effects. The results of this study show the accuracy of general UAV geospatial information through statistical analysis and will be used as a basis for examining the validity of geospatial information construction using a UAV in the future.

Conclusions
In this study, the validity of using a UAV for geospatial information construction was evaluated, and recent case studies related to the accuracy of the results were analyzed. The results of this study are as follows. 1. The results of 34 studies including the quantitative results of UAV accuracy within the last 5 years were investigated and the correlation of the flight altitude, accuracy, and GCP number was presented through analyses of these studies. 2. The horizontal and vertical accuracies of the terrain information obtained using UAVs were 0.51 m and 0.56 cm, respectively. The horizontal and vertical accuracies were correlated with the flight altitude and GCP number. In particular, flight altitude had a greater effect than the GCP number. 3. In all case studies, the vertical accuracy of the UAV was lower than the horizontal accuracy. 4. In the future, the results of this study will be used as a basis for examining the validity of geospatial information construction using a UAV.