Improved Slanted Edge Methods of Measuring Modulation Transfer Function Based on Structured Total Least L1-, L2-norm Edge Fitting for Urban Remote Sensing Images

In this paper, we present improved slanted edge methods of measuring the modulation transfer function (MTF) based on structured total least L1-, L2-norm edge fitting for urban remote sensing images. The structured total least L1-, L2-norm methods are used to establish slanted edge fitting models, which take the errors in both the design matrix and observation vector in the fitting model into consideration. The slanted edge fitting parameters are estimated under the two norm criteria of L1 and L2. The proposed methods are applied to both simulated and actual images. The results showed that the edge fitting parameters and MTF values calculated by the proposed methods are closer to the true values than those obtained by the traditional slanted edge method based on classical least-squares fitting. It is also found that when the data contain a large amount of noise, the structured total least L1-norm edge fitting has the greatest robustness.


Introduction
The modulation transfer function (MTF) depicts the modulation degrees at different spatial frequencies. (1) The MTF can be calculated and analyzed as a way of evaluating the quality of remote sensing images, which can reflect the state of the sensor. (2)(3)(4) Accurate measurement of the MTF is a basic and important task in image quality evaluation and restoration. (5)(6)(7) The slanted edge approach, (7,8) which is an improvement of the edge approach, is widely used for measuring the MTF. It is convenient to use, and only areas in an image with obvious edges need to be selected. It is also not affected by the slanted angle of the edge. Qin and Gong proposed a slanted edge method to calculate the point spread function (PSF) of remote sensing images. (8) Xie et al. analyzed in detail the factors that may affect the MTF results in the slanted edge method, (9) such as the edge angle and measurement error. The slanted edge approach generally includes four steps: (1) Edge extraction: The least squares (LS) method is usually adopted to calculate the slanted edge using the edge points. Steps (2)-(4) are based on the results of step (1). Therefore, it is of high importance to obtain an accurate slanted edge in step (1). The specific steps and the influence of the accuracy of slanted edge fitting will be elaborated on in Sect. 2. In recent years, many scholars have further improved and developed the slanted edge method. Fan et al. (10) and Qu et al. (5) proposed a gradient operator to improve the traditional edge method, which can obtain more accurate edges and increase the accuracy and stability of the PSF. Their methods can reduce the influence of noise on edge fitting. Deng (6) and Birchfield (11) proposed measuring the MTF with the total least squares (TLS) method with the aims of considering the errors in both the observation vector and design matrix in the edge fitting process and improving the accuracy of edge fitting compared with the LS method. Qu and Zhang introduced an edge fitting method based on the robust LS method, which improves the accuracy of slanted edge fitting in the presence of noise. (12) Xie et al. analyzed the illposedness in ESF regression caused by noise and proposed a regularized slanted edge method to solve the problem by incorporating a Tikhonov regularization term. (13) In the edge fitting model, however, some elements of the design matrix also contain errors. Also, when the data contain a large amount of noise, greater robustness of the edge fitting method is required. Therefore, in this paper, we study the edge fitting method with the aim of improving the accuracy and robustness of edge fitting in the measurement of the MTF.
We propose improved slanted edge methods for measuring the MTF based on structured total least L1-, L2-norm fitting for urban remote sensing images. The LS method is usually adopted to calculate slanted edges. However, some of the elements in the design matrix in the edge fitting model could contain errors. The TLS method and its extensions have attracted much research attention recently. (14)(15)(16)(17)(18)(19) The TLS method takes into account the errors in both the observation vector and design matrix caused by noise in the image, instead of simply considering the errors of the observation vector.
The rest of this paper is organized as follows. In Sect. 2, the specific steps of the slanted edge approach and the influence of the accuracy of slanted line fitting are elaborated on. The proposed edge fitting methods based on structured total least L1-, L2-norm fitting are also presented. In Sect. 3, experiments conducted on both simulated and actual remote sensing images are introduced and the results are analyzed and discussed in detail. Finally, the conclusions are presented in Sect. 4.

Improved Slanted Edge Method Based on Structured Total Least L1-, L2-norm Fitting
The commonly used slanted edge approach, (7,8,13) as shown in Fig. 1, generally includes four steps. (1) In edge extraction, it is necessary to select a region of interest (ROI) with a slanted edge in the image, which should contain different uniform features on the two sides of the edge. Details of the two substeps of step (1) are as follows. Substep 1) Determination of the subpixel edge points: A cubic polynomial is usually used to fit a few pixels around the pixel with the largest gray-scale changes in each row of the image. The location of the subpixel is derived from that where the second derivative of the cubic polynomial is zero. Substep 2) Edge fitting: The LS method is usually adopted to calculate the fitted image edge. (2) In the calculation of the ESF, the ESF is obtained by projecting the pixels in the image along the direction of the slanted edge in the direction perpendicular to the slanted edge. (3) In the calculation of the LSF, the Fermi function is widely used to fit the initial ESF points and obtain the ESF curve. The LSF curve is obtained by differentiating the ESF. (4) Finally, the MTF of the image is obtained by performing a one-dimensional fast Fourier transform (FFT) and normalization to the LSF, and the MTF value at the Nyquist frequency is evaluated.
The accuracy of the fitted edge has a relatively significant influence on the establishment of the ESF. Figure 2(a) shows the details of establishing the ESF. In Figs. 2(a) and 2(b), the dots represent the centers of pixels. The two solid slanted lines represent the edge and a line perpendicular to the edge. The dashed lines represent the projections of the points in the direction perpendicular to the edge. Figure 2(b) shows the influence of an inaccurately fitted edge on establishing the ESF. The thick lines represent the ESF established with one inaccurately fitted edge. The distance from the point to the edge is changed, resulting in an inaccurate ESF. The classical LS method is usually used to fit edge points, which generally only considers the errors in the observation vector. However, in the linear edge fitting model, some of the elements in the design matrix also contain errors. Also, the fitting process is affected by random or gross noise, which may lead to an inaccurately fitted edge. In this paper, we propose improved MTF measurement methods based on structured total least L1-and L2norm edge fitting, which take into consideration the errors in both the observation vector and design matrix.

Structured total least L2-norm (STL2N) edge fitting model
To consider the errors in both the design matrix and observation vector in the edge fitting model, the fitting model of the edge based on the errors-in-variable (EIV) model is constructed as follows: where y is the n × 1 observation vector, which represents the rows of the pixels of the slanted edge in the ROI, , A is the n × 2 design matrix, 1 [ ] represents the columns of the pixels on the edge, 1 [ ] , v x is the residual vector of x, Assuming that ∆v x is a small increment in v x and ∆E A is a small increment in E A , we obtain (14,15) where J A is the Jacobian matrix of A with respect to x and . Therefore, the first-order linearized form of the errors in Eq. (1) can be expressed as Under the L2-norm criterion, the objective function can be expressed as Equation (3) can be expressed as The unknown parameter vector ξ can be derived by iteratively solving Eq. (5), (14,15) and the parameters of the fitted edge can be subsequently obtained.

Structured total least L1-norm (STL1N) edge fitting model
In this section, the structured total least L1-norm edge fitting algorithm is derived on the basis of the structured total least norm method. (14,15) It is designed for the case when there is gross noise in the images.

Simulation experiment
A simulation experiment was designed and performed to verify the effectiveness of the proposed slanted edge methods based on STL1N and STL2N edge fitting. A blank image with a size of 40 × 40 pixels was first constructed. Then the position of the edge in each row was calculated using the linear equation y = b 0 + b 1 x with predetermined parameters. To verify the effectiveness of the proposed methods for different edge angles, we considered five slanted edge angles of 8, 15, 22.5, 30, and 37°. The edge angles and the corresponding values of b 0 and b 1 are shown in Table 1.
To obtain a degraded edge, the Fermi function was used to assign values to each line of the image: (20) = exp The Fermi function is commonly used to fit the initial ESF points and to obtain the ESF curve. In our simulation experiment, the Fermi function was used to degrade the edge. Using the Fermi function to fit the ESF can reduce the influence of noise. By designing a slanted edge line, one can derive the positions of points on the edge accordingly. Using the Fermi function to degrade the edge will not affect the positions of the designed edge, because parameter b can control the column of the edge in each row. Therefore, the error caused by degrading the boundary is avoided. Parameters a, d, and c were set as 180, 20, and 0.5, respectively, and b was the column of the subpixel where the line intersected each row of the original image. In this way, a degraded edge image with gray values of 200 in the bright area and 20 in the dark area was obtained. The simulated image is shown in Fig. 3.
Generally, actual remote sensing images have no such uniform gray values on both sides of an edge. Therefore, Gaussian white noise was added to the designed images. Three schemes were designed, in which the added noise had a mean value of 0 with variances of 0.001 × 255 × 255, 0.002 × 255 × 255, and 0.003 × 255 × 255. The simulation experiment was repeated 1000 times for each scheme to obtain statistically reliable results. Subsequently, the results of fitted edges and calculated MTF values were compared and analyzed. The designed and known edge angles were assumed as the true values, and the angles of the fitted straight edge lines obtained by LS, STL2N, and STL1N fitting were compared with the true values. The reason for choosing the edge angle as the evaluation criterion is that it is an important factor for fitted edges.
Furthermore, the fitted edges obtained by the LS, STL2N, and STL1N methods were used for measuring the MTFs of the images. The MTF value at the Nyquist frequency was regarded as the true value, which was calculated from the degraded image without noise. The absolute values of the errors were compared.
For the edge angle of 8°, the edge fitting results obtained by the LS, STL2N, and STL1N methods were first compared and analyzed. In each of the 1000 simulations, the method that generated the smallest absolute difference between the fitted edge angle and the true value was called the optimal method. The numbers of times the LS, STL1N, STLS1N methods were optimal among the 1000 simulations were counted and are shown in Table 2. In the table, numbers in bold represent the highest number of times the method was optimal in each scheme.
The following can be seen from Table 2. (1) When the variance of the added Gaussian white noise was relatively small (Scheme 1), the numbers of times the LS, STL2N, and STL1N methods were optimal were almost the same. (2) However, with increasing variance of the added Gaussian white noise (Schemes 2 and 3), the numbers of times the STL2N and STL1N methods were optimal became significantly higher than that of the LS method, with the STL1N more often optimal than the STL2N method. This indicates that the STL1N method is more robust than the STL2N method in a scheme with a larger amount of noise. The distributions of the absolute angle differences for different slanted edge angles will be further analyzed in Fig. 4.    Table 3 shows the mean fitted edge angles for five slanted edge angles of 8, 15, 22.5, 30, and 37°. Numbers in bold represent the results with the smallest difference between the fitted edge angle and the true value. Tables 4 and 5 respectively show the means and standard deviations of absolute differences between the fitted edge angle and the true value obtained by LS, STL2N, and STL1N fitting for the five angles. Numbers in bold represent the smallest value in schemes for a specific edge angle.
From the results presented in Tables 3-5, it can be seen that the means and standard deviations of absolute angle differences obtained by the STLS1N and STLS2N methods are smaller than those obtained by the LS method in all three schemes for the different slanted angles. Also, the STLS1N method is the most robust in Scheme 3. The results indicate that the proposed improved slanted edge methods based on the STLS1N and STLS2N methods are effective for different edge angles.  Additionally, the absolute angle differences from the true values obtained by the three methods were classified into the ranges of 0-0.01°, 0.01-0.05°, 0.05-0.1°, 0.1-0.15°, and 0.15-0.5°, 0.5-1°, and >1°. The distributions of the absolute angle differences classified into these ranges are shown in Fig. 4.
It can be seen from Fig. 4 that with increasing variance of the noise (Schemes 2 and 3), there are more times that the absolute angle difference is greater than 1° for the LS method than for the STL1N and STLS2N methods. Also, the mean absolute angle differences obtained by the STL1N method are all smaller than 1°. The fitted edges obtained by the three methods were further utilized to calculate the MTF values of the simulated images at the Nyquist frequency. Here, the MTF at the Nyquist and the frequency calculated from the image without Gaussian white noise was used as the true value. The experiment was repeated 1000 times. Tables 6 and 7 respectively show the means and standard deviations of absolute differences between the MTF values obtained by the three methods and the true values. Numbers in bold represent the smallest mean or standard deviation of the MTF differences in each scheme and case.
From the results presented in Tables 6 and 7, the following can be seen. (1) The absolute differences between the MTF values and true values obtained by the slanted edge method based on LS fitting are clearly larger than those for STLS1N and STLS2N. (2) The standard deviations of the absolute differences between the MTF values and true values obtained by the  (3) The absolute difference between the MTF value and the true value increases as the angle increases. This is because the same Fermi function is used to degrade the edge for the different slanted edge angles, and the noise will have a greater impact on the measured MTF of the image with a larger slanted angle.

Experiment on urban remote sensing images
In this section, actual remote sensing images were used to further verify the feasibility of the proposed methods. The data used were ZY-3 nadir-view images with a spatial resolution of 2.1 m. Two images of an urban area (images I and II) were used to measure the MTF value by the proposed methods and the LS-fitting-based method. The study areas are shown in Figs. 5 and 6.   In Fig. 5, the area on the building roof with a slanted edge and obvious uniform features on both sides of the edge was selected. Also the slanted edge was relatively clear and had little noise. In addition, part of the airport in Fig. 6 had a slanted edge suitable for MTF calculation. The slanted edge in Fig. 6 was less clear and contained more noise than that in Fig. 5. Figure 7 shows the fitted edges obtained by the slanted edge methods based on LS, STPL1N, and STL2N for images I and II. Figure 8 shows the ESFs obtained for the fitted edges using the above three methods for image II. Figure 9 shows the MTF values obtained by the above three methods.
From the results presented in Figs. 7-9, the following can be seen. (1) The fitted edges obtained by the three methods are almost the same for image I, which contained relatively small noise. However, in Fig. 7(b), the fitted edge obtained by LS fitting is clearly different from those obtained by the STL1N and STL2N methods, with obvious deviations from the edge points. (2) From Fig. 8, we can see that the points in the red frame are most concentrated for the edge fitted by the STL1N method, which illustrates that the ESF obtained by the STL1N method is more accurate than those obtained by the other two methods for image II.   The results proved the importance of the accuracy of edge fitting for MTF calculation and the effectiveness of the proposed methods.

Conclusions
Improved slanted edge methods of measuring the MTF based on structured total least L1-, L2-norm fitting for urban remote sensing images were proposed in this paper. The accuracy of the fitted edge has a relatively significant influence on the establishment of the ESF. The aim of the proposed methods is to improve the measurement of the MTF by using structured total least L1-, L2-norm edge fitting.
Experiments with simulated and actual images were designed and conducted to test the feasibility of the proposed method. In the simulation experiment, different slanted edge angles and Gaussian white noise with different variances were used. The results showed the following.
(1) When the variance of the added Gaussian white noise was relatively small, the differences between the fitted edge angles and the MTFs obtained by the LS, STL2N, and STL1N methods and the true values were almost the same. (2) However, with increasing variance of the added Gaussian white noise, the differences between the fitted edge angles and the MTFs obtained by the STL2N and STL1N methods and the true values became smaller than those obtained by the LS method. (3) The slanted edge method based on STL1N is more robust than the other two methods in the case of Gaussian white noise with larger variance in terms of the fitted edge angle and the difference in the MTF from the corresponding true value.
Furthermore, two actual remote sensing images were used to test the proposed method. The results indicate the following. (1) The fitted edges and MTFs obtained by the three methods were almost the same for image I, which contained relatively little noise. (2) For image II, which contained more noise, the edge obtained by LS fitting clearly deviated further from the edge points than those obtained by the proposed methods. Also, the MTFs obtained by the proposed methods were larger than that obtained by the LS method. The impact of noise on the establishment of the ESF needs to be further studied in future work.