Application of Differential Global Positioning System and Path Planning to Robot Outdoor Patrol

In this paper, we present a robot outdoor patrol that applies a differential global positioning system (DGPS), an encoder, a web camera, and a laser sensor with an improved A* algorithm to achieve fixed route patrol. Obstacle avoidance by smooth pathfinding in complicated unknown environments is also considered. While in fixed route patrolling, the DGPS provides locations of the wheeled mobile robot (WMR) and the corners of buildings. By image processing and the wheel’s encoder, the orientation of the WMR can be calibrated if the web camera has deviation angles to the target building. Along the patrol path, people or other obstacles may block the WMR while patrolling, so the laser sensor is used to provide the distances and edge values of such obstacles. This information is then used to improve the A* algorithm to achieve the shortest and smoothest path. A fuzzy controller is used to assist the WMR with obstacle avoidance in unknown environments. After finishing a patrol, the WMR system will send a message to a security guard to report whether the patrol region is in normal condition. If the camera detects an unidentified person, the robot will send the location of the person to the control center and ask for help.


Introduction
Automatic guidance vehicle (AGV) technologies and image processing for target tracking have been developed over many years. In an AGV design, a collision-free path planning method for the navigation of mobile robots is required. Many methods have been developed to solve the path planning problem, among which the Dijkstra algorithm, A* algorithm, and B-spline are the most used. (1)(2)(3) In real-time obstacle avoidance control, fuzzy controllers are usually applied to control wheeled mobile robots (WMRs). (4) Target tracking for WMR control uses color image recognition, (5) human contour data recognition, (6) and distances between the object and the camera. (7) In this work, we integrate the differential global positioning system (DGPS), (8) laser sensor, and image processing to WMR control. By integrating the A* algorithm and laser sensor, the WMR can find the shortest path and avoid obstacles. However, using only the A* algorithm for path planning may cause the WMR to have to turn sharply at the corners of some paths. To smooth the paths, an improved A* algorithm, which combines curvature and rotational trajectory equations, (9) was proposed. The camera is then used to search and track targets. After finishing a patrol, the robot will send a message to a supervisor via Wi-Fi.
In this study, coordinate locations of the WMR (or start point) and fixed target points (or goal) are given by the DGPS. The DGPS and wheel's encoder are used to adjust the path of the WMR in complex environments. When there are obstacles in the direction of the goal, the laser sensor is used to search both sides of obstacles to find a new temporary local goal. Path planning is performed by the improved A* algorithm, and the laser sensor is used to calculate the shortest smooth path. The WMR moves along the shortest and smoothest path, and is controlled by the use of a fuzzy controller with obstacle avoidance capability. When the WMR reaches a turning point, it searches for the color of the desired target and fixes the orientation angles, then moves forward to the desired target. In image processing, the hue, saturation, and luminance (HSL) color space is utilized for color segmentation. Then, median filtering is used to remove noises and capture the color of the target. In addition to color tracking, human contour recognition will be used when the camera detects specific colors.

System Description
The hardware structure includes the WMR, laser measurement sensor, power system, DC motors, digital signal processing (DSP) development board, Microsoft LifeCam studio webcam, and GB200 DGPS receiver, as shown in Fig. 1. A WMR called "iHomer" is used in the entity test. (10) In order to simplify the design of the controllers, the nonlinearity of the WMR is given below. The web camera catches surrounding images and the distance between the WMR and the target can be obtained by image processing techniques.

WMR system
The experimental validation of the proposed control scheme has been performed on the WMR. The WMR is 48 cm long, 45.5 cm wide, and 25.5 cm high. iHomer is driven by two wheels. Each wheel has a DC motor that provides a torque of 7.2 kg.m and a maximum speed of 1.6 m/s for the robot. There are two extra small casters attached at the front and rear of the WMR. The WMR has 6 ultrasonic sensors and 25 IR sensors around, which are used for obstacle detection, obstacle avoidance, feature recognition, location detection, and so on.
The WMR uses the Cartesian coordinate system as shown in Fig. 2. The movement of the  WMR can be obtained from the center and angular speeds of the robot. The WMR can turn to a desired direction and move to a designated position. The left wheel speed w l and right wheel speed w r are the inputs of the motion equation, which is shown in Eq. (1). They are used to obtain the center speed v and angular speed w of the WMR. The distance between the left wheel and right wheel is d. The radius of the wheel is r. (11) r r 2 2 Change rates in the X-Y positions and the moving direction angle of the WMR are where θ is the angle of the direction of movement. The center coordinates of the WMR are (x, y). At the time interval i + 1, X-Y positions and the angle of the direction of movement of the WMR can be obtained from Eq. (3).
The sampled time constant is Δt. i is the time index.

GB200 DGPS receiver
The GB200 DGPS system receives satellite information from the GPS, GLONASS, and Beidou. To obtain precise positioning coordinates, this system decodes and computes the satellite data, and then transmits the data to the server via Wi-Fi. The GB200 integrates the associated hardware in a black box. The device is powered by a rechargeable lithium polymer battery. The satellite data can be stored on an SD card inside the black box. Figure 3 shows a schematic view of the system devices. Figure 4 shows the global navigation satellite system (GNSS) antenna and Skytraq single-frequency receiver.
The differential results on the GB200 receiver are given by Eqs. (4)-(7) Equation (5) expresses the single difference between receivers. Equation (6) expresses the single difference between satellites. At the time t, the error from the satellite clock, dt, and the receiver clock error, dT, could be canceled as shown in Eq. (7). At the same time, because the distance between the base and robot is short (less than 100 km), the localized atmospheric errors can also be reduced, which allows a more accurate location to be obtained. The GB200 transmits the location and direction from the receiver to the computer via USB and RS232 connections.
In Eqs. (4)- (7), ρ is the pseudo range between a satellite and the receiver, D is the real distance between a satellite and the receiver, ∇Δ is the double difference (receiver-satellite), d ion is the ionosphere correction, d trop is the tropospheric correction, and ε is the measurement noise. A test of the DGPS is shown in Fig. 5. The receiver was moved around a square path (24 × 24 cm 2 ) at a speed of 20 cm/s. The average dynamic error is less than 5 cm and the static error is less than 3 cm.

Control Scheme
A control scheme is proposed for the WMR to perform shortest path planning and obstacle avoidance control in outdoor environments. The control scheme is divided into several parts. It includes path planning, obstacle avoidance, object positioning, and target searching. The control sequence is shown in Fig. 6.

Image processing
In the first stage of image processing, we capture the desired color of the target by the webcam. The image captured by the webcam is in the RGB color space, so the image is  transformed to the HSL color space. The desired color is obtained by color segmentation. (12) Then, the color of the target is binarized and filtered to eliminate noise in image processing. The RGB color is easily influenced by light sources. In this work, the HSL color space is used to reduce the impact of light sources. The HSL values are obtained from the RGB values. After an RGB image is transformed to the HSL color space, we use color segmentation in the HSL color space. In order to extract the desired color, thresholds of h, s, and l are determined by the color spectrum of the image. The webcam can detect the desired color. After color segmentation, finding the desired color of the target, and the binarization of the image, there was some speckle noise. A test is shown in Fig. 7.
We then use a median filter to filter out the noise from the signals. The median filter is very useful for removing speckle and salt-and-pepper noise. An n × n mask is set first with an odd number n. Then, the mask is sorted and a median value is found to replace the old value of the image. Median filter operation is shown in Fig. 8 and the white framed result is shown in Fig. 9. Finally, a skeleton algorithm (13) is applied to check whether the object is human or not.

Improved A* algorithm
The A* algorithm is derived from Dijkstra's algorithm and the best-first search method. (1) The strategy of the A* algorithm is based on graph traversal. The Heuristic function is implemented in the cost estimation of a path. The path is formed by following the smallest cost from a node to the destination. The function is expressed by Eq. (8). Starting with trying to find the minimum cost path, we use the Euclidean distance heuristic intuitively, which is expressed by Eq. (9). The cost function of p(j) is also calculated by the Euclidean distance [Eq. (10)].
The real cost from the starting node to node j is p( j) (i.e., the cost of finding the optimal path). The minimal estimated cost from node j to the destination node is q*( j). (x j ,y j ) is the position of the node j, (x goal ,y goal ) is the location of the goal point, and (x c ,y c ) is the position of the initial point. When the right wheel velocity of the WMR is not equal to the left, the WMR will move in a curve, the center of which is called the instantaneous center for curvature (ICC). R is the radius of the curved path and can be obtained by the left wheel velocity V L and right wheel velocity V R . D is the distance between the two wheels.
In the improved A* algorithm, the straight-line sharp turns that would be generated by the conventional A* algorithm become curved paths. (9) A curved path consists of three parts with respect to rotation: before, during, and after. P I = [X I Y I θ I ] t is the initial position where the WMR reduces speed for rotation. The WMR starts rotating P RI at and finishes rotating at P RF . The final position of the curve tracking is P F = [X F Y F θ F ] t . In Eq. (12), Δθ is the angle between the initial and final heading directions of the WMR when it rotates from P RI to P RF , θ I is the heading angle at the initial position, θ F is the heading angle at the final position, and L' is the distance from the ICC to the original straight path. The distance between the ICC and the sharp turn node j is the sum of R and L C , where L C is defined in Eq. (13). L is the distance from the initial point to the sharp turn node j, as shown in Eq. (14).
When the robot approaches the initial position P I , it reduces speed and prepares to rotate. At P RI , the robot moves along the predefined curve. While tracking the curve, the WMR continuously calculates the position and direction. Finally, when the WMR reaches the position P RF, the WMR stops rotating and moves to the designated position P F . A simulation of the path planning is shown in Fig. 10.

Fuzzy control
After obtaining the optimal patrol path, obstacle avoidance needs to be considered while the WMR is patrolling. In collision avoidance control, we need to keep the WMR moving safely at a constant speed along the optimal path. We set a safe distance range for security. The safe distance between the WMR and an obstacle is 30 cm. There is a laser measurement sensor positioned on the WMR. Fuzzy rules are applied to the controller design. There are three inputs to the fuzzy controller, namely, SR, SF, and SL, which represent 45, 90, and 135° of the laser scanning angle on the WMR, and the measured distances are dr, df, and dl, respectively. Fuzzy sets are very far (VF), far (F), medium (M), near (N), and very near (VN). The output of the fuzzy controller is the turning angle (TA) and the fuzzy sets are TR3, TR2, TR1, TZ, TL1, TL2, and TL3. Membership functions of the fuzzy sets are given in Fig. 11 and the fuzzy rules are shown as follows.

Experimental Results
By using LabVIEW human-machine interaction, we can successfully obtain information from the GNSS and laser sensor (Figs. 12 and 13) and calculate the shortest and smoothest path for goal tracking (Fig. 14). After reaching the goal, the WMR searches for the color of the target and moves forward to the target (Figs. 15 and 16). Different colors can be set beforehand, and each color is then checked by the skeleton algorithm to determine whether the object is human. Before outdoor patrolling, we need to train the WMR to move smoothly by using the improved A* path in an indoor environment, as shown in Fig. 17.       (c) (d) Figure 18 shows the test field. Figures 19 and 20 show the outdoor building patrol using fuzzy control and the improved A* algorithm. Unidentified person image and location are reported to the control center by Wi-Fi.

Conclusions
In this study, a WMR is applied for outdoor security service. A DGPS is used to locate the position of the WMR and provide designated locations. The position accuracy of the DGPS is much higher than that of a GPS; the average error of DGPS position is about 5 cm. Image processing is used for detecting and tracking a specific target. The location and image of undefined targets can be sent to the control center. The locations of the WMR, building, obstacles, reference points, and desired target can be obtained from the DGPS, laser sensor, and webcam. The proposed WMR can patrol complex environments; searching and moving can be accomplished without whole-map information. In addition, we use an improved A* algorithm to plan a smooth path to the goal position and utilize fuzzy rules to allow the WMR to avoid obstacles. Real-time PC-based control of the WMR is performed successfully.